app_elevenlabs_convai

KO | EN

gitlite — search

#javascript #hacktoberfest #ai #golang #go #android #machine-learning #rust #macos #java #awesome-list #security #mcp #openai

app_elevenlabs_convai

(★ 13)

No description available.

app_elevenlabs_convai Latest Version Download

Download Latest Version (.zip)

// repository documentation

Was this content helpful?

★ 0 (0 ratings)

# Asterisk ElevenLabs Conversational AI Integration

A production-ready Asterisk module that integrates ElevenLabs voice agents with Asterisk channels via WebSocket connections. This module uses Asterisk's framehook API to capture and inject audio in real-time, enabling bidirectional voice conversations between callers and AI agents.

The module handles WebSocket communication, audio forking, buffering, and conversation management automatically. You configure your voice agents and add a single dialplan application - the rest works transparently.

## Key Features

**Core Functionality:**
- **Framehook Integration**: Deep integration with Asterisk's framehook API for real-time audio capture and injection
- **WebSocket Communication**: Handles ElevenLabs voice AI protocol including audio streaming, VAD scores, and interruption events
- **Audio Processing**: Configurable buffering, base64 encoding, and precise 20ms playback timing
- **Interruption Handling**: Event-based audio queue filtering that responds immediately to user interruptions
- **Multi-Agent Support**: Configure multiple voice agents with different personalities and settings
- **Error Recovery**: Automatic connection retry, health monitoring, and graceful failure handling

**Technical Implementation:**
- **Framehook API**: Uses `ast_framehook_attach()` for real-time audio processing on read/write events
- **Threading**: Separate threads for WebSocket communication and audio playback with proper synchronization
- **Memory Management**: Efficient buffer allocation and cleanup with leak prevention
- **Protocol Compliance**: Full implementation of ElevenLabs WebSocket protocol with proper message handling
- **Configuration**: Hot-reloadable config file supporting multiple agent profiles
- **CLI Commands**: Debug control and status monitoring via Asterisk CLI

**Production Features:**
- **Automatic Cleanup**: Sessions terminate cleanly when WebSocket closes, with proper resource deallocation
- **Thread Safety**: Mutex-protected operations across all shared data structures
- **Scalability**: Supports multiple concurrent conversations with minimal resource overhead
- **Debugging**: Comprehensive logging with configurable verbosity levels

## Getting Started: Transform Your Asterisk System

Ready to bring AI conversations to your phone system? The installation process is straightforward, and you'll be amazed how quickly you can have intelligent agents handling calls.

### Prerequisites and Dependencies

#### Ubuntu/Debian
```bash
# Update package manager
sudo apt-get update

# Install build dependencies
sudo apt-get install -y \
    build-essential \
    libcurl4-openssl-dev \
    libwebsockets-dev \
    libcjson-dev \
    libssl-dev \
    pkg-config \
    asterisk-dev

# For Ubuntu 20.04+ you may need:
sudo apt-get install -y libwebsockets16
```

#### CentOS 7
```bash
# Enable EPEL repository
sudo yum install -y epel-release

# Install development tools
sudo yum groupinstall -y "Development Tools"

# Install dependencies
sudo yum install -y \
    libcurl-devel \
    openssl-devel \
    pkgconfig \
    asterisk-devel

# Install libwebsockets (may need to build from source on CentOS 7)
sudo yum install -y libwebsockets-devel || {
    echo "Building libwebsockets from source..."
    cd /tmp
    git clone https://github.com/warmcat/libwebsockets.git
    cd libwebsockets
    mkdir build && cd build
    cmake .. -DCMAKE_INSTALL_PREFIX=/usr
    make && sudo make install
}

# Install cJSON
sudo yum install -y libcjson-devel || {
    echo "Building cJSON from source..."
    cd /tmp
    git clone https://github.com/DaveGamble/cJSON.git
    cd cJSON
    make && sudo make install
}
```

### Building Your AI-Powered Asterisk Module

The build process is designed for simplicity. Whether you're a seasoned Asterisk administrator or new to telephony development, these steps will get your AI integration up and running quickly.

```bash
# Clone the repository and navigate to the module
git clone <repository-url>
cd asterisk_elevenlabs/app_elevenlabs_convai/

# Compile the Asterisk ElevenLabs integration module
make clean
make

# Deploy to your Asterisk modules directory
sudo cp app_elevenlabs_convai.so /usr/lib/asterisk/modules/
# OR for 64-bit systems:
# sudo cp app_elevenlabs_convai.so /usr/lib64/asterisk/modules/

# Ensure proper permissions for Asterisk service
sudo chown asterisk:asterisk /usr/lib/asterisk/modules/app_elevenlabs_convai.so
sudo chmod 755 /usr/lib/asterisk/modules/app_elevenlabs_convai.so
```

**Pro Tip**: If you're developing or customizing the module, use `make DEBUG=1` for additional debugging symbols that make troubleshooting easier.

### Verification: Confirming Your Installation Success

Once you've built and installed the module, it's time to verify that your Asterisk system recognizes the new ElevenLabs integration capability.

```bash
# Verify the module is recognized by Asterisk
sudo asterisk -rx "module show like elevenlabs"

# If the module isn't loaded yet, activate it
sudo asterisk -rx "module load app_elevenlabs_convai"

# Confirm the module loaded successfully and check for any initialization messages
sudo asterisk -rx "module show app_elevenlabs_convai"
```

**Success Indicators**: You should see the module listed as "Running" in the output. If you encounter any issues, check the Asterisk logs for detailed error messages that will guide your troubleshooting efforts.

## Configuring Your AI Agents: From Setup to Success

Configuration is where the magic happens. You'll define your AI personalities, set up audio parameters, and fine-tune the conversation experience. The process is intuitive, but the possibilities are endless.

### 1. Setting Up Your Configuration Foundation

```bash
# Copy example configuration
sudo cp elevenlabs.conf.example /etc/asterisk/elevenlabs.conf
sudo chown asterisk:asterisk /etc/asterisk/elevenlabs.conf
sudo chmod 640 /etc/asterisk/elevenlabs.conf
```

### 2. Crafting Your Voice Agent Personalities

This is where your vision comes to life. Each voice agent configuration represents a distinct AI personality that can handle different types of voice interactions. Think of it as assembling your intelligent voice AI team.

Edit `/etc/asterisk/elevenlabs.conf`:

```ini
[general]
# Global settings for your Asterisk ElevenLabs integration
debug=false

[sales_agent]
# Your charismatic sales voice agent that knows when to listen and when to pitch
elevenlabs_api_key=sk-your-api-key-here
elevenlabs_get_signed_url_endpoint=https://api.elevenlabs.io/v1/convai/conversation/get-signed-url
elevenlabs_agent_id=agent_0301k2f1kzshe85t9w74t66vz1km
elevenlabs_send_caller_number=true  # Share caller ID for personalized greetings
elevenlabs_audio_frames_buffer_ms=240  # Optimized for responsive sales conversations
debug=false
max_connection_attempts=3
connection_timeout_seconds=10

[support_agent]
# Your patient, helpful support voice agent that excels at problem-solving
elevenlabs_api_key=sk-your-api-key-here
elevenlabs_get_signed_url_endpoint=https://api.elevenlabs.io/v1/convai/conversation/get-signed-url
elevenlabs_agent_id=agent_another_agent_id_here
elevenlabs_send_caller_number=false  # Privacy-focused support interactions
elevenlabs_audio_frames_buffer_ms=300  # Slightly longer buffer for detailed explanations
debug=true  # Enable detailed logging for support quality monitoring
max_connection_attempts=5  # Extra resilience for critical support calls
```

**Configuration Philosophy**: Each parameter shapes the conversation experience. Shorter audio buffers create snappier interactions perfect for sales, while longer buffers provide stability for complex support scenarios.

### 3. Integrating Voice AI into Your Asterisk Dialplan

The dialplan integration is beautifully simple, yet incredibly powerful. With just one line, you can transform any extension into a voice AI-powered conversation endpoint. The module's framehook architecture handles all the complex audio routing automatically.

Add to your dialplan (`/etc/asterisk/extensions.conf`):

```ini
[default]
; Elegant simplicity - your voice AI sales agent handles everything
exten => 100,1,Answer()
exten => 100,2,ElevenLabsConversationalAI(sales_agent)
; The conversation flows naturally and ends when appropriate - no manual intervention needed

; Support line with optional timeout for testing
exten => 200,1,Answer()
exten => 200,2,ElevenLabsConversationalAI(support_agent)
exten => 200,3,Wait(3600)  ; Safety net: maximum 1-hour conversation limit
exten => 200,4,Hangup()

; Advanced routing example - different voice agents based on caller ID
exten => 300,1,Answer()
exten => 300,2,GotoIf($["${CALLERID(num)}" = "VIP_CUSTOMER"]?vip:standard)
exten => 300,3(standard),ElevenLabsConversationalAI(sales_agent)
exten => 300,4,Hangup()
exten => 300,5(vip),ElevenLabsConversationalAI(vip_support_agent)
exten => 300,6,Hangup()
```

**The Beauty of Framehook Integration**: Unlike traditional IVR systems that require complex audio routing, this module uses Asterisk's native framehook system to seamlessly fork audio in both directions. This means crystal-clear voice agent conversations with minimal latency.

### 4. Load and Test

```bash
# Reload configuration
sudo asterisk -rx "module reload app_elevenlabs_convai"

# Reload dialplan
sudo asterisk -rx "dialplan reload"

# Enable debug for testing
sudo asterisk -rx "elevenlabs debug on"

# Test the setup
# Make a call to extension 100 or 200
```

## Understanding the Technical Architecture

### The Journey of a Voice AI-Powered Call

Ever wondered what happens behind the scenes when someone calls your voice AI-enabled number? The process is a sophisticated orchestration of Asterisk components, WebSocket protocols, and real-time voice processing. Let's walk through this fascinating journey of how voice agents come to life.

1. **The Spark**: When someone dials your extension, Asterisk executes `ElevenLabsConversationalAI(agent_name)` and the voice agent magic begins
2. **Voice Agent Loading**: Your carefully crafted voice agent configuration springs into action, defining personality and conversational behavior
3. **Secure Handshake**: A secure HTTP request to ElevenLabs fetches a signed WebSocket URL – your golden ticket to voice AI conversations
4. **Connection Establishment**: A rock-solid WebSocket connection forms the secure bridge between your Asterisk system and ElevenLabs' voice AI infrastructure
5. **Voice Processing Activation**: Here's where the framehook brilliance shines – bidirectional voice processing begins seamlessly:
   - Every word from your caller gets captured, buffered intelligently, and transmitted as pristine base64 audio chunks to the voice agent
   - Voice AI responses flow back through optimized queues and get injected into the call with microsecond precision (20ms frame timing)
6. **Intelligent Voice Agent Management**: 
   - Voice Activity Detection continuously monitors conversation dynamics between caller and voice agent
   - Interruption events trigger sophisticated audio queue filtering – when someone interrupts the voice agent, the system responds instantly
   - Event-based voice processing ensures natural conversation flow between human and voice AI
7. **Background Orchestration**: Multiple threads work in harmony – framehooks capture voice data, WebSocket threads manage voice agent communication, and playback threads deliver voice AI responses
8. **Graceful Conclusions**: Conversations end naturally when the voice agent determines completion, triggering automatic call termination
9. **Perfect Cleanup**: Every resource, buffer, and connection gets cleaned up automatically – no memory leaks, no lingering voice processing threads

### Core Components: The Engine Room

#### WebSocket Mastery and Real-time Voice Communication
The heart of this integration beats through sophisticated WebSocket management that makes real-time voice agent conversations possible. This isn't just about sending data – it's about creating a reliable, intelligent voice communication channel.

- **Industrial-Strength WebSocket Client**: Built on libwebsockets, providing enterprise-grade reliability and security
- **Protocol Intelligence**: Seamlessly handles the complete ElevenLabs voice AI protocol – ping/pong heartbeats, voice message routing, voice activity detection scores, and intelligent interruption event management
- **Bulletproof Connection Management**: Automatic error recovery, connection health monitoring, and resource cleanup that keeps voice agent conversations flowing even when networks hiccup
- **Event Processing Excellence**: Comprehensive logging and monitoring of voice agent conversation dynamics, VAD scores, and interruption patterns for optimization insights

#### Advanced Audio Processing and Asterisk Framehook Integration
This is where the technical magic happens. The module doesn't just connect to Asterisk – it becomes part of Asterisk's audio processing pipeline through sophisticated framehook integration.

- **Native Asterisk Framehook Integration**: Deep integration with Asterisk's framehook API provides zero-latency voice capture and injection directly from the channel processing pipeline
- **Intelligent Voice Processing Management**: Sophisticated bidirectional voice processing ensures perfect synchronization between caller voice and voice AI responses without audio conflicts
- **Smart Buffer Orchestration**: Configurable voice buffering (200-500ms range) optimizes for your specific voice agent use case – snappy responses for sales agents, stable buffers for support agents
- **Microsecond-Precise Voice Playback**: Threading architecture delivers voice agent audio with exact 20ms frame timing, complete with intelligent interruption handling
- **Secure Voice Encoding**: OpenSSL-powered base64 encoding ensures voice data travels securely over WebSocket connections

#### Interruption Handling
- **Immediate Response**: Interruption flag stops current playback instantly
- **Smart Queue Filtering**: Removes audio chunks with event_id ≤ interruption event_id
- **Thread Synchronization**: Mutex-protected operations prevent race conditions
- **Conversation Flow**: Newer audio (after interruption) continues playing naturally

#### Configuration System
- **Multi-Agent Support**: Configure multiple ConvAI agents
- **Hot Reload**: Update configuration without restarting Asterisk
- **Validation**: Comprehensive parameter validation and error reporting

## CLI Commands

```bash
# Debug control
asterisk -rx "elevenlabs debug on"   # Enable debug logging
asterisk -rx "elevenlabs debug off"  # Disable debug logging

# Configuration management
asterisk -rx "elevenlabs reload"     # Reload configuration

# Status monitoring
asterisk -rx "elevenlabs show agents"  # Show configured agents

# Module management
asterisk -rx "module show like elevenlabs"  # Check module status
asterisk -rx "module load app_elevenlabs_convai"    # Load module
asterisk -rx "module unload app_elevenlabs_convai"  # Unload module
```

## Configuration Reference

### Agent Configuration Parameters

| Parameter | Required | Default | Description |
|-----------|----------|---------|-------------|
| `elevenlabs_api_key` | Yes | - | ElevenLabs API key from account dashboard |
| `elevenlabs_get_signed_url_endpoint` | Yes | - | API endpoint for WebSocket signed URLs |
| `elevenlabs_agent_id` | Yes | - | Unique ConvAI agent identifier |
| `elevenlabs_send_caller_number` | No | false | Send caller ID to agent as dynamic variable |
| `elevenlabs_audio_frames_buffer_ms` | No | 240 | Audio buffer duration in milliseconds |

### Buffer Duration Guidelines
- **200ms**: Faster response, may cause audio fragmentation
- **240ms**: Recommended default for most use cases  
- **300ms**: More stable for poor network conditions
- **500ms**: Maximum recommended for very unstable connections

## Logging and Monitoring

### VAD Score Logging
When debug is enabled, you'll see detailed VAD information:
```
=== VAD SCORE MESSAGE ===
VAD Score: 0.847
Timestamp: 2024-08-17T16:07:45.123Z
HIGH voice activity detected (score=0.847)
=== END VAD SCORE ===
```

### Interruption Event Logging
Detailed interruption processing information:
```
=== INTERRUPTION MESSAGE ===
Event ID: 46
User interrupted AI speech - stopping playback
Processing interruption for event_id=46 - clearing older audio
Removing audio chunk with event_id=42 (≤46)
Removing audio chunk with event_id=45 (≤46)
Interruption processing complete: removed 2 audio chunks with event_id ≤ 46
Remaining audio chunks in queue: 1
=== END INTERRUPTION ===
```

## Troubleshooting

### Common Issues

#### Module Loading Fails
```bash
# Check dependencies
ldd /usr/lib/asterisk/modules/app_elevenlabs_convai.so

# Check Asterisk logs
tail -f /var/log/asterisk/messages | grep -i elevenlabs

# Verify module file
ls -la /usr/lib/asterisk/modules/app_elevenlabs_convai.so
```

#### WebSocket Connection Issues
```bash
# Enable debug logging
asterisk -rx "elevenlabs debug on"

# Check configuration
asterisk -rx "elevenlabs show agents"

# Test API connectivity
curl -H "xi-api-key: YOUR_API_KEY" \
  "https://api.elevenlabs.io/v1/convai/conversation/get-signed-url?agent_id=YOUR_AGENT_ID"
```

#### Audio Quality Issues
- Increase buffer duration: `elevenlabs_audio_frames_buffer_ms=300`
- Check network latency to ElevenLabs servers
- Monitor Asterisk logs for frame processing warnings
- Ensure proper μ-law audio format support

#### Crash Issues (Resolved)
The module includes comprehensive crash protection:
- Automatic session cleanup on WebSocket disconnection
- Thread-safe resource management
- Proper channel reference counting
- Graceful handling of interrupted connections

### Debug Logging

Enable comprehensive logging:
```bash
# Enable module debug
asterisk -rx "elevenlabs debug on"

# Enable Asterisk verbose logging
asterisk -rx "core set verbose 4"

# Monitor logs in real-time
tail -f /var/log/asterisk/messages | grep -E "(elevenlabs|ElevenLabs)"

# Filter for specific events
tail -f /var/log/asterisk/messages | grep -E "(VAD SCORE|INTERRUPTION|WebSocket)"
```

## Performance That Impresses: Built for Production Scale

When you're running a business on conversations, performance isn't just nice to have – it's everything. This module has been engineered from the ground up for production environments where reliability and speed matter.

**Real-World Performance Metrics:**
- **Lightning-Fast Voice Agent Response Times**: Sub-50ms voice processing latency ensures voice agent conversations feel natural and immediate
- **Concurrent Voice Agent Mastery**: Effortlessly handles multiple simultaneous voice AI conversations without performance degradation  
- **Memory Efficiency Excellence**: Intelligent buffer management and automatic cleanup prevent memory bloat even during extended voice agent operations
- **CPU Optimization**: Minimal system overhead thanks to optimized voice processing and efficient framehook integration
- **Bulletproof Reliability**: Comprehensive error handling, automatic connection recovery, and crash resistance keep your voice agents running 24/7

**Why This Matters**: In the world of voice AI, every millisecond counts. Delays kill the natural flow of voice conversations. This module's performance characteristics ensure your voice agents respond as quickly as human agents – sometimes faster.

## API Requirements

### ElevenLabs Account Setup
1. Create account at [elevenlabs.io](https://elevenlabs.io)
2. Create ConvAI agents in the dashboard
3. Generate API key with ConvAI access
4. Note agent IDs for configuration

### Network Requirements
- Outbound HTTPS (443) access to `api.elevenlabs.io`
- Outbound WSS (443) access to ElevenLabs WebSocket servers
- Stable internet connection (recommended: <100ms latency)

## Development

### Building from Source
```bash
# Development build with debug info
make clean
make DEBUG=1

# Hot-reload during development
make && sudo cp app_elevenlabs_convai.so /usr/lib/asterisk/modules/ && asterisk -rx "module reload app_elevenlabs_convai"
```

### Testing
```bash
# Test build
make test

# Memory leak testing
valgrind --tool=memcheck --leak-check=full asterisk -vvv
```

## Getting Help: You're Not Alone

Implementing voice AI and voice agents in telephony can feel overwhelming, but you don't have to figure it out alone. This section provides the resources and troubleshooting guidance you need to succeed with your voice agent deployment.

### Systematic Troubleshooting Approach
When things don't work as expected (and sometimes they won't), follow this proven troubleshooting methodology:

1. **Start with the Logs**: Check Asterisk logs at `/var/log/asterisk/messages` – they tell the story of what's happening
2. **Enable Detailed Debugging**: Use `elevenlabs debug on` to get granular insights into the voice processing, audio forking and WebSocket communication
3. **Verify Your Foundation**: Confirm API credentials are correct and your network can reach ElevenLabs voice AI endpoints
4. **Test Incrementally**: Begin with the simplest possible dialplan configuration and add complexity gradually
5. **Confirm Module Status**: Use `module show like elevenlabs` to ensure the Asterisk integration loaded properly

**Pro Tip**: Most voice agent issues stem from network connectivity or API configuration. Get those right first, and your voice AI integration usually falls into place.

## Open Source Commitment

**GNU General Public License Version 2 (GPL-2.0)**

This Asterisk ElevenLabs integration module is free software that you can redistribute and modify under the terms of the GNU General Public License. We believe in the power of open source to advance conversational AI in telephony.

## Standing on the Shoulders of Giants

This project exists because of the incredible work done by many open source communities and companies:

- **Asterisk Telephony Framework**: The rock-solid foundation that powers millions of phone systems worldwide, with its sophisticated framehook API that makes real-time audio processing possible
- **ElevenLabs Voice AI Platform**: The cutting-edge voice agent technology that brings natural, intelligent voice interactions to telephony
- **Industry-Standard Libraries**: Built with battle-tested components – libcurl for reliable HTTP communications, libwebsockets for robust WebSocket connections, cJSON for efficient data parsing, and OpenSSL for secure voice communications
- **Community Contributions**: The collective wisdom of developers, system administrators, and telephony professionals who understand that great voice AI software comes from collaboration

This voice agent integration represents the convergence of traditional telephony expertise and modern voice AI innovation – proof that the best solutions come from combining proven technologies in innovative ways to create intelligent voice experiences.

app_elevenlabs_convai

app_elevenlabs_convai Latest Version Download

Today's picks

Was this content helpful?

Recent Feedback

app_elevenlabs_convai

app_elevenlabs_convai Latest Version Download

Today's picks

Was this content helpful?

Recent Feedback

Download README