Skip to navigation

Pipecat

View as Markdown

Connect AgentDuet to a Pipecat pipeline. Use the pipecat-agentduet transport package.

What this integration provides

AgentDuetTransport transfers audio between an AgentDuet call and a Pipecat pipeline. It also sends call events to the pipeline.

Your application opens a connection to AgentDuet and keeps it open. AgentDuet sends incoming calls through this connection. You do not need a public webhook endpoint or an ngrok tunnel for call delivery.

Key Features

  • No Webhook Hosting: Public URLs, ngrok tunnels, and reverse proxies are not necessary.
  • PCM audio: Uses mono 16-bit PCM at 8 kHz, 16 kHz, or 24 kHz. Manual format conversion is not necessary.
  • Call events: Answers the call when the pipeline starts. Releases resources when the pipeline ends. Cancels tasks when the caller disconnects.
  • Built-in Barge-in: Works with Pipecat’s VAD and turn processors to clear audio on interruption.

Prerequisites

pip install "agentduet==1.1.0b10" pipecat-agentduet "pipecat-ai[silero,deepgram,google]" python-dotenv

Step 1: Create a project folder

mkdir agentduet-pipecat-bot && cd $_
python3.12 -m venv .venv && source .venv/bin/activate
pip install "agentduet==1.1.0b10" pipecat-agentduet "pipecat-ai[silero,deepgram,google]" python-dotenv

Step 2: Configure .env

Create a .env file in your project directory:

cat > .env << 'EOF'
AGENTDUET_API_KEY=your-agentduet-api-key
AGENTDUET_CONNECTOR_UUID=your-connector-uuid
DEEPGRAM_API_KEY=your-deepgram-api-key
GOOGLE_API_KEY=your-google-api-key
EOF

Step 3: Write voice_bot.py

Create voice_bot.py next to .env:

import asyncio
import os
from dotenv import load_dotenv
from agentduet import (
IncomingCallNotification,
SessionManager,
SessionManagerConfig,
new_session_id,
)
from pipecat.audio.vad.silero import SileroVADAnalyzer
from pipecat.frames.frames import LLMRunFrame
from pipecat.pipeline.pipeline import Pipeline
from pipecat.pipeline.worker import PipelineWorker
from pipecat.processors.aggregators.llm_context import LLMContext
from pipecat.processors.aggregators.llm_response_universal import LLMContextAggregatorPair
from pipecat.processors.audio.vad_processor import VADProcessor
from pipecat.services.deepgram.stt import DeepgramSTTService
from pipecat.services.deepgram.tts import DeepgramTTSService
from pipecat.services.google.llm import GoogleLLMService
from pipecat.turns.user_turn_processor import UserTurnProcessor
from pipecat.workers.runner import WorkerRunner
from pipecat_agentduet import AgentDuetTransport
load_dotenv()
async def run_call(sm: SessionManager, noti: IncomingCallNotification):
# Open an AgentDuet session for the incoming call
session = await sm.open_session(
noti.session_id or new_session_id(), noti.subscriber_id
)
call = await session.process_call(noti)
# Wrap the call with the Pipecat transport
transport = AgentDuetTransport(call)
# Initialize conversational context
context = LLMContext([
{
"role": "system",
"content": (
"You are a friendly voice assistant on a telephone call. "
"Keep responses concise, natural, and conversational."
),
}
])
aggregators = LLMContextAggregatorPair(context)
# Build the Pipecat pipeline
pipeline = Pipeline([
transport.input(),
VADProcessor(vad_analyzer=SileroVADAnalyzer()),
UserTurnProcessor(),
DeepgramSTTService(api_key=os.environ["DEEPGRAM_API_KEY"]),
aggregators.user(),
GoogleLLMService(
api_key=os.environ["GOOGLE_API_KEY"],
settings=GoogleLLMService.Settings(model="gemini-3.5-flash-lite"),
),
DeepgramTTSService(api_key=os.environ["DEEPGRAM_API_KEY"]),
transport.output(),
aggregators.assistant(),
])
worker = PipelineWorker(pipeline, idle_timeout_secs=None)
# Greet caller immediately when call connects
@transport.event_handler("on_dialin_connected")
async def on_connected(t, payload):
context.add_message({
"role": "developer",
"content": "Start by briefly greeting the caller and asking how you can help.",
})
await worker.queue_frames([LLMRunFrame()])
runner = WorkerRunner(handle_sigint=False)
await runner.add_workers(worker)
await runner.run()
async def main():
config = SessionManagerConfig.create(
api_key=os.environ["AGENTDUET_API_KEY"],
connector_uuid=os.environ["AGENTDUET_CONNECTOR_UUID"],
)
async with SessionManager(config) as sm:
print("Bot is running and waiting for incoming calls...")
@sm.on_incoming_call
async def on_call(noti: IncomingCallNotification):
await run_call(sm, noti)
await sm.run_forever()
if __name__ == "__main__":
asyncio.run(main())

Step 4: Run the bot

Start your voice bot:

python voice_bot.py

Call your AgentDuet phone number. The bot answers and greets you. Speak while it replies to test interruption handling.


Event Lifecycle

Register call lifecycle handlers on the AgentDuetTransport instance with @transport.event_handler(event_name):

EventWhen it occursPayload Type
on_client_connectedCall becomes active (inbound or outbound)CallEventPayload
on_client_disconnectedCall ends (inbound or outbound)CallEventPayload
on_dialin_connectedInbound answer() succeedsCallEventPayload
on_dialin_stoppedRemote caller hangs upCallEventPayload
on_dialin_errorInbound call fails to connectCommandResult
on_call_state_updatedEvery state change on the underlying CallCallState
on_before_disconnectBefore disconnect handlers run (for final logging or flushing)CallEventPayload
on_errorAn underlying SDK error occursError object

Audio Architecture

AgentDuetTransport reads the audio configuration from the Call object. It sets the pipeline input and output to the same format. The format is mono 16-bit PCM at 8 kHz, 16 kHz, or 24 kHz.

Pipecat automatically converts sample rates between services. For example, it can convert between speech recognition and speech generation rates.

Next Step