Connecting a telephony provider to an AI voice agent means bridging that carrier's media stream to whatever runs your speech recognition, model and voice — and every carrier speaks a different dialect of “streaming audio”. Twilio wraps base64 μ-law inside JSON. Vonage sends raw binary PCM16 with no envelope at all. Exotel uses snake_case keys and silently discards frames under 100 milliseconds. They also disagree about the single most important message in a phone agent: the one that flushes queued playback the instant a caller interrupts. We built adapters for nine providers, verified each against the vendor's own published reference, and released the result as Setu (सेतु — bridge) under an MIT licence at github.com/dvaarikai/setu. This is what we learned doing it.
Why is carrier plumbing the part that stalls projects?
Because it is invisible in every demo and absent from every architecture diagram.
A voice agent is four genuinely hard problems stacked on each other: hearing (speech recognition that survives 8 kHz phone audio), thinking (a model that answers from your business data), speaking (natural voice, streamed so replies start fast), and turn-taking (knowing when the caller has finished, and when they have interrupted). Those four are what a voice API sells you.
The fifth problem is getting the audio from your carrier to that API and back. It has nothing to do with your product, no user ever sees it, and it reliably takes about a week: byte formats, base64 envelopes, sample-rate conversion, buffer management, and per-provider quirks that appear in no documentation. Teams budget for the AI and get ambushed by the plumbing.
What does each provider actually send?
Seven dialects, one table. Each row is taken from the adapter that implements it, which was written from the vendor's own published reference.
| Provider | Wire format | Sample rate | Flush frame |
|---|---|---|---|
| Twilio | JSON, base64 μ-law | 8 kHz | `{"event":"clear"}` |
| SignalWire | Twilio-compatible | 8 kHz | `{"event":"clear"}` |
| Plivo | JSON, base64 L16 or μ-law | 8 kHz | `{"event":"clearAudio"}` |
| Exotel | JSON snake_case, base64 L16 | 8 kHz | `{"event":"clear"}` |
| FreJun / Teler | JSON, base64 PCM16 chunks | 8 kHz | `{"type":"clear"}` |
| Vonage | Raw binary PCM16 | 16 kHz | `{"action":"clear"}` |
| Telnyx | JSON, base64 RTP | 8 kHz | `{"event":"clear"}` |
| jambonz | Binary PCM16 | 16 kHz | `{"type":"killAudio"}` |
Read the last column twice. Three providers spell the same instruction three different ways, and Vonage keys it on `action` where everyone else uses `event`. A handler written against Twilio connects to Vonage, streams audio correctly, and then silently never interrupts — the hardest class of bug, because everything looks like it works.
Why does the flush frame matter so much?
Because carriers queue seconds of audio ahead of the caller's ear.
When a caller talks over the agent, your voice engine stops generating immediately — that part is easy and every API handles it. But the audio already sent to the carrier is sitting in a playback buffer, and the caller keeps hearing it. Without an explicit flush, someone who interrupts at second two hears the agent finish a sentence it abandoned, then start a new one. In a real conversation that reads as rudeness, and it is the single most common reason a technically working voice agent feels wrong on the phone.
The fix is one message per provider, sent the moment an interruption is detected. It is trivial once you know the exact frame. Finding the exact frame, for seven providers, is the week.
The traps that cost a day each
Four we hit, documented so nobody repeats them.
Twilio's stream URL takes no query string. The `<Stream url>` attribute in TwiML drops anything you append, so per-call configuration cannot ride in the URL. It has to travel in `<Parameter>` children or be looked up on your side from the stream identifier.
Telnyx defaults to MP3. Its streaming mode must be set to `rtp` or raw audio is silently discarded. The call connects, the agent believes it is speaking, and nothing reaches the caller — with no error anywhere.
Exotel enforces a 100 ms floor. It drops any frame shorter than 100 milliseconds or not a multiple of 320 bytes, which is exactly the shape a voice engine emits as it speaks. Pass the engine's output straight through and the caller hears silence. The adapter buffers to legal sizes; without that, this integration simply does not work.
FreJun and Teler need pacing, not just format. Play-out is fixed at 8 kHz, so 16 kHz audio returns as a half-speed ghost. A voice engine emits faster than realtime, and overflowing the buffer produces a periodic burst that is audible on the live call but absent from the recording — the recording is stitched from the bytes you sent, so the fault cannot be replayed to debug it. The adapter paces to realtime and grows its frames so the first word stays fast and the rest stays smooth.
jambonz opens with metadata. The first frame on the socket is JSON call metadata, not audio, so a handler assuming every text frame is an event misreads the opening one.
What if you have a SIP trunk rather than a CPaaS account?
Put a media gateway in front of it and treat it like any other WebSocket source.
jambonz is open source and self-hosted: it accepts a SIP trunk on one side and gives you a WebSocket on the other. That means anyone with a raw trunk uses the jambonz adapter and is finished — no SIP stack in your application, and none in ours either.
That is a deliberate architectural choice, not just convenience. Running a SIP stack pulls a voice product into telecom licensing questions that a media gateway sidesteps entirely. The bridge stays a bridge.
Why give this away?
Because a bridge that crosses to only one bank is not a bridge.
Setu's engine interface is public, so it is not locked to Dvaarik — point it at another voice backend and it still works. That is what makes it an open-source project rather than a brochure with a licence file attached, and it is the only version of this worth publishing.
The commercial logic is not complicated either. The week of plumbing was never the product; the voice engine is. Removing the plumbing removes the most common reason a voice project dies before it reaches a phone call — and a developer who bridges their carrier in an afternoon is a developer who gets to evaluate the thing we actually sell. `npx @dvaarik/setu`, MIT licensed, at github.com/dvaarikai/setu. There is also a complete Python and FastAPI implementation of the whole protocol, barge-in included, in the examples folder — one readable file for shops that would rather copy than depend.
Frequently asked questions
How do I connect a telephony provider to an AI voice agent?
Point the provider's media stream at a WebSocket bridge that translates its wire format to what your voice engine expects, and send that provider's specific flush frame whenever the caller interrupts. Setu does this for Twilio, SignalWire, Plivo, Exotel, Vonage, Telnyx, jambonz and raw PCM16 sources: npx @dvaarik/setu, MIT licensed.
Why does the caller hear the agent talking over them?
Because the carrier has queued seconds of audio ahead of the caller and nothing told it to discard the queue. Stopping generation is not enough; you must send the provider's flush frame, which is `{"event":"clear"}` on Twilio, `{"event":"clearAudio"}` on Plivo, `{"action":"clear"}` on Vonage and `{"type":"killAudio"}` on jambonz.
Does Setu only work with Dvaarik?
No. The engine interface is public and the licence is MIT, so it can bridge a carrier to any voice backend. It ships with a Dvaarik implementation because that is what we run, not because it is required.
What if my provider is not on the list?
Use the raw path, which accepts binary PCM16 from anything — another carrier, a browser, an app, or a test harness. If you hold a SIP trunk, put jambonz in front of it and use the jambonz adapter rather than building a SIP stack.
Is there a Python version?
Yes. The repository contains a complete Python and FastAPI implementation of the protocol, barge-in included, in one readable file. Copying that file is a better outcome for a Python shop than depending on a half-maintained second library.
Carrier plumbing is unglamorous, undocumented and roughly a week of work that no customer will ever thank you for. It is also the reason a lot of promising voice agents never make it to a real phone call. The seven adapters, the flush frames and the four traps above are the whole of what we learned, published under a licence that lets anyone use them with any voice backend. If you run a paid call through an adapter we have only written from documentation, open an issue either way — the status column in that table should tell the truth, and today only Exotel has earned the production label.
Bridge your carrier in an afternoon: npx @dvaarik/setu, then read the per-provider guides at developers.dvaarik.com/integrations.
LoginTagged
Written by
Rohith Sriramula
Founder & CEO, Dvaarik AI
A laid-off engineer who went all in on Dvaarik AI. He builds the platform and product workflows from hands-on work with Indian businesses, not theory.