Bhashini.ai API Docs

End User Interaction

A Caller / End User initiates a voice call using a regular telephone or VoIP client. The call traverses the PSTN or VoIP network, ensuring compatibility with traditional telephony infrastructure.

Media Gateway Layer

The call is terminated at a Media Gateway, which can be:

• An on-premise SIP PBX such as Asterisk

• A cloud telephony provider like Twilio

• A WebRTC platform such as LiveKit

The Media Gateway is responsible for:

• SIP signaling

• RTP audio handling

• Bridging telephony audio to WebSocket-based AI services

This layer enables seamless integration between telecom networks and real-time AI pipelines.

Speech-to-Text (STT): Real-Time Transcription

The Media Gateway streams raw audio to Bhashini.ai Streaming STT using a low-latency WebSocket API.

Key capabilities of Bhashini STT:

• Real-time streaming transcription

• Built-in Voice Activity Detection (VAD)

• Automatic speech segmentation

• Interim (partial) and final transcripts

• Optimized for conversational latency

Transcribed text (both interim and final) is forwarded to the LLM / Dialog Engine as soon as it becomes available, allowing the AI system to prepare the response while the caller is still speaking.

LLM / Dialog Engine: Conversation Intelligence

The LLM or Dialog Engine acts as the brain of the AI call center. It:

• Maintains conversation context

• Applies business logic and policies

• Performs intent detection and slot filling

• Invokes tools and APIs

When needed, it interacts with Enterprise Backend Systems such as:

• CRM platforms

• Ticketing systems

• Payment gateways

• Knowledge bases

This allows the AI agent to provide personalized, actionable responses in real time.

Text-to-Speech (TTS): Streaming Voice Responses

Once the LLM generates a response, text is sent to Bhashini.ai Streaming TTS via WebSocket.

Key TTS capabilities:

• Low-latency, streaming audio synthesis

• Natural, expressive voices

• Optimized for conversational turn-taking

• Suitable for barge-in and interruptions

Synthesized speech is streamed back to the Media Gateway, which injects it into the live call using RTP. The audio then flows through the PSTN/VoIP network and is played back to the caller.