> ## Documentation Index
> Fetch the complete documentation index at: https://docs.cruq.ai/llms.txt
> Use this file to discover all available pages before exploring further.

# Voice agents

> A cruq agent that answers and makes phone calls, in real time, in Indian languages.

A voice agent is a normal cruq agent that runs over a phone call instead of a
chat box. Someone dials in, or the agent dials out, and the two of you talk. The
first use case we built is lead qualification in Hinglish: the agent greets the
caller, asks the qualifying questions, and records a structured result.

The agent on the call is the same governed cruq agent you build everywhere else.
It owns the prompt, tools, knowledge, tracing, and metering. Voice is a channel
on top, not a separate product.

## The loop

A voice turn runs a streaming pipeline:

```
telephony -> STT -> LLM -> TTS -> audio
```

The caller's speech is transcribed as they talk, the agent's reply streams into
speech, and the audio plays back down the phone line. Voice activity detection
handles turn-taking and barge-in, so the caller can talk over the agent and it
stops and listens.

The number that matters is **perceived latency**: the gap from the caller
finishing their sentence to the first audio of the reply. Below about 800ms a
call feels like a conversation. Above it, people start talking over the pause.

## The stack

The Phase 0 spike measured every stage on real calls from India and picked the
vendors from the numbers:

* **Orchestration: Pipecat.** Transport-swappable, good observability.
* **Telephony: Exotel.** India-native and DLT-ready. Handles inbound and
  outbound on the same handler; the call's `start` event tags it inbound or
  outbound and carries the caller number.
* **STT: Sarvam.** Accurate Hindi and Hinglish transcription, and it finalizes
  before the end of the turn, so it adds no perceived latency.
* **TTS: Smallest Lightning.** 5 to 140ms to first byte.
* **LLM: governed by cruq.** The pipeline drives a real cruq agent through
  `agent-service` rather than calling a provider directly, so the call inherits
  the agent's prompt, tools, KB, tracing, and metering.

## Latency, honestly

The loop, telephony, STT, TTS, and Hinglish quality all validated end to end on
real calls. One thing sits above target:

| Stage                      | Cost                         | Verdict            |
| -------------------------- | ---------------------------- | ------------------ |
| STT (Sarvam)               | overlapped, effectively free | keep               |
| TTS first byte (Smallest)  | 5 to 140ms                   | keep               |
| Telephony playout (Exotel) | \~200ms fixed                | keep               |
| **LLM first token**        | **760 to 1000ms**            | **the bottleneck** |

With gemini-2.5-flash-lite, perceived p50 landed around 1130ms, with the LLM's
first token being almost all of it. Everything else is already within budget.
The next lever is a low-time-to-first-token model (a Groq or Cerebras served
Llama), which should cut first token to a few hundred milliseconds and bring
perceived latency toward 550 to 650ms, at a possible cost to Hinglish quality
worth A/B testing.

## Language

Forcing Hinglish in the system prompt produces a natural romanized Hinglish
greeting and qualification, with no drift back to English. Sarvam transcribes
the caller's Hindi and Hinglish accurately in both directions.

## Every call is recorded

A call persists three things, all tenant-scoped:

* A **recording** (WAV, caller and agent mixed) in object storage.
* A **transcript** (the conversation, timestamped) as JSON.
* A **`voice_calls` row** in Postgres with the vendors used, turn count,
  duration, and perceived-latency p50, under row-level security like every other
  tenant table.

You review them in the app's **Voice Calls** page: a list of calls, and a detail
view with the transcript and recording playback.

## Pausing on a call

The **Ask a Human** tool works over voice. When the agent hits something it
should not decide alone, it speaks the question, parks the run as paused, and
resumes on the caller's next turn using the same thread. The call does not drop
while it waits.

## Status

Phase 0 (the throwaway spike, `apps/voice-spike`) validated the loop, vendors,
and Hinglish quality on real phone calls. Phase 1 shipped the governed
cruq-agent backend, call recording and transcript persistence to object storage
and Postgres, the Voice Calls page, and the Kubernetes deploy scaffolding.
Bulk or commercial calling in India still requires DLT registration before it
goes wide.

<Note>
  `backchannel` (in the sibling `hf-voice` repo) is a separate, open experiment in
  a full-duplex voice framework with first-class barge-in and turn detection. It
  is not part of the product runtime described here.
</Note>