Glossary · Updated Sep 2, 2026

Latency

The delay between sending a request to an AI system and receiving its response, critical for voice and real-time uses.

Definition

Latency includes network time, queueing, and the model's generation time, which grows with output length. Streaming shows partial output early to feel faster. Voice agents need sub-second response to feel natural, while batch content generation can tolerate minutes. Providers publish or hint at latency tiers.

Why it matters when choosing a tool

For chat, voice, and in-editor coding, latency shapes usability as much as accuracy does.

Where you will meet it

AI Audio & Voice, AI Customer Support, AI Coding & Development

Related terms

Inference · Text-to-speech (TTS) · Streaming responses

Tools where this matters

Reviewed products in the related categories