Three open-weights voice models dropped on March 26, 2026 — the same day Google announced Gemini 3.1 Flash Live. That's not a coincidence; it's a roadmap. Gemini Flash Live is impressive real-time voice AI, but it sends everything to Google's servers, retains your conversations for up to 18 months by default, and charges per minute. With Cohere Transcribe, Covo-Audio, and Voxtral TTS, you can build a local voice stack on a consumer GPU that keeps conversations on your hardware and costs $0 per minute of audio. This article shows you what the choice actually looks like.
What Gemini 3.1 Flash Live Actually Does
Google announced Gemini 3.1 Flash Live on March 26, 2026, and the architecture is genuinely different from prior voice products. Where older voice AI pipelines converted your speech to text, ran the text through a language model, then converted the response back to speech, Flash Live processes native audio directly — pitch, pace, pauses, and background noise all go in as acoustic data instead of a text transcript.
The practical effect: fewer awkward pauses, better handling of interruptions and mid-sentence corrections, and context retention across multi-turn exchanges without transcription artifacts degrading the conversation. Google hasn't published a specific millisecond latency benchmark — the announcement describes it as "faster responses with fewer awkward pauses" — but the native audio pipeline eliminates one transcription round-trip that prior systems had.
What It's Actually Good At
Flash Live targets customer service deployments, tutoring, and real-time content collaboration. It supports 90+ languages, handles full-duplex audio and video input via Google Lens, and includes tool-calling for live agent use. For building voice-forward AI applications quickly, it's a compelling starting point.
The catch is structural, not technical.
The Hidden Costs: Privacy and Pricing
Every conversation you have with Gemini Flash Live travels to Google's servers. Consumer Gemini accounts default to 18-month conversation retention, and your data can be used for model training unless you manually opt out. When you turn conversation history off, new chats are saved for up to 72 hours for service delivery before deletion — but that window still means your voice data hit Google's infrastructure.
Warning
Default consumer Gemini settings do not meet HIPAA, GDPR, or CCPA requirements. Google Workspace Enterprise and Vertex AI with zero-data-retention are the appropriate tiers for regulated data — but those come with enterprise pricing, not Flash Live's preview rate.
If you're using Flash Live to prototype personal productivity tools, the default settings might be fine. If you're handling client conversations, medical queries, legal discussions, or anything under a data regulation, they're not.
The Latency You Can't Control
Cloud voice AI adds a fixed network overhead on top of model inference time. For a user in the US Northeast with a solid connection, that overhead is 30-50ms. For users in Southeast Asia or on congested business networks, it's 150ms or more — and it varies unpredictably by ISP, time of day, and server load. Google can't give you a guaranteed latency SLA on a preview product.
Local voice stacks don't have this problem. Inference runs on your GPU, latency is determined by your hardware, and variance is near zero once you've profiled the pipeline.
Pricing at Scale
Gemini 3.1 Flash Live is in preview as of March 2026 with no confirmed pricing. The prior Gemini 2.0 Flash Live rate was approximately $0.023 per minute combined for audio input and output. At $1.38/hour of voice interaction, a single heavy-use workload — say, 8 hours of voice AI for a small customer service operation — costs about $11/day, $330/month, or roughly $4,000/year. A local GPU pays that back in under a year for any sustained workload.
The Local Stack: Architecture and Components
Three open-weights models released in the same week as Gemini Flash Live form a credible local alternative. Each handles one layer of the voice pipeline.
Cohere Transcribe (released March 26, 2026) is a 2B-parameter speech-to-text model that tops the HuggingFace Open ASR Leaderboard with a 5.42% average word error rate — beating OpenAI Whisper Large v3. On LibriSpeech clean audio it hits 1.25% WER. On noisy conditions and meeting audio it runs 8.13% WER. Throughput is 525 minutes of audio processed per minute of compute. At 2B parameters, it fits comfortably in approximately 6-8 GB of VRAM and runs on consumer GPUs.
Covo-Audio (Tencent, released March 26, 2026) is a 7B-parameter end-to-end audio language model that eliminates the STT→LLM→TTS cascade entirely. You feed it voice input; it reasons and generates voice output in a single unified architecture built on a Qwen2.5-7B backbone. The full-duplex variant, Covo-Audio-Chat-FD, reports 99.7% turn-taking success and 96.8% interruption handling accuracy — metrics that matter for natural conversation flow. In BF16 it needs approximately 16 GB VRAM; 8-bit quantization brings that to roughly 8 GB, making it accessible on a 16 GB card.
Voxtral TTS (Mistral AI, released March 26, 2026) is a 4B-parameter text-to-speech model with voice cloning from as little as 3 seconds of reference audio. It captures accent, intonation, and natural disfluencies. Mistral reports 70ms model latency for a typical 10-second voice sample. Self-hosting via vLLM requires at least 16 GB VRAM — the 4B weights alone run 8.04 GB in BF16, and inference needs headroom on top. This is the TTS layer for a premium three-component pipeline; it requires a larger card.
Two Ways to Architect the Stack
Option A (simpler, fits 16 GB): Cohere Transcribe for transcription feeding Covo-Audio-Chat-FD for end-to-end conversational audio. Covo-Audio handles reasoning and voice output internally. Combined VRAM footprint at 8-bit quantization is roughly 14-16 GB — fits on an RTX 5080 or a used RTX 3090.
Option B (three layers, 32 GB): Cohere Transcribe (STT) → a local reasoning LLM (Qwen 3-4B or similar) → Voxtral TTS (speech synthesis). More modular, easier to swap components, and gives you better voice cloning control. But Voxtral alone needs ≥16 GB, so you're looking at an RTX 5090 (32 GB) to run all three without degrading quality.
Tip
Stream LLM tokens to TTS in real time rather than waiting for the full inference response. This single optimization cuts perceived end-to-end latency roughly in half on any three-layer local stack — you start hearing output while the model is still generating.
Hardware Tiers and Real Costs
Note
No published benchmark exists yet for these specific three-model combinations. The latency estimates below are extrapolated from published benchmarks on comparable pipeline configurations. Treat them as directional, not precise.
Mid-range: RTX 4090 (24 GB) or RTX 5080 (16 GB)
The RTX 4090 remains the best value for a local voice stack at this tier. Used cards run $800-$1,200. It handles Option A (Cohere Transcribe + Covo-Audio BF16) with room to spare — roughly 22 GB for both models in BF16, which fits.
The RTX 5080 has a $999 MSRP but retail prices in March 2026 run $1,200-$1,500 due to supply constraints. At 16 GB VRAM it only supports Option A with Covo-Audio quantized to 8-bit. That's a real trade-off in voice quality. For a dedicated voice AI workstation, the used RTX 4090 is a better starting point at current prices.
Published benchmarks for a three-layer pipeline (Whisper Turbo + 7-8B LLM + Piper TTS) on a 12 GB GPU show approximately 1 second end-to-end latency without streaming. Option A with Covo-Audio on a 24 GB card should do better — but confirm in your own environment before committing to a deployment.
Full build cost (mid-range): RTX 4090 used ($900-$1,100) + motherboard/CPU/RAM/PSU/case ($600-800) = approximately $1,500-$2,000 total.
High-end: RTX 5090 (32 GB)
This is the right card for Option B and for any deployment that needs 3+ concurrent voice sessions. At 32 GB you run Cohere Transcribe (~6 GB), a 4B reasoning LLM (~5 GB quantized), and Voxtral TTS (~16 GB) simultaneously. RTX 5090 MSRP is $1,999; actual retail in March 2026 runs $2,900-$3,700+ for third-party AIB cards. Founders Edition is closest to MSRP but rarely in stock.
Full build cost (high-end): RTX 5090 ($2,000-$3,500) + base system ($700) = approximately $2,700-$4,500.
Local vs Cloud: The Real Trade-off Table
Local Stack (Option B)
Potentially sub-500ms with streaming
Zero
Yes
$0
~$2,700–$4,500
3–8 sessions (32 GB)
None
Full control The break-even math for sustained workloads is straightforward. At $0.023/min, an 8-hour/day voice AI operation costs ~$330/month. A $1,800 local build recoups hardware cost in roughly 5-6 months.
If your use case is intermittent — a few hours per week of voice interaction — the cloud economics win. If it's a core workflow, local wins on cost within a year, and it wins on privacy from day one.
Getting Started with the Local Stack
Prerequisites: NVIDIA GPU (16 GB minimum for Option A, 32 GB for Option B), CUDA 12.1+ toolkit, Ubuntu 22.04 or Windows 11 with WSL2.
1. Install NVIDIA drivers and CUDA 12.1+ from the NVIDIA developer site. Confirm with nvidia-smi.
2. For Option A — clone the Covo-Audio repository from tencent/Covo-Audio on HuggingFace. If you're on a 16 GB card, apply 8-bit quantization via the bitsandbytes loader before running the inference server.
3. Install Cohere Transcribe from CohereLabs/cohere-transcribe-03-2026 on HuggingFace. Run a quick accuracy check against a sample of your actual audio environment — WER varies significantly between clean studio audio and a noisy home office.
4. For Option B — install vLLM 0.18.0+ (required for Voxtral TTS) and clone Voxtral from mistralai/Voxtral-TTS-4B-2603. vLLM handles memory management and streaming output automatically.
5. Wire the pipeline: audio capture → Cohere Transcribe (STT) → Covo-Audio or LLM+Voxtral TTS → audio output. Measure end-to-end latency in your environment and enable streaming output from the LLM before committing the setup to production.
For a complete hardware build guide and cost breakdown, see our RTX 5080 and 5090 build guide. For quantization options and how Q4/Q6 trade-offs affect voice quality specifically, the quantization glossary entry covers the key decision points.
The CraftRigs Take
Gemini 3.1 Flash Live is the best cloud voice AI available right now. The native audio processing is a real architectural improvement, and for anyone prototyping quickly or running low-volume personal tools, it's a reasonable starting point.
But three open-weights models released the same week aren't a coincidence — they're proof that the gap between cloud voice AI and local voice AI is closing fast. Cohere Transcribe beats Whisper Large v3 on accuracy. Covo-Audio handles full-duplex conversation on a consumer GPU. Voxtral TTS delivers voice cloning with 70ms synthesis latency.
If you're running sensitive workloads, building for regulated industries, or doing enough voice volume that $330/month in cloud fees is real money — local voice is viable today, in March 2026, on hardware you can order this week. The broader cloud vs. local AI comparison covers how this pattern repeats across every modality.
FAQ
Does Gemini 3.1 Flash Live store my voice conversations?
Yes. Consumer Gemini accounts default to 18-month conversation retention, and your data can be used for Google model training unless you opt out through account settings. When you turn conversation history off, chats are saved for up to 72 hours before deletion — they still hit Google's servers. If you need data minimization for compliance, Google Workspace Enterprise with admin-controlled retention or Vertex AI with zero-data-retention is the appropriate tier. Consumer settings don't satisfy HIPAA, GDPR, or CCPA requirements by default.
Can I run a local voice AI stack on a consumer GPU in 2026?
Yes. Cohere Transcribe (2B parameters, ~6-8 GB VRAM) paired with Covo-Audio-Chat (7B, ~8 GB VRAM at 8-bit quantization) fits on a 16 GB card like the RTX 5080 or a used RTX 3090. If you want the premium three-layer stack with Voxtral TTS, Voxtral alone requires at least 16 GB VRAM, which means you need a 32 GB card like the RTX 5090 to run the full stack simultaneously. All three models are open weights and available on HuggingFace.
How much does Gemini 3.1 Flash Live cost per minute?
Gemini 3.1 Flash Live is in preview as of March 2026 and pricing isn't confirmed for this version. The prior Gemini 2.0 Flash Live rate was approximately $0.005/min for audio input and $0.018/min for audio output — about $0.023/min combined for a two-way voice session, or roughly $1.38/hour. A local stack has zero per-minute cost after the hardware purchase. At 8 hours/day of voice use, you recover a $1,800 GPU build in under 6 months compared to cloud pricing.
What is Voxtral and how is it different from Covo-Audio?
Voxtral TTS (Mistral AI, 4B parameters, released March 26, 2026) is a text-to-speech model: you give it text, it produces natural speech with voice cloning from a 3-second audio sample, in 9 languages. Covo-Audio (Tencent, 7B parameters, released March 26, 2026) is an end-to-end audio language model — it takes voice input and produces voice output in a single architecture without separate STT or TTS steps. Voxtral gives you precise voice control and integrates into any LLM pipeline. Covo-Audio is simpler to deploy for full-duplex conversation but is harder to customize.
What latency should I expect from a local voice stack?
No published benchmark exists for the specific Covo-Audio or Voxtral TTS + Cohere Transcribe combination yet. Comparable three-layer pipelines (Whisper Turbo + 7-8B LLM + lightweight TTS) benchmark around 1 second end-to-end on a 12 GB GPU without streaming. Enabling LLM token streaming to TTS in parallel cuts perceived latency significantly — sub-500ms response is achievable on 24 GB+ hardware. Voxtral TTS alone reports 70ms for a typical 10-second voice sample. Profile your actual environment before making latency commitments in production.