Skip to content
Thursday, 19 November 2026
Tenesys AI News
Subscribe

Voice AI

Speech-to-Text, Text-to-Speech, realtime models and voice AI agents.

“Agents” — 20 articles ✕

Voice AI· Important· 🧪 Worth Testing

AssemblyAI releases specialized STT model for voice agents

AssemblyAI has introduced AssemblyAI-STT-3.7-Realtime-Agent, a speech-to-text model specifically designed for voice agent interaction. It aims to solve common issues in live telephony, such as distinguishing user speech from background noise and capturing critical data points like numbers at the end of a sentence.

AssemblyAI
Voice AI· Important· 🧪 Worth Testing

AssemblyAI introduces specialized STT and TTS models for voice agents

AssemblyAI has launched the STT-3.7 model family, featuring dedicated versions for pre-recorded audio, live conversations, and voice agents. Additionally, the company introduced its first text-to-speech engine, AssemblyAI-TTS-1, optimized for high-speed, natural-sounding interactions.

AssemblyAI
Voice AI· Important· 🧪 Worth Testing

Medical Mode update reduces drug name transcription errors

AssemblyAI has released an update to its Medical Mode feature designed to improve the transcription of high-stakes clinical terms like medications and diagnoses. The update specifically reduces drug name miss rates by 30% in English and up to 62% in Spanish, while maintaining existing word error rates.

AssemblyAI
Voice AI· Important· 🧪 Worth Testing

Google Launches Offline On-Device Speech Recognition and TTS APIs

Google has expanded its AI Edge Gallery with new on-device speech capabilities, enabling offline, low-latency transcription and synthesis. Developers can now integrate these high-performance models via Android's ML Kit or cross-platform MediaPipe Tasks, supporting over 100 languages with significant energy efficiency.

Google Developers
Voice AI· Important· 🧪 Worth Testing

Cartesia releases SSM-based voice models on AWS for sub-100ms latency

Cartesia has introduced two models for voice applications: Sonic (text-to-speech) and Ink (speech-to-text). Built on State Space Model (SSM) architecture, these models are designed for ultra-low-latency, real-time performance suitable for production voice agents.

AWS Machine Learning
Voice AI· Important· 🧪 Worth Testing

Audible enhances audiobook synthesis with controllable TTS

Audible has developed a controllable text-to-speech pipeline that uses LLMs to profile characters and attribute dialogue with up to 97% accuracy. By providing 'stage directions' as natural-language captions, the system ensures distinct, consistent character voices over long-form content.

Amazon Science
Voice AI· Important· 🧪 Worth Testing

Amazon Polly's Generative TTS engine gets a major accuracy and voice upgrade

AWS has upgraded the Amazon Polly Generative engine with more natural, expressive speech and better accuracy for numbers, addresses and confirmation codes. The update adds 14 new voices across 9 locales, bringing the total to 57 generative voices in 11 languages and 25 locales, including first-time general availability for Hindi and Finnish. It is a drop-in upgrade: voice names, APIs and pricing stay the same, so existing applications work unchanged.

AWS Machine Learning
Voice AI· Important· 🧪 Worth Testing

IBM refreshes Granite Speech lineup: new ASR model, quicker edge transcription, sharper speaker attribution

IBM rolled out three updates to its Granite Speech family: a brand-new 2B speech-language model (granite-speech-5.0-2b), a revised 470M Turbo CTC model built for speed, and an improved 4.1-2B-Plus model tuned for multi-speaker recordings. Together they target high-accuracy transcription/translation, fast on-device recognition, and accurate speaker/timestamp labeling. All three remain under Apache 2.0.

Hugging Face
Voice AI· Important· 🧪 Worth Testing

Mistral launches Voxtral 3 voice stack: speech models plus a real-time agent SDK

Mistral AI released Voxtral 3, a set of speech recognition and speech synthesis models paired with an SDK that lets a single real-time pipeline listen, reason, call tools and speak back. The recognition models (Small, Mini and Realtime Transcribe 3) cover 50+ languages with speaker diarization, keyword-biasing for technical vocabulary, and transcription latency under 500ms, which Mistral claims beats Whisper, Scribe v2 and Gemini 3.5 Transcribe on its benchmarks. The speech synthesis model, Mini TTS 3, handles 35+ languages and 50+ dialects with emotion, pacing and emphasis that adapt to context, priced at $13 per million characters, and a separate Pro Voice feature can reproduce a person's voice and emotional range from just 30 minutes of recorded audio. All components can run through Mistral's EU-hosted API or be deployed on a customer's own servers.

Mistral AI
Voice AI· Important· 🧪 Worth Testing

OpenAI makes custom voice generation available to every GPT-Live API user

OpenAI has rolled out prompt-to-voice access to all GPT-Live API customers, allowing a spoken voice to be designed simply by describing it in text, either through the API or the voice Playground, and then reused inside live voice sessions. Designing the voice costs nothing, but speaking with it still runs on normal GPT-Live session billing. Alongside this rollout, nine additional built-in voices covering German, French, Hindi, Japanese and Korean were added to the platform.

OpenAI API Changelog
Voice AI· Important· 🧪 Worth Testing

Deepgram's Flux STT cuts unwanted interruptions in Anam's avatar conversations by ~50%

Anam, which builds photorealistic interactive avatars, switched to Deepgram's Flux speech-to-text model to solve turn-taking problems in unscripted, real-time conversation. In A/B testing against a self-hosted open-source STT option, Flux cut unwanted avatar barge-ins by roughly 50% with no added latency penalty, and complaints about turn-taking on English sessions dropped to near zero. Anam deployed Flux self-hosted on its own Kubernetes/H100 infrastructure, rolling it out to 100% of turnkey users after a two-week self-hosted deployment (versus days for Deepgram's hosted option).

Deepgram
Voice AI· 🧪 Worth Testing

Small Fine-Tuned Model Learns to Follow New Regulated Call Scripts Without Retraining

Researchers describe a system called FlowSpec that converts a creditor's legal call procedure into machine-readable instructions, tool restrictions, training signals and test cases for a voice agent. A 9-billion-parameter model was trained on calls covering nearly 300 randomly generated procedures, then tested on four procedures it had never encountered, including one entirely held-out family of scripts, to see whether compliance behavior transfers.

arXiv cs.CL
Voice AI· Important· 🧪 Worth Testing

xAI upgrades Grok Voice to version 3.0 with no price increase

xAI has rolled out Grok Voice 3.0, its latest speech-to-speech voice model, without changing pricing from the prior Grok Voice 2.0. Developers who reference the grok-voice-latest alias get the new model automatically, with no code changes needed. The API itself remains the same as before.

xAI API Release Notes
Voice AI· Important· 🧪 Worth Testing

Mistral releases open-source real-time Arabic speech-to-text model Voxtral Mini 4B Realtime Arabic

Mistral AI has released Voxtral Mini 4B Realtime Arabic, a streaming speech-to-text model fine-tuned for Arabic dialects, Modern Standard Arabic, and code-switching between languages. The 4.4-billion-parameter model processes 16 kHz audio in real time with a configurable transcription delay, achieving an 8.82% average Character Error Rate across seven Arabic benchmarks at a 480ms delay — close to the offline Voxtral Transcribe Arabic model's 7.91% CER. It was co-developed with Morocco's digital transition ministry as part of a sovereign AI partnership and is released under Apache 2.0.

Mistral AI (Hugging Face)
Voice AI· Important· 🧪 Worth Testing

Deepgram Extends Medical Speech Recognition to 10 Languages

Deepgram has expanded its medical-specialized speech model, Nova-3 Medical, from English-only to 10 languages including Spanish, German, Hindi, Japanese and Russian, with automatic code switching in both batch and streaming modes. On a 115-hour curated medical audio test set, the multilingual medical model cut word error rate by 25-30% and entity error rate (drugs, doses, conditions) by 37-40% compared to Deepgram's general-purpose multilingual model. It also beat competing models from ElevenLabs and Microsoft on average word error rate across all 10 languages.

Deepgram
Voice AI· Important· 🧪 Worth Testing

AssemblyAI launches Sync API: full transcript in a single HTTP call, ~134ms latency

AssemblyAI introduced the Sync API, a new transport for transcribing short audio clips: one HTTP POST request returns a finished Universal-3.5 Pro transcript in the same response, at roughly 134ms median latency. It fills the gap between the Async API (submit-and-poll, 5-6 seconds of added latency) and the Realtime API (WebSocket, built for ongoing sessions), targeting workloads like dictation, voice-agent turn transcription, IVR, and push-to-talk. Clips from 80ms to 2 minutes and up to 40MB are supported, with word error rate of 1.59% on short-form audio. Pricing is $0.45/hr, the same rate as Universal-3.5 Pro Realtime.

AssemblyAI
Voice AI· Important· 🧪 Worth Testing

AssemblyAI adds Qwen3.5 4B to LLM Gateway for fast voice-transcript rewriting

AssemblyAI launched a hosted deployment of the open-source Qwen3.5 4B model on its LLM Gateway, optimized specifically for rewriting raw speech-to-text output into clean, formatted text. On voice rewrite tasks like dictation cleanup and transcript formatting, the model averaged 612ms response time—1.9x faster than GPT-4.1—at 94% lower cost per hour of audio processed. The model is priced at $0.10/$0.50 per million tokens (prompt/completion) and supports only max_tokens, temperature, and stream parameters, with no tool calling.

AssemblyAI
Voice AI· Important· 🧪 Worth Testing

NVIDIA Publishes Fine-Tuning Recipe for Dialect-Specific Speech Recognition with Nemotron

NVIDIA released a detailed workflow for fine-tuning its Nemotron 3.5 ASR multilingual streaming model on Saudi Arabic dialects (Najdi and Hijazi). Using minimal data curation, replay mixing with FLEURS data, duration-based bucketing, and partial encoder unfreezing, they reduced word error rate on the target dialects from 55.05% to 29.96% while also slightly improving English and other Arabic performance. The post also covers decoding tweaks and speaker diarization extensions for multi-speaker transcription.

NVIDIA Developer