Skip to content
Thursday, 19 November 2026
Tenesys AI News
Subscribe

All News

TENESYS AI NEWS tracks the most important AI news and explains what changed, why it matters and whether a technology is worth testing.

#Speech-to-Text — 19 articles ✕

Research· Important· 🧪 Worth Testing

New Data-Efficient Technique Enables Full-Duplex Voice Agents Using Frozen LLMs

Researchers have developed a method to adapt general-purpose, frozen LLMs into full-duplex voice systems using only 97 million trainable parameters and 30 GPU-hours of training. The system functions as a thinker-talker cascade with an 80 ms frame clock, allowing for realistic turn-taking, interruptions, and backchanneling without massive speech-data pretraining.

arXiv cs.CL
Voice AI· Important· 🧪 Worth Testing

Medical Mode update reduces drug name transcription errors

AssemblyAI has released an update to its Medical Mode feature designed to improve the transcription of high-stakes clinical terms like medications and diagnoses. The update specifically reduces drug name miss rates by 30% in English and up to 62% in Spanish, while maintaining existing word error rates.

AssemblyAI
Voice AI· Important· 🧪 Worth Testing

AssemblyAI updates streaming diarization to provide post-call accuracy in real-time

AssemblyAI has introduced SpeakerRevision, a feature that delivers a corrected set of speaker labels at the end of a live stream. By utilizing future context unavailable during real-time processing, it achieves accuracy levels comparable to asynchronous transcription, reducing the need for duplicate processing pipelines.

AssemblyAI
Voice AI· Important· 🧪 Worth Testing

Gradium launches Phonon, a lightweight on-device TTS model

Gradium has released Phonon, a suite of compact (65-75 MB) text-to-speech models designed to run entirely on edge devices. It supports five languages, operates offline, and performs competitively against models significantly larger in size.

Hugging Face
Voice AI· Important· 🧪 Worth Testing

Cartesia releases SSM-based voice models on AWS for sub-100ms latency

Cartesia has introduced two models for voice applications: Sonic (text-to-speech) and Ink (speech-to-text). Built on State Space Model (SSM) architecture, these models are designed for ultra-low-latency, real-time performance suitable for production voice agents.

AWS Machine Learning
Voice AI· Important· 🧪 Worth Testing

AssemblyAI launches promptable speech model Universal-3 Pro

AssemblyAI has introduced Universal-3 Pro, a specialized speech-to-text model that allows developers to provide instructions directly to the transcription process. It supports verbatim and clean transcription, custom audio event tagging, and handles code-switching between six languages.

AssemblyAI
Models· Important· 🧪 Worth Testing

NVIDIA Releases Nemotron 3.5 Transcribe for Multilingual Audio Analysis

NVIDIA has introduced Nemotron 3.5 Transcribe, an open-weight, promptable model for speech-to-text, speaker diarization, and translation. It utilizes a hybrid Mamba-Transformer architecture to achieve industry-leading performance with only 4 billion active parameters.

Hugging Face
Models· Important· 🧪 Worth Testing

LiquidAI releases LFM3 multimodal models for edge devices

LiquidAI has launched LFM3, a new generation of small foundation models built on a shared 2B-parameter backbone. The series includes text, vision, audio, and omni models designed for high-efficiency on-device deployment.

Hugging Face
Voice AI· Important· 🧪 Worth Testing

Audible enhances audiobook synthesis with controllable TTS

Audible has developed a controllable text-to-speech pipeline that uses LLMs to profile characters and attribute dialogue with up to 97% accuracy. By providing 'stage directions' as natural-language captions, the system ensures distinct, consistent character voices over long-form content.

Amazon Science
Voice AI· Important· 🧪 Worth Testing

IBM refreshes Granite Speech lineup: new ASR model, quicker edge transcription, sharper speaker attribution

IBM rolled out three updates to its Granite Speech family: a brand-new 2B speech-language model (granite-speech-5.0-2b), a revised 470M Turbo CTC model built for speed, and an improved 4.1-2B-Plus model tuned for multi-speaker recordings. Together they target high-accuracy transcription/translation, fast on-device recognition, and accurate speaker/timestamp labeling. All three remain under Apache 2.0.

Hugging Face
Voice AI· Important· 🧪 Worth Testing

Mistral launches Voxtral 3 voice stack: speech models plus a real-time agent SDK

Mistral AI released Voxtral 3, a set of speech recognition and speech synthesis models paired with an SDK that lets a single real-time pipeline listen, reason, call tools and speak back. The recognition models (Small, Mini and Realtime Transcribe 3) cover 50+ languages with speaker diarization, keyword-biasing for technical vocabulary, and transcription latency under 500ms, which Mistral claims beats Whisper, Scribe v2 and Gemini 3.5 Transcribe on its benchmarks. The speech synthesis model, Mini TTS 3, handles 35+ languages and 50+ dialects with emotion, pacing and emphasis that adapt to context, priced at $13 per million characters, and a separate Pro Voice feature can reproduce a person's voice and emotional range from just 30 minutes of recorded audio. All components can run through Mistral's EU-hosted API or be deployed on a customer's own servers.

Mistral AI
Voice AI· Important· 🧪 Worth Testing

Deepgram cuts batch transcription costs by porting Nova-3 to AWS Trainium2

AWS and Deepgram jointly rebuilt the Nova-3 speech-to-text model to run natively on Trainium2 chips instead of standard GPU-based PyTorch setups, targeting large offline transcription jobs. The work involved rewriting bottleneck layers as custom kernels and replacing Deepgram's Python serving layer with a Rust runtime calling Trainium directly. The result: a single trn2.3xlarge instance running eight parallel model copies delivers 11-70% more transcribed audio per dollar than Deepgram's prior infrastructure, with similar savings potentially available to customers self-hosting on SageMaker.

AWS Machine Learning
Models· Important· 🧪 Worth Testing

IBM combines transcription, decision-making and chat in a single Granite Switch model

IBM has released Granite Switch 4.1 in 8B and 30B sizes, a modular system that runs chat, decision-making and speech transcription from one hosted endpoint instead of separate models. The update adds a general-purpose Decision adapter, a specialized Guardian adapter for safety-related classifications, and an embedded ASR model, Granite-Speech-5.0-Turbo-CTC, which tops the OpenASR Leaderboard. Despite bundling three capabilities, the combined model uses roughly the same VRAM as the base model alone.

Hugging Face
Voice AI· Important· 🧪 Worth Testing

Mistral releases open-source real-time Arabic speech-to-text model Voxtral Mini 4B Realtime Arabic

Mistral AI has released Voxtral Mini 4B Realtime Arabic, a streaming speech-to-text model fine-tuned for Arabic dialects, Modern Standard Arabic, and code-switching between languages. The 4.4-billion-parameter model processes 16 kHz audio in real time with a configurable transcription delay, achieving an 8.82% average Character Error Rate across seven Arabic benchmarks at a 480ms delay — close to the offline Voxtral Transcribe Arabic model's 7.91% CER. It was co-developed with Morocco's digital transition ministry as part of a sovereign AI partnership and is released under Apache 2.0.

Mistral AI (Hugging Face)
Voice AI· Important· 🧪 Worth Testing

AssemblyAI launches Sync API: full transcript in a single HTTP call, ~134ms latency

AssemblyAI introduced the Sync API, a new transport for transcribing short audio clips: one HTTP POST request returns a finished Universal-3.5 Pro transcript in the same response, at roughly 134ms median latency. It fills the gap between the Async API (submit-and-poll, 5-6 seconds of added latency) and the Realtime API (WebSocket, built for ongoing sessions), targeting workloads like dictation, voice-agent turn transcription, IVR, and push-to-talk. Clips from 80ms to 2 minutes and up to 40MB are supported, with word error rate of 1.59% on short-form audio. Pricing is $0.45/hr, the same rate as Universal-3.5 Pro Realtime.

AssemblyAI
Voice AI· Important· 🧪 Worth Testing

AssemblyAI adds Qwen3.5 4B to LLM Gateway for fast voice-transcript rewriting

AssemblyAI launched a hosted deployment of the open-source Qwen3.5 4B model on its LLM Gateway, optimized specifically for rewriting raw speech-to-text output into clean, formatted text. On voice rewrite tasks like dictation cleanup and transcript formatting, the model averaged 612ms response time—1.9x faster than GPT-4.1—at 94% lower cost per hour of audio processed. The model is priced at $0.10/$0.50 per million tokens (prompt/completion) and supports only max_tokens, temperature, and stream parameters, with no tool calling.

AssemblyAI
Voice AI· Important· 🧪 Worth Testing

AssemblyAI launches Universal-3.5 Pro with native code-switching and improved speaker diarization

AssemblyAI released Universal-3.5 Pro, a new flagship async speech-to-text model priced at $0.21/hr. It natively transcribes code-switched conversations across 18 languages without separate configuration, introduces jointly modeled speaker diarization built directly into the transcript (rather than stitched from a separate system), and supports contextual prompting to prime the model with domain knowledge. Benchmarks show lower word error rate on code-switched audio and higher cpWER accuracy on diarization compared to several competitor models and AssemblyAI's own prior Universal-3 Pro.

AssemblyAI
Voice AI· Important· 🧪 Worth Testing

NVIDIA Publishes Fine-Tuning Recipe for Dialect-Specific Speech Recognition with Nemotron

NVIDIA released a detailed workflow for fine-tuning its Nemotron 3.5 ASR multilingual streaming model on Saudi Arabic dialects (Najdi and Hijazi). Using minimal data curation, replay mixing with FLEURS data, duration-based bucketing, and partial encoder unfreezing, they reduced word error rate on the target dialects from 55.05% to 29.96% while also slightly improving English and other Arabic performance. The post also covers decoding tweaks and speaker diarization extensions for multi-speaker transcription.

NVIDIA Developer