Skip to content
Thursday, 19 November 2026
Tenesys AI News
Subscribe

All News

TENESYS AI NEWS tracks the most important AI news and explains what changed, why it matters and whether a technology is worth testing.

#Speech-to-Text — 18 articles ✕

Models· Important· 🧪 Worth Testing

Unified Multilingual Speech Tagging with Multimodal LLMs

Researchers introduced OmniTag-Bench, a new five-language framework for evaluating multimodal LLMs on simultaneous speech transcription and metadata tagging. Instead of using separate specialist models, the study demonstrates that a single, fine-tuned multimodal LLM can extract transcripts, emotion, gender, language, and named entities in a single inference pass.

arXiv cs.CL
Voice AI· Important· 🧪 Worth Testing

AssemblyAI releases specialized STT model for voice agents

AssemblyAI has introduced AssemblyAI-STT-3.7-Realtime-Agent, a speech-to-text model specifically designed for voice agent interaction. It aims to solve common issues in live telephony, such as distinguishing user speech from background noise and capturing critical data points like numbers at the end of a sentence.

AssemblyAI
Voice AI· Important· 🧪 Worth Testing

AssemblyAI introduces specialized STT and TTS models for voice agents

AssemblyAI has launched the STT-3.7 model family, featuring dedicated versions for pre-recorded audio, live conversations, and voice agents. Additionally, the company introduced its first text-to-speech engine, AssemblyAI-TTS-1, optimized for high-speed, natural-sounding interactions.

AssemblyAI
Research· Important· 🧪 Worth Testing

New Data-Efficient Technique Enables Full-Duplex Voice Agents Using Frozen LLMs

Researchers have developed a method to adapt general-purpose, frozen LLMs into full-duplex voice systems using only 97 million trainable parameters and 30 GPU-hours of training. The system functions as a thinker-talker cascade with an 80 ms frame clock, allowing for realistic turn-taking, interruptions, and backchanneling without massive speech-data pretraining.

arXiv cs.CL
Voice AI· Important· 🧪 Worth Testing

Medical Mode update reduces drug name transcription errors

AssemblyAI has released an update to its Medical Mode feature designed to improve the transcription of high-stakes clinical terms like medications and diagnoses. The update specifically reduces drug name miss rates by 30% in English and up to 62% in Spanish, while maintaining existing word error rates.

AssemblyAI
Voice AI· Important· 🧪 Worth Testing

AssemblyAI updates streaming diarization to provide post-call accuracy in real-time

AssemblyAI has introduced SpeakerRevision, a feature that delivers a corrected set of speaker labels at the end of a live stream. By utilizing future context unavailable during real-time processing, it achieves accuracy levels comparable to asynchronous transcription, reducing the need for duplicate processing pipelines.

AssemblyAI
Voice AI· Important· 🧪 Worth Testing

Gradium launches Phonon, a lightweight on-device TTS model

Gradium has released Phonon, a suite of compact (65-75 MB) text-to-speech models designed to run entirely on edge devices. It supports five languages, operates offline, and performs competitively against models significantly larger in size.

Hugging Face
Voice AI· Important· 🧪 Worth Testing

Cartesia releases SSM-based voice models on AWS for sub-100ms latency

Cartesia has introduced two models for voice applications: Sonic (text-to-speech) and Ink (speech-to-text). Built on State Space Model (SSM) architecture, these models are designed for ultra-low-latency, real-time performance suitable for production voice agents.

AWS Machine Learning
Voice AI· Important· 🧪 Worth Testing

AssemblyAI launches promptable speech model Universal-3 Pro

AssemblyAI has introduced Universal-3 Pro, a specialized speech-to-text model that allows developers to provide instructions directly to the transcription process. It supports verbatim and clean transcription, custom audio event tagging, and handles code-switching between six languages.

AssemblyAI
Models· 🧪 Worth Testing

IBM releases Granite Switch 4.1.1 with modular updates

IBM has released Granite Switch 4.1.1, a patch update for its modular language model architecture. The update introduces an improved multilingual speech-to-text model and a refined Decision adapter, all achieved without retraining the base model.

Hugging Face
Models· Important· 🧪 Worth Testing

NVIDIA Releases Nemotron 3.5 Transcribe for Multilingual Audio Analysis

NVIDIA has introduced Nemotron 3.5 Transcribe, an open-weight, promptable model for speech-to-text, speaker diarization, and translation. It utilizes a hybrid Mamba-Transformer architecture to achieve industry-leading performance with only 4 billion active parameters.

Hugging Face
Models· Important· 🧪 Worth Testing

LiquidAI releases LFM3 multimodal models for edge devices

LiquidAI has launched LFM3, a new generation of small foundation models built on a shared 2B-parameter backbone. The series includes text, vision, audio, and omni models designed for high-efficiency on-device deployment.

Hugging Face
Voice AI· Important· 🧪 Worth Testing

Audible enhances audiobook synthesis with controllable TTS

Audible has developed a controllable text-to-speech pipeline that uses LLMs to profile characters and attribute dialogue with up to 97% accuracy. By providing 'stage directions' as natural-language captions, the system ensures distinct, consistent character voices over long-form content.

Amazon Science
Voice AI· Important· 🧪 Worth Testing

IBM refreshes Granite Speech lineup: new ASR model, quicker edge transcription, sharper speaker attribution

IBM rolled out three updates to its Granite Speech family: a brand-new 2B speech-language model (granite-speech-5.0-2b), a revised 470M Turbo CTC model built for speed, and an improved 4.1-2B-Plus model tuned for multi-speaker recordings. Together they target high-accuracy transcription/translation, fast on-device recognition, and accurate speaker/timestamp labeling. All three remain under Apache 2.0.

Hugging Face
Voice AI· Important· 🧪 Worth Testing

Mistral launches Voxtral 3 voice stack: speech models plus a real-time agent SDK

Mistral AI released Voxtral 3, a set of speech recognition and speech synthesis models paired with an SDK that lets a single real-time pipeline listen, reason, call tools and speak back. The recognition models (Small, Mini and Realtime Transcribe 3) cover 50+ languages with speaker diarization, keyword-biasing for technical vocabulary, and transcription latency under 500ms, which Mistral claims beats Whisper, Scribe v2 and Gemini 3.5 Transcribe on its benchmarks. The speech synthesis model, Mini TTS 3, handles 35+ languages and 50+ dialects with emotion, pacing and emphasis that adapt to context, priced at $13 per million characters, and a separate Pro Voice feature can reproduce a person's voice and emotional range from just 30 minutes of recorded audio. All components can run through Mistral's EU-hosted API or be deployed on a customer's own servers.

Mistral AI
Voice AI· Important· 🧪 Worth Testing

Deepgram cuts batch transcription costs by porting Nova-3 to AWS Trainium2

AWS and Deepgram jointly rebuilt the Nova-3 speech-to-text model to run natively on Trainium2 chips instead of standard GPU-based PyTorch setups, targeting large offline transcription jobs. The work involved rewriting bottleneck layers as custom kernels and replacing Deepgram's Python serving layer with a Rust runtime calling Trainium directly. The result: a single trn2.3xlarge instance running eight parallel model copies delivers 11-70% more transcribed audio per dollar than Deepgram's prior infrastructure, with similar savings potentially available to customers self-hosting on SageMaker.

AWS Machine Learning