Deepgram cuts batch transcription costs by porting Nova-3 to AWS Trainium2
In short: AWS and Deepgram jointly rebuilt the Nova-3 speech-to-text model to run natively on Trainium2 chips instead of standard GPU-based PyTorch setups, targeting large offline transcription jobs. The work involved rewriting bottleneck layers as custom kernels and replacing Deepgram's Python serving layer with a Rust runtime calling Trainium directly. The result: a single trn2.3xlarge instance running eight parallel model copies delivers 11-70% more transcribed audio per dollar than Deepgram's prior infrastructure, with similar savings potentially available to customers self-hosting on SageMaker.
This summary was generated automatically by AI from AWS Machine Learning's publication. It is our own text, not a copy of the original — facts, figures and quotes belong to the source, linked above and below.
What changed?
- 1Nova-3 fully recompiled for Trainium2 using TorchNeuron, with zero operations falling back to CPU
- 2Hand-written NKI kernels for FFN, depthwise conv1d and conv2d layers produced up to 50x speedup on convolution steps at smaller batch sizes
- 3Python-based serving replaced with a lightweight Rust runtime that calls the Neuron Runtime API directly, cutting interpreter and lock overhead
- 4A single trn2.3xlarge instance now hosts eight parallel model replicas, giving a 7.5% throughput edge over the prior multiprocessing Python setup
- 5Net result: 11-70% more audio transcribed per dollar on offline batch workloads versus Deepgram's previous compute
| Parameter | Before | Now |
|---|---|---|
| Hardware target | Standard PyTorch on prior compute options | AWS Trainium2 (trn2.3xlarge) via Neuron/TorchNeuron |
| Throughput per dollar | Baseline compute options Deepgram previously ran | 11-70% more audio processed per dollar |
| Serving runtime | Python multiprocessing/multithreaded data-parallel runtime | Custom Rust runtime, 7.5% faster than Python MP baseline at max batch size |
| Replicas per device | Not specified | 8 model replicas (DP8) on one trn2.3xlarge instance |
Why it matters
For any team running large-scale offline transcription - QA review, compliance archiving, medical or call-center documentation - this shows a path to materially lower per-minute transcription costs by shifting workloads to purpose-built AI chips rather than general GPUs, without touching model accuracy.
What it means for AI agents and contact centers
If batch transcription volume for call analytics or QA is significant, tracking whether Deepgram passes these cost reductions through to API pricing, or evaluating a self-hosted Nova-3-on-Trainium2 deployment via SageMaker, could lower per-minute STT costs; worth benchmarking against current providers on cost-per-hour and word error rate.
🧪 Worth Testing
The documented 11-70% cost-per-audio-hour improvement on real batch transcription workloads is a concrete, measurable efficiency gain worth validating against your own call volumes and accuracy requirements.
Sources
- AWS Machine LearningOfficialPrimary sourceOriginal article →„How Deepgram optimized voice AI at scale with AWS Trainium“19 Oct 2026, 19:23
- Published by source
- 19 Oct 2026, 19:23
- Found by our system
- 19 Oct 2026, 19:43
- Summary generated
- 19 Oct 2026, 19:46
This article was written by AI from the original source. Facts, numbers and prices come from the source; missing values are marked “Not specified”. Legal notice, copyright and privacy