NVIDIA and AI4Bharat Release 2.8 Trillion-Token Indic Pretraining Corpus and Open Data Pipeline

In short: NVIDIA, working with IIT Madras's AI4Bharat lab, is releasing Nemotron-pretraining-Indic-v1, a roughly 2.8 trillion token synthetic pretraining corpus covering 22 Indian languages, along with the open GPU-accelerated pipeline used to generate it. The corpus includes native-script data for all 22 languages and romanized text for twelve, built by translating curated English source material (web text, math, code, Q&A) using NeMo Data Designer and quality-checked with an MQM-inspired rubric. NVIDIA already used a 10-billion-token Indic subset for continued pretraining of Nemotron 3.5 Super VL, which showed measurable gains on Indic benchmarks.

Source: Hugging Face16 Oct 2026, 04:44NVIDIA Original article ↗

This summary was generated automatically by AI from Hugging Face's publication. It is our own text, not a copy of the original — facts, figures and quotes belong to the source, linked above and below.

What changed?

1New dataset: Nemotron-pretraining-Indic-v1, ~2.8 trillion tokens across 22 Indian languages
22,601.0 billion native-script tokens and 224.6 billion romanized tokens (Gemma-4 tokenizer)
3Romanized data available for 12 of the 22 languages, native-script for all 22
4Pipeline uses Gemma-4-31B-it for translation and quality assessment, retaining only outputs scoring 75/100 or higher on an MQM-style rubric
5438.4 billion tokens recovered via a repair-and-reassess step on lower-scoring translations
6Nemotron 3.5 Super VL continued pretraining with 10B Indic tokens improved all five reported Indic benchmarks by 4.98–9.82 percentage points

Why it matters

This gives developers building language models or NLP tools for Indian languages a much larger, auditable training resource and a reusable recipe for generating synthetic data in low-resource languages, which is relevant beyond India for any team working on underrepresented languages.

What it means for AI agents and contact centers

This release does not directly affect voice AI deployments for Lithuanian or contact-center automation, since it targets text pretraining corpora for Indian languages rather than speech models or APIs. However, the open data-generation pipeline (translation, quality scoring, repair) is a useful reference methodology if building or improving synthetic training data for smaller languages like Lithuanian in future LLM or STT fine-tuning work.

Pretraining Data Indic Languages Synthetic Data NeMo Data Designer Open Dataset Low-resource Languages

Sources

Hugging FaceOfficialPrimary source
„Scaling Indic Pretraining: 2.8 Trillion Tokens and an Open Data Generation Pipeline“
16 Oct 2026, 04:44
Original article →

Published by source: 16 Oct 2026, 04:44
Found by our system: 16 Oct 2026, 05:11
Summary generated: 16 Oct 2026, 05:12

This article was written by AI from the original source. Facts, numbers and prices come from the source; missing values are marked “Not specified”. Legal notice, copyright and privacy