NVIDIA and AI4Bharat Release 2.8 Trillion-Token Indic Pretraining Corpus and Open Data Pipeline
NVIDIA, working with IIT Madras's AI4Bharat lab, is releasing Nemotron-pretraining-Indic-v1, a roughly 2.8 trillion token synthetic pretraining corpus covering 22 Indian languages, along with the open GPU-accelerated pipeline used to generate it. The corpus includes native-script data for all 22 languages and romanized text for twelve, built by translating curated English source material (web text, math, code, Q&A) using NeMo Data Designer and quality-checked with an MQM-inspired rubric. NVIDIA already used a 10-billion-token Indic subset for continued pretraining of Nemotron 3.5 Super VL, which showed measurable gains on Indic benchmarks.