NVIDIA, IIT Madras Build 2.8 Trillion Token Indic AI Corpus
NVIDIA and IIT Madras detail a 2.8 trillion token Indic corpus for 22 languages, plus an open GPU pipeline, ahead of its planned release later in 2026.
Summary
NVIDIA, IIT Madras and AI4Bharat announced Nemotron-pretraining-Indic-v1 on October 16, 2026, with release planned soon. The corpus contains 2,601.0 billion native-script tokens and 224.6 billion romanized tokens, measured with the Gemma-4 tokenizer, across all 22 constitutionally recognized Indian languages. Those languages were the first languages of 97% of Indians, nearly 1.2 billion people, in the 2011 Census. Twelve languages include both scripts, producing 34 language-script configurations. The translated material comes from 15 Nemotron-CC-v2 subsets spanning web text, questions and answers, mathematics and code, with every output linked to its source.
The open, GPU-accelerated pipeline uses NeMo Data Designer, Gemma-4-31B-it and Bodhan AI's Indic Translate for document translation, scoring and repair while preserving code, equations, tables, URLs and formatting. Outputs must score at least 75 out of 100, and feedback-guided repairs recovered 438.4 billion tokens. Nemotron 3.5 Super VL subsequently used 10 billion Indic tokens within a 100 billion token continued-pretraining stage, improving all five reported Indic benchmarks by 4.98 to 9.82 percentage points. The released model beat Nemotron 3 Super on five of six task benchmarks and all four IndiVibe dimensions, although IFEval 1.5 fell 2.04 points. Automated assessment, variable human-review coverage, translator-as-judge use and absent semantic deduplication leave errors and cultural gaps possible. NVIDIA plans future Nemotron use, fuller corpus studies and an open Indic-LLM-Eval-Suite with AI4Bharat.
Positives
- 2,825.6 billion retained tokens cover all 22 constitutionally recognized Indian languages, including 224.6 billion romanized tokens.
- 438.4 billion tokens were recovered through feedback-guided repair after initially scoring below the 75-point acceptance threshold.
- 10 billion Indic tokens lifted Nemotron 3.5 Super VL by 4.98 to 9.82 percentage points across five benchmarks.
- 34 language-script configurations give 12 languages both native-script and Latin-script training exposure.
- Open NeMo Data Designer workflows let developers adapt prompts, quality checks, document windowing and script conversion to their own sources.
Risks & concerns
- IFEval 1.5 instruction-following performance declined 2.04 points from Nemotron 3 Super to Nemotron 3.5 Super VL.
- Gemma-4-31B-it serves as both translator and judge in the 12-language workflow, weakening assessment independence.
- Human-review coverage varies by language, while the 10-language workflow still needs broader native-speaker validation.
- Source-ID checks remove processing duplicates but do not semantically deduplicate similar translated content.
- English-derived translations cannot guarantee Indian cultural knowledge and do not replace naturally authored Indic text.