NVIDIA expands Nemotron with new retrieval, reranking and document-parsing models for AI agents
In short: NVIDIA has added three new open, commercially licensed components to its Nemotron Retriever lineup: Embed models (8B and 1B, in BF16 and NVFP4) for searching text and images, a Rerank model (8B) that reorders retrieved evidence by relevance, and a compact 631M Parse model that converts document images into structured text. The goal is to feed AI agents with better, more relevant evidence so they need fewer search rounds and fewer inference tokens to answer a question. NVIDIA reports the new Embed and Rerank models top the ViDoRe v3.1 leaderboards for dense retrieval and reranking respectively.
This summary was generated automatically by AI from Hugging Face's publication. It is our own text, not a copy of the original — facts, figures and quotes belong to the source, linked above and below.
What changed?
- 1New release: Nemotron 3.5 Embed 8B BF16, 1B BF16, 1B NVFP4, Rerank 8B BF16, and Parse at 631M parameters
- 2Embed 8B BF16 tops the ViDoRe v3.1 dense retrieval leaderboard at 61.17 nDCG@10, up from 52.23 for the prior-generation 1B model
- 3Pairing Rerank with Embed cuts agent token cost per query by about 13-14% while improving retrieval accuracy
- 4Rerank 8B leads the new ViDoRe v3.1 reranking leaderboard at 60.49 nDCG@10 versus 59.17 for Qwen-3-VL-Reranker-8B
- 5The NVFP4-quantized 1B Embed model runs 27.1% faster than its BF16 counterpart (128.8 vs 101.3 docs/sec on GB200) while keeping 99.85% of retrieval accuracy
- 6Domain fine-tuning recipes boost nDCG@10 by 18% (1B model) and 3% (8B model) on chart/infographic-heavy data
- 7NVIDIA's NIM serving stack reaches roughly 2.9x the throughput of a comparable vLLM setup (6,031 vs 2,099 requests/sec)
| Parameter | Before | Now |
|---|---|---|
| Lineup | Nemotron 3 Embed / Llama Nemotron Embed VL 1B | Nemotron 3.5 Embed (8B BF16, 1B BF16, 1B NVFP4), Nemotron 3.5 Rerank 8B BF16, Nemotron 3.5 Parse 631M |
| ViDoRe v3.1 dense retrieval score | 52.23 nDCG@10 (Llama Nemotron Embed VL 1B) | 61.17 nDCG@10 (Nemotron 3.5 Embed 8B BF16), rank 1 |
| Reranking score | 59.17 nDCG@10 (Qwen-3-VL-Reranker-8B) | 60.49 nDCG@10 (Nemotron 3.5 Rerank 8B BF16) |
| Agentic token cost | Baseline embedding-only retrieval | ~13-14% lower with reranking added, plus higher accuracy |
| NVFP4 throughput | 101.3 docs/sec (1B BF16 on GB200) | 128.8 docs/sec (1B NVFP4), 99.85% of accuracy retained |
| NIM serving throughput | 2,099 req/sec (vLLM) | 6,031 req/sec (NIM, RTX PRO 6000 Blackwell) |
Why it matters
Teams building RAG-based agents often pay for wasted inference tokens when retrieval surfaces irrelevant or incomplete evidence; better embedding and reranking directly reduces that cost while improving answer quality, and multimodal support means tables, charts and scanned documents become searchable alongside plain text.
What it means for AI agents and contact centers
If a voice agent or contact-center knowledge base relies on retrieval over manuals, call logs, or scanned PDFs, these models are worth comparing on retrieval accuracy for your document types, token-cost savings per query, and throughput/latency under your expected request volume, especially since the parser handles visual layouts like tables and charts that plain-text extraction would miss.
🧪 Worth Testing
The reported token-cost reduction and top retrieval benchmark scores suggest meaningful gains for any RAG-style pipeline, making it worth benchmarking against current retrieval and reranking setups for accuracy, latency, and cost per query.
Sources
- Hugging FaceOfficialPrimary sourceOriginal article →„NVIDIA Nemotron 3.5 Embed, Rerank, and Parse: Advancing Agents with World-Class Retrieval Models“21 Oct 2026, 13:41
- Published by source
- 21 Oct 2026, 13:41
- Found by our system
- 21 Oct 2026, 13:46
- Summary generated
- 21 Oct 2026, 13:49
This article was written by AI from the original source. Facts, numbers and prices come from the source; missing values are marked “Not specified”. Legal notice, copyright and privacy