Siemens uses NVIDIA NeMo Gym to make chip-design agents faster and more accurate
In short: NVIDIA published a case study showing how Siemens EDA applied its open NeMo Gym evaluation library to tighten up an agentic tool used for chip characterization. Instead of checking individual tool calls, the team scored complete final output files with a domain-specific verifier, then used the failure patterns to rewrite the agent's skill instructions and MCP tool descriptions. After several rounds of this loop, a composite accuracy benchmark climbed from roughly 55% to 95%, while the agent needed far fewer steps and far less output to get there. The experiments relied on about 25 evaluation rollouts running Claude Opus 4.8 as the agent model inside Claude Code.
This summary was generated automatically by AI from NVIDIA Developer's publication. It is our own text, not a copy of the original — facts, figures and quotes belong to the source, linked above and below.
What changed?
- 1NeMo Gym grades final deliverables (Liberty characterization files) instead of isolated tool calls, tracking the full chain of actions in each run
- 2A custom verifier checks five gates — coverage, validity, relative validation, accuracy and integrity — and zeroes out the reward if any gate fails
- 3Composite benchmark score rose from about 55% to about 95% after the skill and MCP tool-description rewrites
- 4Agent turns fell from 155 to 53, roughly a 66% drop, and output tokens fell from about 81K to about 29K, roughly a 64% drop
- 5About 25 rollouts were run with Claude Opus 4.8 serving as the fixed model and agent inside Claude Code
- 6NVIDIA frames the same evaluate-then-refine approach as reusable for other engineering domains beyond chip design, such as CAE
Why it matters
It's a concrete demonstration that measuring agents against final, verified outputs — rather than trusting individual tool calls — can sharply cut both error rates and wasted compute/tokens in multistep agentic workflows, a problem that applies well beyond chip design.
What it means for AI agents and contact centers
The underlying method — define a strict verifier for the final result, run repeated rollouts, then tighten skill and tool descriptions based on failure patterns — maps directly onto voice agent development: it offers a template for benchmarking call-handling agents on turn count, token/cost efficiency and task success, which matters for latency and per-minute cost in contact-center deployments.
🧪 Worth Testing
Teams building tool-calling voice or automation agents could borrow this evaluate-refine-rerun loop to measure and reduce agent turns, token usage, and error rates in their own MCP-based tool integrations.
Sources
- NVIDIA DeveloperOfficialPrimary sourceOriginal article →„Optimizing Agent Reliability and Tool Calling Efficiency for Chip Design Using NVIDIA NeMo Gym“14 Oct 2026, 19:10
- Published by source
- 14 Oct 2026, 19:10
- Found by our system
- 14 Oct 2026, 19:11
- Summary generated
- 14 Oct 2026, 19:13
This article was written by AI from the original source. Facts, numbers and prices come from the source; missing values are marked “Not specified”. Legal notice, copyright and privacy