Skip to content
Friday, 20 November 2026
Tenesys AI News
Subscribe

All News

TENESYS AI NEWS tracks the most important AI news and explains what changed, why it matters and whether a technology is worth testing.

#Evaluation — 9 articles ✕

Agents· Important· 🧪 Worth Testing

AWS introduces framework-agnostic evaluation for AI agents

Amazon has released the Universal Agent Evaluation Framework (UAEF), an open-source tool designed to standardize the evaluation of AI agents. It uses adapters to normalize execution traces from various frameworks like LangChain and LangGraph into a single format, enabling consistent measurement across 36 metrics and 7 quality dimensions.

AWS Machine Learning
Agents· Important· 🧪 Worth Testing

Amazon Bedrock AgentCore adds native support for DeepEval and AutoEval

Amazon Bedrock AgentCore now includes a comprehensive catalog of built-in, third-party, and custom evaluators to monitor agent behavior in production. The platform allows you to run metrics from DeepEval and AutoEval in a managed environment, supporting on-demand, online, and batch evaluation workflows.

AWS Machine Learning
Research· 🧪 Worth Testing

New Benchmark Reveals Retrieval Accuracy Doesn't Equal Long-Context Memory

A new research paper introduces the Simple Retrieval-Reconstruction Memory (SRRM) benchmark to better evaluate long-context performance in Small Language Models (SLMs). Findings show that as context length increases, a model's ability to reconstruct information degrades much faster than its ability to retrieve specific facts, proving that standard retrieval tests are insufficient for measuring true memory.

arXiv cs.CL
Agents· Important· 🧪 Worth Testing

Automate Agent Optimization with NVIDIA NeMo

NVIDIA introduced a workflow for turning production agent failures into structured test cases using NeMo Compass and NeMo Eval Author. This process analyzes trace logs to identify recurring issues, generates targeted evaluations, and validates harness improvements.

NVIDIA Developer
Research· Important· 🧪 Worth Testing

Google releases benchmark for engineering AI agents

Google has introduced Physical-Engineering's Last Exam (PLE), a new benchmark designed to test AI agents' capabilities in mechanical, electrical, civil, and structural engineering. Unlike standard textbook tests, PLE requires agents to process large project folders, including CAD drawings and building codes, to produce compliant, safe deliverables.

Google Cloud AI
Agents· Important· 🧪 Worth Testing

Automated Quality Evaluation for AI Agents

Google released a new Agents CLI evaluation namespace designed to help developers perform 'hill-climbing'—an iterative process of improving agent performance through testing, grading, and analysis. The tool enables regression tracking by comparing agent traces across model swaps or prompt changes, surfacing issues like tool-use regressions that standard unit tests miss.

Google Developers
Research· Important· 🧪 Worth Testing

New Metric Improves Reliability of LLM Agents via Consistency Training

Researchers introduced Recursive Semantic Divergence (RSD), an unsupervised metric that measures and penalizes inconsistency in LLM agent trajectories. By applying RSD as a reward signal during fine-tuning, models exhibit significantly more stable behavior and higher success rates across tool-use benchmarks.

arXiv cs.CL
Agents· 🧪 Worth Testing

Siemens uses NVIDIA NeMo Gym to make chip-design agents faster and more accurate

NVIDIA published a case study showing how Siemens EDA applied its open NeMo Gym evaluation library to tighten up an agentic tool used for chip characterization. Instead of checking individual tool calls, the team scored complete final output files with a domain-specific verifier, then used the failure patterns to rewrite the agent's skill instructions and MCP tool descriptions. After several rounds of this loop, a composite accuracy benchmark climbed from roughly 55% to 95%, while the agent needed far fewer steps and far less output to get there. The experiments relied on about 25 evaluation rollouts running Claude Opus 4.8 as the agent model inside Claude Code.

NVIDIA Developer
Research· Important· 🧪 Worth Testing

GitHub opens up ReviewBench to standardize how AI code reviewers get graded

GitHub has published ReviewBench, a research-preview benchmark designed to compare AI code review agents on a common, auditable standard. The dataset draws on 219 real public pull requests across 19 languages and 187 repositories, chosen to mirror the language and size distribution found across 103.9 million GitHub pull requests. Ground truth comes from a blended pool of human reviewers, static analysis tools, and multiple frontier LLMs, cross-checked by senior engineers who agreed with the benchmark's labels 96.6% of the time. GitHub says it has already used ReviewBench internally to track its own Copilot code review (CCR) product, and that offline benchmark movement reliably predicted the direction of later production A/B tests.

GitHub