Skip to content
Sunday, 22 November 2026
Tenesys AI News
Subscribe

AI Research

Research papers, benchmarks, architectures, reasoning and training methods.

“Agentai” — 12 articles ✕

Research· Important· 🧪 Worth Testing

MCQ benchmarks artificially mask LLM safety refusals

Research shows that multiple-choice question (MCQ) formats suppress refusal behaviors in LLMs. Experiments on Llama-3.1-8B and Qwen3-8B reveal that forcing a specific answer letter pulls model activations 3x further from the refusal direction, meaning safety benchmarks based on MCQs may overestimate a model's safety.

arXiv cs.CL
Research· Important· 🧪 Worth Testing

Mitigating AI Agent Distraction in Large Codebases

AI agents frequently misinterpret legacy code comments, such as TODOs or FIXMEs, as current instructions. Research shows that 6% of coding sessions result in agents chasing irrelevant, closed, or obsolete bug tickets, leading to increased costs and scope creep.

Google Developers
Research· Important· 🧪 Worth Testing

New BACE method reduces premature AI responses in ambiguous dialogues

Researchers have identified that LLMs suffer from a bias toward providing immediate, often incorrect answers in ambiguous situations instead of asking clarifying questions. They introduced BACE (Belief-Aware Critic with Epistemic tracking), a lightweight method that runs on top of a frozen language model to strategically decide when to request clarification.

arXiv cs.CL
Research· Important· 🧪 Worth Testing

Google releases benchmark for engineering AI agents

Google has introduced Physical-Engineering's Last Exam (PLE), a new benchmark designed to test AI agents' capabilities in mechanical, electrical, civil, and structural engineering. Unlike standard textbook tests, PLE requires agents to process large project folders, including CAD drawings and building codes, to produce compliant, safe deliverables.

Google Cloud AI
Research· Important· 🧪 Worth Testing

Scalable Online Reinforcement Learning for Mobile Interface Agents

Apple researchers have unveiled an online training infrastructure for mobile UI agents that utilizes a two-part architecture to split navigational logic from visual interpretation. By employing step-level feedback provided by an LLM-as-a-Judge and a novel value estimation technique known as GiGPPO, these agents demonstrate significantly better adaptability than static models trained on pre-recorded sequences.

Apple Machine Learning
Research· Important· 🧪 Worth Testing

Current LLMs struggle with shifting user preferences

Recent research reveals that language models fail to adapt to evolving user preferences, instead defaulting to 'exploit-first' behavior. While models can identify the correct strategy in multiple-choice scenarios, they show a 35-46% drop in calibration during open-ended generation, persistently over-relying on stale historical data.

arXiv cs.CL
Research· Important· 🧪 Worth Testing

New Metric Improves Reliability of LLM Agents via Consistency Training

Researchers introduced Recursive Semantic Divergence (RSD), an unsupervised metric that measures and penalizes inconsistency in LLM agent trajectories. By applying RSD as a reward signal during fine-tuning, models exhibit significantly more stable behavior and higher success rates across tool-use benchmarks.

arXiv cs.CL
Research· Important

New research: agents don't 'decide' to obey a prompt injection all at once — the choice unfolds as they reason

A new study questions the common belief that whether an AI agent falls for a prompt injection is settled the moment it reads the malicious text. The researchers built a branching-rollout technique on AgentDojo benchmarks: they replayed an agent's run up to the point it encounters an injected instruction, then let the rest of the episode play out many different ways to measure how much of the final outcome was already locked in versus still open. For a Qwen3 agent running without extended reasoning, 91% of the outcome was already determined at that moment; switching on reasoning ('thinking') cut that figure to 62%, showing the agent's compliance decision often keeps forming well after it first sees the attack. Enabling reasoning also cut Qwen3's susceptibility to successful attacks, but made two other tested models more vulnerable, and benchmark scores for reasoning agents flipped on a rerun as often as 35% of the time.

arXiv cs.CL
Research· Important

Google Research builds AI agents that design and run wet-lab experiments

Google Research describes new work on grounding AI agents in physical lab reality, using a digital-twin framework called VERID to let agents safely plan and execute liquid-handling robot protocols. Combined with closed-loop molecular design cycles for aptamers and proteins, the agents iteratively learned from wet-lab results, improving hit rates over multiple rounds and discovering single-digit nanomolar protein binders for a cancer-related target.

Google Research
Research· Important

IBM shows open-model AI agents can cut chip-design search costs by over 11x

IBM Research used a coordinated multi-agent 'agent factory' to explore hardware design optimizations for a quantum error-correction decoder, comparing proprietary and open frontier models under the same 100-hypothesis search budget. The open-model configuration, run on-premises via IBM's llm-d distributed inference framework, achieved a 52x speedup over baseline for about $26 in inference cost, versus $299 for a proprietary-model configuration that reached a higher 204x speedup. The study highlights that cache-aware routing and efficient serving (98% prefix-cache hit rate, ~$0.11 per million tokens) made the open-model approach dramatically cheaper while still finding valid, compact hardware designs.

IBM Research
Research· Important· 🧪 Worth Testing

GitHub opens up ReviewBench to standardize how AI code reviewers get graded

GitHub has published ReviewBench, a research-preview benchmark designed to compare AI code review agents on a common, auditable standard. The dataset draws on 219 real public pull requests across 19 languages and 187 repositories, chosen to mirror the language and size distribution found across 103.9 million GitHub pull requests. Ground truth comes from a blended pool of human reviewers, static analysis tools, and multiple frontier LLMs, cross-checked by senior engineers who agreed with the benchmark's labels 96.6% of the time. GitHub says it has already used ReviewBench internally to track its own Copilot code review (CCR) product, and that offline benchmark movement reliably predicted the direction of later production A/B tests.

GitHub