Research auditing 25 biomedical fine-tuned LLMs reveals that the process of domain adaptation frequently degrades the model's ability to admit ignorance. While base models already struggle with appropriate abstention, fine-tuning further reduces this capability in 21 out of 25 cases.
Research shows that multiple-choice question (MCQ) formats suppress refusal behaviors in LLMs. Experiments on Llama-3.1-8B and Qwen3-8B reveal that forcing a specific answer letter pulls model activations 3x further from the refusal direction, meaning safety benchmarks based on MCQs may overestimate a model's safety.
AI agents frequently misinterpret legacy code comments, such as TODOs or FIXMEs, as current instructions. Research shows that 6% of coding sessions result in agents chasing irrelevant, closed, or obsolete bug tickets, leading to increased costs and scope creep.
Researchers have identified that LLMs suffer from a bias toward providing immediate, often incorrect answers in ambiguous situations instead of asking clarifying questions. They introduced BACE (Belief-Aware Critic with Epistemic tracking), a lightweight method that runs on top of a frozen language model to strategically decide when to request clarification.
A new method called FlagRot optimizes key-value (KV) cache quantization for long-context models. By selectively rotating key segments based on attention error rather than applying global rotation, it reduces accuracy loss in 2-bit or 4-bit caches.
Researchers introduced SLATE, a benchmark for evaluating long-horizon, large-scale API selection in enterprise software environments. Unlike current benchmarks, it forces models to navigate roughly 1,000 tools to complete complex tasks.
New research shows that reasoning history can be rewritten into compact state records before new tool results are introduced. This method effectively reduces future reasoning tokens and total agent spending by eliminating redundant thoughts without requiring model retraining.
Google has introduced Physical-Engineering's Last Exam (PLE), a new benchmark designed to test AI agents' capabilities in mechanical, electrical, civil, and structural engineering. Unlike standard textbook tests, PLE requires agents to process large project folders, including CAD drawings and building codes, to produce compliant, safe deliverables.
Recent research shows that commercial AI text humanizers, designed to bypass AI detectors, frequently alter or drop essential facts like names, numbers, and dates. Across 11 tested tools, 35-65% of documents containing factual details suffered from errors during the re-writing process.
Research shows that identical persona instructions produce different behavioral outcomes in LLMs when switching from English to Korean. While buyers followed instructions similarly, sellers became significantly more accommodating and polite in Korean, leading to higher agreement rates and lower prices.
Apple researchers have unveiled an online training infrastructure for mobile UI agents that utilizes a two-part architecture to split navigational logic from visual interpretation. By employing step-level feedback provided by an LLM-as-a-Judge and a novel value estimation technique known as GiGPPO, these agents demonstrate significantly better adaptability than static models trained on pre-recorded sequences.
Recent research reveals that language models fail to adapt to evolving user preferences, instead defaulting to 'exploit-first' behavior. While models can identify the correct strategy in multiple-choice scenarios, they show a 35-46% drop in calibration during open-ended generation, persistently over-relying on stale historical data.
Researchers have proposed 'Artifact Distillation,' a method where frontier models externalize their reasoning into an 'artifact bank.' Smaller models query this bank at inference time to improve performance without requiring expensive model retraining or gradient updates.
Researchers have proposed LOTUS, a framework for hallucination detection that uses multilinear low-rank decomposition to extract signals from hidden LLM layers. By analyzing peak activations and cross-layer patterns, the method identifies incorrect responses more effectively than existing pooling-based techniques.
Researchers introduced Recursive Semantic Divergence (RSD), an unsupervised metric that measures and penalizes inconsistency in LLM agent trajectories. By applying RSD as a reward signal during fine-tuning, models exhibit significantly more stable behavior and higher success rates across tool-use benchmarks.
IBM Research released vLLM-Hook MIA, a plugin for the open-source vLLM inference engine that lets developers observe and intervene in a model's internal states at runtime — for example to decline malicious prompts, flag hallucinated facts, or adjust tone — without disabling vLLM's fast graph-replay execution mode. According to IBM's technical report, MIA is up to 20x faster than the earlier vLLM-Hook and up to 14x faster than methods that insert code directly into the model. In tests, steering an 8B Llama or 1.5B Qwen model added only about 2% latency, and a 70B Llama model added about 6%.
Google Research announced the final milestone of its Coral NPU open hardware standard — a 32-bit RISC-V architecture with a new Matrix Execution Unit for up to 1 TOPS at roughly 10mW — plus a v1.0 open-source IREE/MLIR compiler that takes JAX, PyTorch and TensorFlow models directly to the hardware. Synaptics is launching the first commercial silicon built on this standard, including the Astra SR200 microcontroller series (400 GOPS) and new Coralboard SR and SL developer kits, with the SL now generally available via DigiKey. Google is also partnering with VeriSilicon and GlobalFoundries to bring the open RTL into production chips.
A new study questions the common belief that whether an AI agent falls for a prompt injection is settled the moment it reads the malicious text. The researchers built a branching-rollout technique on AgentDojo benchmarks: they replayed an agent's run up to the point it encounters an injected instruction, then let the rest of the episode play out many different ways to measure how much of the final outcome was already locked in versus still open. For a Qwen3 agent running without extended reasoning, 91% of the outcome was already determined at that moment; switching on reasoning ('thinking') cut that figure to 62%, showing the agent's compliance decision often keeps forming well after it first sees the attack. Enabling reasoning also cut Qwen3's susceptibility to successful attacks, but made two other tested models more vulnerable, and benchmark scores for reasoning agents flipped on a rerun as often as 35% of the time.
AWS released a push-based NIXL connector for vLLM that changes how key-value (KV) cache data moves between prefill and decode instances in disaggregated LLM serving. Instead of the decode instance pulling KV blocks after prefill finishes, the decode instance registers its target memory in advance and the prefill instance writes the KV data directly once ready, moving coordination work off the critical path. Benchmarked on a 1-prefill/1-decode Qwen3-32B deployment using EC2 P5en instances with EFA, push mode delivered up to 3x better time-to-first-token (TTFT) and up to 30% better time-per-output-token (TPOT) than the default pull mode, especially under high request rates.
Google Research describes new work on grounding AI agents in physical lab reality, using a digital-twin framework called VERID to let agents safely plan and execute liquid-handling robot protocols. Combined with closed-loop molecular design cycles for aptamers and proteins, the agents iteratively learned from wet-lab results, improving hit rates over multiple rounds and discovering single-digit nanomolar protein binders for a cancer-related target.
IBM Research used a coordinated multi-agent 'agent factory' to explore hardware design optimizations for a quantum error-correction decoder, comparing proprietary and open frontier models under the same 100-hypothesis search budget. The open-model configuration, run on-premises via IBM's llm-d distributed inference framework, achieved a 52x speedup over baseline for about $26 in inference cost, versus $299 for a proprietary-model configuration that reached a higher 204x speedup. The study highlights that cache-aware routing and efficient serving (98% prefix-cache hit rate, ~$0.11 per million tokens) made the open-model approach dramatically cheaper while still finding valid, compact hardware designs.
Amazon Web Services researchers published a method called 'priming' that converts pretrained Transformer models into hybrid architectures mixing attention and state space model (SSM) layers, reusing existing weights instead of retraining from scratch. The approach cuts pretraining costs by over 99% and lets a 32-billion-parameter hybrid model match its source Transformer's quality while decoding up to twice as fast. Amazon tested three SSM types and released an open-source model zoo, training code, and serving tools under Apache 2.0.
GitHub has published ReviewBench, a research-preview benchmark designed to compare AI code review agents on a common, auditable standard. The dataset draws on 219 real public pull requests across 19 languages and 187 repositories, chosen to mirror the language and size distribution found across 103.9 million GitHub pull requests. Ground truth comes from a blended pool of human reviewers, static analysis tools, and multiple frontier LLMs, cross-checked by senior engineers who agreed with the benchmark's labels 96.6% of the time. GitHub says it has already used ReviewBench internally to track its own Copilot code review (CCR) product, and that offline benchmark movement reliably predicted the direction of later production A/B tests.