Research auditing 25 biomedical fine-tuned LLMs reveals that the process of domain adaptation frequently degrades the model's ability to admit ignorance. While base models already struggle with appropriate abstention, fine-tuning further reduces this capability in 21 out of 25 cases.
Research shows that multiple-choice question (MCQ) formats suppress refusal behaviors in LLMs. Experiments on Llama-3.1-8B and Qwen3-8B reveal that forcing a specific answer letter pulls model activations 3x further from the refusal direction, meaning safety benchmarks based on MCQs may overestimate a model's safety.
AI agents frequently misinterpret legacy code comments, such as TODOs or FIXMEs, as current instructions. Research shows that 6% of coding sessions result in agents chasing irrelevant, closed, or obsolete bug tickets, leading to increased costs and scope creep.
Researchers have identified that LLMs suffer from a bias toward providing immediate, often incorrect answers in ambiguous situations instead of asking clarifying questions. They introduced BACE (Belief-Aware Critic with Epistemic tracking), a lightweight method that runs on top of a frozen language model to strategically decide when to request clarification.
A new research paper introduces the Simple Retrieval-Reconstruction Memory (SRRM) benchmark to better evaluate long-context performance in Small Language Models (SLMs). Findings show that as context length increases, a model's ability to reconstruct information degrades much faster than its ability to retrieve specific facts, proving that standard retrieval tests are insufficient for measuring true memory.
A new method called FlagRot optimizes key-value (KV) cache quantization for long-context models. By selectively rotating key segments based on attention error rather than applying global rotation, it reduces accuracy loss in 2-bit or 4-bit caches.
Researchers introduced SLATE, a benchmark for evaluating long-horizon, large-scale API selection in enterprise software environments. Unlike current benchmarks, it forces models to navigate roughly 1,000 tools to complete complex tasks.
New research shows that reasoning history can be rewritten into compact state records before new tool results are introduced. This method effectively reduces future reasoning tokens and total agent spending by eliminating redundant thoughts without requiring model retraining.
Researchers have developed a method to calculate the theoretical upper bound for Multi-Draft Speculative Decoding (MDSD) efficiency by solving the dual of the optimal transport problem. The study reveals that current verification algorithms often fall short of this theoretical limit, and that sampling without replacement significantly improves acceptance rates compared to sampling with replacement.
Google has introduced Physical-Engineering's Last Exam (PLE), a new benchmark designed to test AI agents' capabilities in mechanical, electrical, civil, and structural engineering. Unlike standard textbook tests, PLE requires agents to process large project folders, including CAD drawings and building codes, to produce compliant, safe deliverables.
Recent research shows that commercial AI text humanizers, designed to bypass AI detectors, frequently alter or drop essential facts like names, numbers, and dates. Across 11 tested tools, 35-65% of documents containing factual details suffered from errors during the re-writing process.
Research shows that identical persona instructions produce different behavioral outcomes in LLMs when switching from English to Korean. While buyers followed instructions similarly, sellers became significantly more accommodating and polite in Korean, leading to higher agreement rates and lower prices.
Researchers have introduced Circuit Oracle, a multi-agent LLM system designed to interpret language model computations by analyzing attribution graphs. The tool uses tool calling to answer natural-language queries about target model behavior and safety.