Researchers have proposed 'Artifact Distillation,' a method where frontier models externalize their reasoning into an 'artifact bank.' Smaller models query this bank at inference time to improve performance without requiring expensive model retraining or gradient updates.
Researchers have proposed LOTUS, a framework for hallucination detection that uses multilinear low-rank decomposition to extract signals from hidden LLM layers. By analyzing peak activations and cross-layer patterns, the method identifies incorrect responses more effectively than existing pooling-based techniques.
Researchers have proposed an embedding-based k-center approach to select smaller, highly representative subsets of training data for Whisper fine-tuning. This method outperforms random sampling by capturing greater linguistic diversity, particularly effective when training budgets are limited.
Researchers introduced Recursive Semantic Divergence (RSD), an unsupervised metric that measures and penalizes inconsistency in LLM agent trajectories. By applying RSD as a reward signal during fine-tuning, models exhibit significantly more stable behavior and higher success rates across tool-use benchmarks.
Recent research on Qwen3.5-27B demonstrates that while activation interventions can force a model to produce a specific target output, they often fail to instill correct underlying reasoning. Researchers found that gains in target selection collapse when facts are recombined, suggesting that forced outputs do not reliably represent valid tool-use condition logic.
IBM Research released vLLM-Hook MIA, a plugin for the open-source vLLM inference engine that lets developers observe and intervene in a model's internal states at runtime — for example to decline malicious prompts, flag hallucinated facts, or adjust tone — without disabling vLLM's fast graph-replay execution mode. According to IBM's technical report, MIA is up to 20x faster than the earlier vLLM-Hook and up to 14x faster than methods that insert code directly into the model. In tests, steering an 8B Llama or 1.5B Qwen model added only about 2% latency, and a 70B Llama model added about 6%.
AWS released a push-based NIXL connector for vLLM that changes how key-value (KV) cache data moves between prefill and decode instances in disaggregated LLM serving. Instead of the decode instance pulling KV blocks after prefill finishes, the decode instance registers its target memory in advance and the prefill instance writes the KV data directly once ready, moving coordination work off the critical path. Benchmarked on a 1-prefill/1-decode Qwen3-32B deployment using EC2 P5en instances with EFA, push mode delivered up to 3x better time-to-first-token (TTFT) and up to 30% better time-per-output-token (TPOT) than the default pull mode, especially under high request rates.
AWS detailed three kernel-level optimizations for Mixture-of-Experts (MoE) model serving on Hopper GPUs, implemented in collaboration with the vLLM community and shipped in Amazon SageMaker AI's LMI container. The changes target the inefficiencies of fused MoE computation during low-batch decode, which previously left Tensor Cores underutilized. Combined, the optimizations deliver up to 10% higher throughput and up to 9% faster time-to-first-token (TTFT).
Apple ML researchers published a NeurIPS paper introducing SELFCOMPACT, a scaffold that lets language model agents decide for themselves when and how to compact their own accumulated context, rather than relying on fixed-interval summarization triggered by a token count. It combines a compaction tool the model can call with a lightweight rubric telling it when to fire (a sub-task is resolved) and when to hold off (mid-derivation or stuck). Tested on six benchmarks and seven models, it matched or beat fixed-interval summarization while using far fewer tokens.
Amazon Web Services researchers published a method called 'priming' that converts pretrained Transformer models into hybrid architectures mixing attention and state space model (SSM) layers, reusing existing weights instead of retraining from scratch. The approach cuts pretraining costs by over 99% and lets a 32-billion-parameter hybrid model match its source Transformer's quality while decoding up to twice as fast. Amazon tested three SSM types and released an open-source model zoo, training code, and serving tools under Apache 2.0.
GitHub has published ReviewBench, a research-preview benchmark designed to compare AI code review agents on a common, auditable standard. The dataset draws on 219 real public pull requests across 19 languages and 187 repositories, chosen to mirror the language and size distribution found across 103.9 million GitHub pull requests. Ground truth comes from a blended pool of human reviewers, static analysis tools, and multiple frontier LLMs, cross-checked by senior engineers who agreed with the benchmark's labels 96.6% of the time. GitHub says it has already used ReviewBench internally to track its own Copilot code review (CCR) product, and that offline benchmark movement reliably predicted the direction of later production A/B tests.