Skip to content
Thursday, 19 November 2026
Tenesys AI News
Subscribe

AI Research

Research papers, benchmarks, architectures, reasoning and training methods.

“Inference” — 14 articles ✕

Research· Important· 🧪 Worth Testing

Measuring the cost of LLM refusals via existing logs

A new causal method named RAW (Refusal Aversion in the Wild) uses LLM stochasticity to measure how often user refusals lead to platform abandonment. By analyzing existing interaction logs, you can identify the exact causal effect of refusals on subsequent user re-engagement.

arXiv cs.CL
Research· Important· 🧪 Worth Testing

LLM Behavior Control via Input Optimization

Researchers introduced ActGCG, a method to replicate the effects of activation steering—typically requiring internal model access—by optimizing input suffixes. The technique successfully reproduces specific behaviors like language steering through external text inputs alone.

arXiv cs.CL
Research· Important· 🧪 Worth Testing

Mathematical bounds for LLM alignment discovered

Researchers have established the theoretical maximum reward gain possible during language model alignment under fixed KL-divergence constraints. The findings demonstrate that 'best-of-N' sampling effectively reaches these theoretical bounds, whereas common reinforcement learning approaches like PPO and GRPO currently fall short of optimal performance.

Apple Machine Learning
Research· Important· 🧪 Worth Testing

FlagRot Technique Improves Low-Bit KV Cache Precision

A new method called FlagRot optimizes key-value (KV) cache quantization for long-context models. By selectively rotating key segments based on attention error rather than applying global rotation, it reduces accuracy loss in 2-bit or 4-bit caches.

arXiv cs.CL
Research· Important· 🧪 Worth Testing

Compacting reasoning history reduces AI agent costs

New research shows that reasoning history can be rewritten into compact state records before new tool results are introduced. This method effectively reduces future reasoning tokens and total agent spending by eliminating redundant thoughts without requiring model retraining.

arXiv cs.CL
Research· 🧪 Worth Testing

Optimizing Multi-Draft Speculative Decoding Efficiency

Researchers have developed a method to calculate the theoretical upper bound for Multi-Draft Speculative Decoding (MDSD) efficiency by solving the dual of the optimal transport problem. The study reveals that current verification algorithms often fall short of this theoretical limit, and that sampling without replacement significantly improves acceptance rates compared to sampling with replacement.

arXiv cs.CL
Research· Important· 🧪 Worth Testing

Scalable Online Reinforcement Learning for Mobile Interface Agents

Apple researchers have unveiled an online training infrastructure for mobile UI agents that utilizes a two-part architecture to split navigational logic from visual interpretation. By employing step-level feedback provided by an LLM-as-a-Judge and a novel value estimation technique known as GiGPPO, these agents demonstrate significantly better adaptability than static models trained on pre-recorded sequences.

Apple Machine Learning
Research· Important· 🧪 Worth Testing

Pruning Redundant Attention Heads Reduces LLM Memorisation

Recent research shows that many attention heads in large language models are redundant and specifically contribute to verbatim memorisation of training data. By selectively pruning these heads at inference time, researchers achieved a 91.5% reduction in memorisation events while simultaneously improving performance on standard benchmarks.

arXiv cs.CL
Research· Important· 🧪 Worth Testing

Artifact Distillation: Boosting Small Models Without Fine-Tuning

Researchers have proposed 'Artifact Distillation,' a method where frontier models externalize their reasoning into an 'artifact bank.' Smaller models query this bank at inference time to improve performance without requiring expensive model retraining or gradient updates.

arXiv cs.CL
Research· Important· 🧪 Worth Testing

TraceQuery optimizes inference for diffusion language models

Researchers have developed TraceQuery, a sparse decoding framework that reduces the computational cost of diffusion large language models (dLLMs). By analyzing denoising trajectories, the method selectively recomputes only necessary token representations rather than performing redundant operations, maintaining high accuracy with fewer computations.

arXiv cs.CL
Research· Important· 🧪 Worth Testing

IBM plugin lets vLLM monitor and steer LLMs in real time with minimal slowdown

IBM Research released vLLM-Hook MIA, a plugin for the open-source vLLM inference engine that lets developers observe and intervene in a model's internal states at runtime — for example to decline malicious prompts, flag hallucinated facts, or adjust tone — without disabling vLLM's fast graph-replay execution mode. According to IBM's technical report, MIA is up to 20x faster than the earlier vLLM-Hook and up to 14x faster than methods that insert code directly into the model. In tests, steering an 8B Llama or 1.5B Qwen model added only about 2% latency, and a 70B Llama model added about 6%.

IBM Research
Research· Important· 🧪 Worth Testing

AWS introduces push-based KV transfer to cut latency in disaggregated vLLM inference

AWS released a push-based NIXL connector for vLLM that changes how key-value (KV) cache data moves between prefill and decode instances in disaggregated LLM serving. Instead of the decode instance pulling KV blocks after prefill finishes, the decode instance registers its target memory in advance and the prefill instance writes the KV data directly once ready, moving coordination work off the critical path. Benchmarked on a 1-prefill/1-decode Qwen3-32B deployment using EC2 P5en instances with EFA, push mode delivered up to 3x better time-to-first-token (TTFT) and up to 30% better time-per-output-token (TPOT) than the default pull mode, especially under high request rates.

AWS Machine Learning
Research· 🧪 Worth Testing

AWS open-sources MoE kernel optimizations for vLLM, cuts latency and boosts throughput on H200

AWS detailed three kernel-level optimizations for Mixture-of-Experts (MoE) model serving on Hopper GPUs, implemented in collaboration with the vLLM community and shipped in Amazon SageMaker AI's LMI container. The changes target the inefficiencies of fused MoE computation during low-batch decode, which previously left Tensor Cores underutilized. Combined, the optimizations deliver up to 10% higher throughput and up to 9% faster time-to-first-token (TTFT).

AWS Machine Learning
Research· Important· 🧪 Worth Testing

Amazon's 'Priming' Turns Pretrained Transformers Into Efficient Hybrid Models for Under 1% of Training Cost

Amazon Web Services researchers published a method called 'priming' that converts pretrained Transformer models into hybrid architectures mixing attention and state space model (SSM) layers, reusing existing weights instead of retraining from scratch. The approach cuts pretraining costs by over 99% and lets a 32-billion-parameter hybrid model match its source Transformer's quality while decoding up to twice as fast. Amazon tested three SSM types and released an open-source model zoo, training code, and serving tools under Apache 2.0.

Amazon Science