Skip to content
Thursday, 19 November 2026
Tenesys AI News
Subscribe

AI Research

Research papers, benchmarks, architectures, reasoning and training methods.

“Inference” — 24 articles ✕

Research· Important· 🧪 Worth Testing

Measuring the cost of LLM refusals via existing logs

A new causal method named RAW (Refusal Aversion in the Wild) uses LLM stochasticity to measure how often user refusals lead to platform abandonment. By analyzing existing interaction logs, you can identify the exact causal effect of refusals on subsequent user re-engagement.

arXiv cs.CL
Research· Important· 🧪 Worth Testing

LLM Behavior Control via Input Optimization

Researchers introduced ActGCG, a method to replicate the effects of activation steering—typically requiring internal model access—by optimizing input suffixes. The technique successfully reproduces specific behaviors like language steering through external text inputs alone.

arXiv cs.CL
Research· Important· 🧪 Worth Testing

Mathematical bounds for LLM alignment discovered

Researchers have established the theoretical maximum reward gain possible during language model alignment under fixed KL-divergence constraints. The findings demonstrate that 'best-of-N' sampling effectively reaches these theoretical bounds, whereas common reinforcement learning approaches like PPO and GRPO currently fall short of optimal performance.

Apple Machine Learning
Research· Important· 🧪 Worth Testing

FlagRot Technique Improves Low-Bit KV Cache Precision

A new method called FlagRot optimizes key-value (KV) cache quantization for long-context models. By selectively rotating key segments based on attention error rather than applying global rotation, it reduces accuracy loss in 2-bit or 4-bit caches.

arXiv cs.CL
Research· Important· 🧪 Worth Testing

Compacting reasoning history reduces AI agent costs

New research shows that reasoning history can be rewritten into compact state records before new tool results are introduced. This method effectively reduces future reasoning tokens and total agent spending by eliminating redundant thoughts without requiring model retraining.

arXiv cs.CL
Research· 🧪 Worth Testing

Optimizing Multi-Draft Speculative Decoding Efficiency

Researchers have developed a method to calculate the theoretical upper bound for Multi-Draft Speculative Decoding (MDSD) efficiency by solving the dual of the optimal transport problem. The study reveals that current verification algorithms often fall short of this theoretical limit, and that sampling without replacement significantly improves acceptance rates compared to sampling with replacement.

arXiv cs.CL
Research· Important· 🧪 Worth Testing

Scalable Online Reinforcement Learning for Mobile Interface Agents

Apple researchers have unveiled an online training infrastructure for mobile UI agents that utilizes a two-part architecture to split navigational logic from visual interpretation. By employing step-level feedback provided by an LLM-as-a-Judge and a novel value estimation technique known as GiGPPO, these agents demonstrate significantly better adaptability than static models trained on pre-recorded sequences.

Apple Machine Learning
Research· Important· 🧪 Worth Testing

Pruning Redundant Attention Heads Reduces LLM Memorisation

Recent research shows that many attention heads in large language models are redundant and specifically contribute to verbatim memorisation of training data. By selectively pruning these heads at inference time, researchers achieved a 91.5% reduction in memorisation events while simultaneously improving performance on standard benchmarks.

arXiv cs.CL
Research· Important· 🧪 Worth Testing

Artifact Distillation: Boosting Small Models Without Fine-Tuning

Researchers have proposed 'Artifact Distillation,' a method where frontier models externalize their reasoning into an 'artifact bank.' Smaller models query this bank at inference time to improve performance without requiring expensive model retraining or gradient updates.

arXiv cs.CL
Research· Important· 🧪 Worth Testing

TraceQuery optimizes inference for diffusion language models

Researchers have developed TraceQuery, a sparse decoding framework that reduces the computational cost of diffusion large language models (dLLMs). By analyzing denoising trajectories, the method selectively recomputes only necessary token representations rather than performing redundant operations, maintaining high accuracy with fewer computations.

arXiv cs.CL

Apple Researchers Introduce GHARP for Real-Time 3D Head Animation

Apple has unveiled GHARP, a new method for animating 3D human heads in real time using a two-stage Gaussian Splatting approach. By separating static identity reconstruction from lightweight, mobile-optimized expression prediction, the model achieves state-of-the-art results on the Ava-256 benchmark while remaining computationally efficient.

Apple Machine Learning

NVIDIA and University of Washington Launch RFD4-Proteina for Faster AI-Driven Protein Design

NVIDIA's BioNeMo team, working with the University of Washington's Institute for Protein Design, has introduced RFD4-Proteina, a model that generates both a protein's sequence and its full atomic structure in a single pass instead of handing off to a separate inverse-folding tool. By working with a residue-level representation and a streamlined diffusion process, the model needs only about 60 denoising steps instead of roughly 400, which translates into markedly quicker runs and a much smaller memory footprint than its predecessor RFD3 when generating a 200-residue protein. During generation, the model can also run a reward-guided search that scores and branches partial designs using GPU-accelerated evaluation tools such as OpenFold3 and TMol, steering computation toward the most promising candidates. It ships both as an open-source model on GitHub and as a NIM microservice reachable over HTTP, with toolkit skills available to let automated agents chain generation, scoring and evaluation steps together.

NVIDIA Developer
Research· Important· 🧪 Worth Testing

IBM plugin lets vLLM monitor and steer LLMs in real time with minimal slowdown

IBM Research released vLLM-Hook MIA, a plugin for the open-source vLLM inference engine that lets developers observe and intervene in a model's internal states at runtime — for example to decline malicious prompts, flag hallucinated facts, or adjust tone — without disabling vLLM's fast graph-replay execution mode. According to IBM's technical report, MIA is up to 20x faster than the earlier vLLM-Hook and up to 14x faster than methods that insert code directly into the model. In tests, steering an 8B Llama or 1.5B Qwen model added only about 2% latency, and a 70B Llama model added about 6%.

IBM Research

NVIDIA Unveils Cosmos3-H-Dreams, a Real-Time Generative Simulator for Surgical Robotics

NVIDIA released Cosmos3-H-Dreams, a world foundation model that generates interactive, real-time surgical scenes based on robot kinematics from actual clinical cholecystectomy procedures recorded on the CMR Surgical Versius platform. Built on the 16-billion-parameter Cosmos3-H-Embodied-Nano model and accelerated with NVIDIA's FlashDreams streaming-inference library, it runs on a single RTX PRO 6000 GPU. The system learns visual surgical dynamics directly from video and kinematics data rather than explicit physics modeling, and offers four-step and two-step distilled checkpoints trading quality for speed.

Hugging Face
Research· Important

Google completes open Coral NPU edge-AI architecture and compiler, first commercial chips ship

Google Research announced the final milestone of its Coral NPU open hardware standard — a 32-bit RISC-V architecture with a new Matrix Execution Unit for up to 1 TOPS at roughly 10mW — plus a v1.0 open-source IREE/MLIR compiler that takes JAX, PyTorch and TensorFlow models directly to the hardware. Synaptics is launching the first commercial silicon built on this standard, including the Astra SR200 microcontroller series (400 GOPS) and new Coralboard SR and SL developer kits, with the SL now generally available via DigiKey. Google is also partnering with VeriSilicon and GlobalFoundries to bring the open RTL into production chips.

Google Developers
Research· Important· 🧪 Worth Testing

AWS introduces push-based KV transfer to cut latency in disaggregated vLLM inference

AWS released a push-based NIXL connector for vLLM that changes how key-value (KV) cache data moves between prefill and decode instances in disaggregated LLM serving. Instead of the decode instance pulling KV blocks after prefill finishes, the decode instance registers its target memory in advance and the prefill instance writes the KV data directly once ready, moving coordination work off the critical path. Benchmarked on a 1-prefill/1-decode Qwen3-32B deployment using EC2 P5en instances with EFA, push mode delivered up to 3x better time-to-first-token (TTFT) and up to 30% better time-per-output-token (TPOT) than the default pull mode, especially under high request rates.

AWS Machine Learning

Apple Shows Trillion-Parameter-Class LLMs Running Locally on Mac Studio Clusters with MLX

Apple's MLX team published benchmarks showing large open-weight models — including a near-trillion-parameter MoE model called Muse Spark Open — running on Mac Studio hardware, both on a single machine and distributed across up to four Mac Studios using a new RDMA-over-Thunderbolt driver. The new M5 Ultra chip, with dedicated GPU Neural Accelerators and 1.2 TB/s memory bandwidth, delivers up to 4.6x faster time-to-first-token and 1.6x faster generation speed versus the previous M3 Ultra. For models too large to fit on one machine, MLX now supports pipeline and tensor parallelization across multiple Macs connected via Thunderbolt RDMA, letting a four-Mac-Studio cluster generate over 36 tokens per second on an 816B-parameter model.

Apple Machine Learning
Research· 🧪 Worth Testing

AWS open-sources MoE kernel optimizations for vLLM, cuts latency and boosts throughput on H200

AWS detailed three kernel-level optimizations for Mixture-of-Experts (MoE) model serving on Hopper GPUs, implemented in collaboration with the vLLM community and shipped in Amazon SageMaker AI's LMI container. The changes target the inefficiencies of fused MoE computation during low-batch decode, which previously left Tensor Cores underutilized. Combined, the optimizations deliver up to 10% higher throughput and up to 9% faster time-to-first-token (TTFT).

AWS Machine Learning
Research· Important

IBM shows open-model AI agents can cut chip-design search costs by over 11x

IBM Research used a coordinated multi-agent 'agent factory' to explore hardware design optimizations for a quantum error-correction decoder, comparing proprietary and open frontier models under the same 100-hypothesis search budget. The open-model configuration, run on-premises via IBM's llm-d distributed inference framework, achieved a 52x speedup over baseline for about $26 in inference cost, versus $299 for a proprietary-model configuration that reached a higher 204x speedup. The study highlights that cache-aware routing and efficient serving (98% prefix-cache hit rate, ~$0.11 per million tokens) made the open-model approach dramatically cheaper while still finding valid, compact hardware designs.

IBM Research

Apple Researchers Propose Turning Idle Mac GPUs Into a Shared Inference Cluster

A new Apple research paper introduces Minions, a system that pools unused GPU capacity across Apple Silicon Macs into a single distributed cloud for running machine learning inference. Instead of buying additional GPU hardware, teams could tap spare compute sitting idle on existing macOS devices. The authors say the design handles task scheduling, failure recovery, and efficient use of available resources, offering an alternative when dedicated GPU capacity is scarce or expensive.

Apple Machine Learning
Research· Important· 🧪 Worth Testing

Amazon's 'Priming' Turns Pretrained Transformers Into Efficient Hybrid Models for Under 1% of Training Cost

Amazon Web Services researchers published a method called 'priming' that converts pretrained Transformer models into hybrid architectures mixing attention and state space model (SSM) layers, reusing existing weights instead of retraining from scratch. The approach cuts pretraining costs by over 99% and lets a 32-billion-parameter hybrid model match its source Transformer's quality while decoding up to twice as fast. Amazon tested three SSM types and released an open-source model zoo, training code, and serving tools under Apache 2.0.

Amazon Science

NVIDIA and Princeton Cut Decoding Bottlenecks in Hybrid SSM Models

NVIDIA, working with Princeton's Dao AI Lab, built ReplaySSM, a new way to store state in hybrid models that mix state-space (SSM) layers with attention layers. Instead of rewriting the entire state on every token, the method keeps a periodic checkpoint plus a short list of recent inputs, splitting the job of producing output from the job of updating memory. The result is roughly half the memory traffic, decode kernels running up to 1.80x faster at scale, and speculative decoding throughput more than doubling on 120B-class models, all while keeping accuracy close to full precision. The method already ships inside TensorRT-LLM, vLLM, and FlashInfer, and is running in production for GLM-5.3-Flash and Kimi-K3.

NVIDIA Developer

NVIDIA Opens Vera Rubin Rack Architecture and Pushes Open Standards for AI Data Centers

At the OCP Global Summit, NVIDIA contributed its Vera Rubin rack and compute tray designs to the open MGX architecture and announced a series of partnerships covering AI factory power, cooling and networking standards. Partners include hyperscalers such as Meta, Google, Microsoft and AMD, plus Cisco and Visionbay. The announcements cover rack design specs (MGX 1.2), new open-source software for managing AI data centers (DSX OS components), grid resilience standards, an 800VDC power architecture, new Ethernet switching (Spectrum-6) and a new optical transceiver standard (USFP) with Cisco.

NVIDIA

Google Shows How to Speed Up Video Diffusion Attention on TPUs by 2.4x

Google engineers describe how they implemented sparse spatio-temporal attention for video diffusion models on TPU v6e chips, converting the theoretical sparsity of the Sparse VideoGen (SVG) approach into real hardware speedups. Through a progression of kernel optimizations (full/boundary tile specialization, tile-size tuning, and mask-tile alignment), they reduced attention kernel latency from 96.37ms (naive sparse) to 32.76ms, a 2.40x speedup over dense Splash Attention on a single TPU v6e chip, using 75.6K tokens, 10 heads, and head dimension 128.

Google Developers