Recent research shows that many attention heads in large language models are redundant and specifically contribute to verbatim memorisation of training data. By selectively pruning these heads at inference time, researchers achieved a 91.5% reduction in memorisation events while simultaneously improving performance on standard benchmarks.
NVIDIA has released AI Cluster Runtime (AICR) to package Dynamo inference infrastructure into validated recipes. This update simplifies deployment by pinning versions and removing mandatory external dependencies like NATS, while introducing new KV-aware routing paths for Kubernetes.
NVIDIA has released a new command-line interface for the TensorRT Edge-LLM runtime, allowing developers to quickly prototype and test AI models on the DGX Spark 64GB workstation. The tool chain facilitates local model serving and supports DSpark speculative decoding to boost generation performance on edge-constrained hardware.
NVIDIA introduced NemoClaw, an open blueprint framework designed to combine agent harnesses with optimized Nemotron models and secure runtimes. It enables developers to deploy agents that are governed by strict security policies while maintaining high performance on owned infrastructure.
Researchers have proposed 'Artifact Distillation,' a method where frontier models externalize their reasoning into an 'artifact bank.' Smaller models query this bank at inference time to improve performance without requiring expensive model retraining or gradient updates.
NVIDIA has introduced Nemotron 3.5 Super VL, a 120B-parameter multimodal model featuring a 1M token context window. The model utilizes a hybrid Mamba-2 and MoE architecture to balance reasoning capabilities with efficient inference, and is now available for deployment on Amazon EKS and SageMaker HyperPod.
Mistral AI has released Mistral Large 4 (Le Chonk), a 1-trillion-parameter open-weight model optimized for coding, cybersecurity, and reasoning. The model introduces three distinct reasoning modes to balance performance with inference costs and features enhanced guardrails for secure agent deployments.
Cisco Talos has identified 'NIK C2,' a sophisticated Go-based Windows malware framework that integrates direct calls to the OpenAI API. The framework enables attackers to perform credential theft, keystroke logging, and lateral movement while using AI to process stolen data in real-time. It notably attempts to use any locally configured OPENAI_API_KEY from the victim's machine to fund its AI operations.
AWS introduced pre-built Ray Train Deep Learning Containers (DLCs) that consolidate PyTorch, CUDA, NCCL, and EFA networking libraries into a single tested image. This approach allows your team to deploy distributed training clusters on Amazon EKS without maintaining custom build pipelines or resolving complex driver compatibility issues.
Xiaomi has released OmniBrain-0, a compact model designed for unified robot control. It uses a progressive training framework and a data flywheel called Rubric-Guided Recursive Self-Improvement to consolidate expert knowledge into a single efficient decision-making agent.
NVIDIA released CUDA 13.4, adding hardware-transparent memory boundary enforcement for GPU sharing. Paired with the new open-source GPU Fractioning Operator and KAI Scheduler v0.18.0, teams can now define specific memory requests, burst limits, and compute modes directly in Kubernetes Pod annotations.
NVIDIA has documented a threefold optimization approach to accelerate MoE inference, tested on the 124-billion-parameter Ling-3.0-flash model. By utilizing specialized Humming kernels, FP8 precision for the LM head, and Multi-Token Prediction, developers can achieve a 42.5% throughput increase and a 24.5% reduction in latency.
Google has updated its A2UI generative UI protocol to include swappable inference formats, addressing latency and reliability issues found in the previous Direct JSON approach. A new format, A2UI Express, reduces token usage by up to 3x and latency by up to 3x by using a concise Python-like syntax.
Researchers have developed TraceQuery, a sparse decoding framework that reduces the computational cost of diffusion large language models (dLLMs). By analyzing denoising trajectories, the method selectively recomputes only necessary token representations rather than performing redundant operations, maintaining high accuracy with fewer computations.
Researchers have introduced the Boa Compressor, an online tool utilizing a Mamba state-space model to compress long-form context into shorter representations in linear time. By distilling knowledge from full-context LLM interactions, this method allows for efficient per-query inference without the massive latency overhead of transformer-based compression.
Cloudflare identified a security flaw in the SGLang inference engine that caused cross-request data exposure for the GLM-5.3 model. Requests exceeding the configured KV cache pool size were incorrectly handled, causing the decode node to generate responses using stale KV cache pages belonging to other users.
Apple has unveiled GHARP, a new method for animating 3D human heads in real time using a two-stage Gaussian Splatting approach. By separating static identity reconstruction from lightweight, mobile-optimized expression prediction, the model achieves state-of-the-art results on the Ava-256 benchmark while remaining computationally efficient.
IBM has updated its software ecosystem to treat the Spyre dataflow accelerator as a first-class PyTorch backend. By leveraging TorchDynamo and TorchInductor, developers can now run models on IBM Z, LinuxONE, and Power systems using standard PyTorch workflows and vLLM plugins.
Cohere has released a public beta of Confidential Computing on Model Vault, running on NVIDIA H100 GPUs via Microsoft Azure. The system encrypts data in use across both CPU and GPU, encrypts the request payload itself using Oblivious HTTP (OHTTP), and uses remote attestation so clients can verify the environment before sending sensitive data. If verification fails, the client refuses to send the protected request. Cohere plans to open-source the confidential inference stack and extend support to NVIDIA Blackwell (B200) GPUs toward general availability.
NVIDIA has introduced Minerva, an operations platform that pulls together telemetry across GPUs, networking gear, storage and running jobs so operators can spot trouble in massive GPU clusters before it causes downtime. Machine learning models build a live map of the hardware and workloads, flag unusual behavior, and bundle related alerts into one incident with a likely cause rather than flooding teams with separate warnings. A built-in assistant lets staff query cluster health in plain language, and the system can link to automation tools to act on fixes. IREN, Aolani, Bitdeer AI, IBM, Jio, Lambda, Quake AI and Tesla are lined up to use it, while Dell, Lenovo and Supermicro will sell it as part of their infrastructure offerings.
NVIDIA has broadened its Cosmos platform, aimed at building 'world models' for physical AI and robotics, with new data curation blueprints, model fine-tuning support and faster deployment pipelines. The update includes five Cosmos Data Factory blueprints for synthetic and augmented training data, expanded Cosmos Framework support for adapting open models such as GR00T, Alpamayo, QWEN and Pi0.5, and upgraded Cosmos 3 NIM microservices for inference. Companies including Porsche, ANYbotics, Ultralytics and ADNOC are already applying these tools to autonomous vehicle data generation, robot anomaly detection, defect inspection and safety monitoring.
AWS published a walkthrough for the Adaptive Agentic Intelligence Framework (AAIF), built on Strands Agents, which routes simple requests to small local models (via Ollama) and only sends complex requests to cloud models through Amazon Bedrock. A lightweight 1B-parameter model (llama3.2:1b) acts as an orchestrator deciding where each request should go, with an example IoT monitoring agent that uses local telemetry retrieval, local vision analysis, and cloud-based expert insights. AWS states that in the hybrid setup, roughly 80 percent of traffic can be handled locally without hitting a paid API, and the architecture can run fully on edge hardware (e.g. Raspberry Pi with 8GB memory), fully in the cloud, or in a hybrid mode.
IBM rolled out three updates to its Granite Speech family: a brand-new 2B speech-language model (granite-speech-5.0-2b), a revised 470M Turbo CTC model built for speed, and an improved 4.1-2B-Plus model tuned for multi-speaker recordings. Together they target high-accuracy transcription/translation, fast on-device recognition, and accurate speaker/timestamp labeling. All three remain under Apache 2.0.
NVIDIA's BioNeMo team, working with the University of Washington's Institute for Protein Design, has introduced RFD4-Proteina, a model that generates both a protein's sequence and its full atomic structure in a single pass instead of handing off to a separate inverse-folding tool. By working with a residue-level representation and a streamlined diffusion process, the model needs only about 60 denoising steps instead of roughly 400, which translates into markedly quicker runs and a much smaller memory footprint than its predecessor RFD3 when generating a 200-residue protein. During generation, the model can also run a reward-guided search that scores and branches partial designs using GPU-accelerated evaluation tools such as OpenFold3 and TMol, steering computation toward the most promising candidates. It ships both as an open-source model on GitHub and as a NIM microservice reachable over HTTP, with toolkit skills available to let automated agents chain generation, scoring and evaluation steps together.