Skip to content
Thursday, 19 November 2026
Tenesys AI News
Subscribe

All News

TENESYS AI NEWS tracks the most important AI news and explains what changed, why it matters and whether a technology is worth testing.

“Inference” — 107 articles ✕

Research· Important· 🧪 Worth Testing

Pruning Redundant Attention Heads Reduces LLM Memorisation

Recent research shows that many attention heads in large language models are redundant and specifically contribute to verbatim memorisation of training data. By selectively pruning these heads at inference time, researchers achieved a 91.5% reduction in memorisation events while simultaneously improving performance on standard benchmarks.

arXiv cs.CL
Automation· Important· 🧪 Worth Testing

Streamlining Kubernetes Inference Deployments with NVIDIA Dynamo 1.4.2

NVIDIA has released AI Cluster Runtime (AICR) to package Dynamo inference infrastructure into validated recipes. This update simplifies deployment by pinning versions and removing mandatory external dependencies like NATS, while introducing new KV-aware routing paths for Kubernetes.

NVIDIA Developer
Models· Important· 🧪 Worth Testing

NVIDIA Simplifies Edge AI Deployment with TensorRT Edge-LLM

NVIDIA has released a new command-line interface for the TensorRT Edge-LLM runtime, allowing developers to quickly prototype and test AI models on the DGX Spark 64GB workstation. The tool chain facilitates local model serving and supports DSpark speculative decoding to boost generation performance on edge-constrained hardware.

NVIDIA Developer
Agents· Important· 🧪 Worth Testing

NVIDIA Releases NemoClaw for Optimized and Secure Agent Deployment

NVIDIA introduced NemoClaw, an open blueprint framework designed to combine agent harnesses with optimized Nemotron models and secure runtimes. It enables developers to deploy agents that are governed by strict security policies while maintaining high performance on owned infrastructure.

NVIDIA Developer
Research· Important· 🧪 Worth Testing

Artifact Distillation: Boosting Small Models Without Fine-Tuning

Researchers have proposed 'Artifact Distillation,' a method where frontier models externalize their reasoning into an 'artifact bank.' Smaller models query this bank at inference time to improve performance without requiring expensive model retraining or gradient updates.

arXiv cs.CL
Models· Important· 🧪 Worth Testing

NVIDIA releases multimodal Nemotron 3.5 Super VL for AWS EKS

NVIDIA has introduced Nemotron 3.5 Super VL, a 120B-parameter multimodal model featuring a 1M token context window. The model utilizes a hybrid Mamba-2 and MoE architecture to balance reasoning capabilities with efficient inference, and is now available for deployment on Amazon EKS and SageMaker HyperPod.

AWS Machine Learning
BreakingModels· 🧪 Worth Testing

Mistral AI Launches Mistral Large 4: A 1T Parameter Open-Weight Model

Mistral AI has released Mistral Large 4 (Le Chonk), a 1-trillion-parameter open-weight model optimized for coding, cybersecurity, and reasoning. The model introduces three distinct reasoning modes to balance performance with inference costs and features enhanced guardrails for secure agent deployments.

Mistral AI

New AI-Enabled NIK C2 Malware Framework Discovered

Cisco Talos has identified 'NIK C2,' a sophisticated Go-based Windows malware framework that integrates direct calls to the OpenAI API. The framework enables attackers to perform credential theft, keystroke logging, and lateral movement while using AI to process stolen data in real-time. It notably attempts to use any locally configured OPENAI_API_KEY from the victim's machine to fund its AI operations.

Cisco Talos
Automation· Important· 🧪 Worth Testing

AWS launches optimized Ray Train Deep Learning Containers for EKS

AWS introduced pre-built Ray Train Deep Learning Containers (DLCs) that consolidate PyTorch, CUDA, NCCL, and EFA networking libraries into a single tested image. This approach allows your team to deploy distributed training clusters on Amazon EKS without maintaining custom build pipelines or resolving complex driver compatibility issues.

AWS Machine Learning

Xiaomi unveils OmniBrain-0 for embodied robotics

Xiaomi has released OmniBrain-0, a compact model designed for unified robot control. It uses a progressive training framework and a data flywheel called Rubric-Guided Recursive Self-Improvement to consolidate expert knowledge into a single efficient decision-making agent.

arXiv cs.CL
Models· Important· 🧪 Worth Testing

Optimizing GPU Infrastructure with CUDA 13.4 Memory Enforcement

NVIDIA released CUDA 13.4, adding hardware-transparent memory boundary enforcement for GPU sharing. Paired with the new open-source GPU Fractioning Operator and KAI Scheduler v0.18.0, teams can now define specific memory requests, burst limits, and compute modes directly in Kubernetes Pod annotations.

NVIDIA Developer
Models· Important· 🧪 Worth Testing

Boosting Large-Scale MoE Model Throughput on NVIDIA Hardware

NVIDIA has documented a threefold optimization approach to accelerate MoE inference, tested on the 124-billion-parameter Ling-3.0-flash model. By utilizing specialized Humming kernels, FP8 precision for the LM head, and Multi-Token Prediction, developers can achieve a 42.5% throughput increase and a 24.5% reduction in latency.

NVIDIA Developer
Models· Important· 🧪 Worth Testing

Google updates A2UI with swappable inference formats and Express mode

Google has updated its A2UI generative UI protocol to include swappable inference formats, addressing latency and reliability issues found in the previous Direct JSON approach. A new format, A2UI Express, reduces token usage by up to 3x and latency by up to 3x by using a concise Python-like syntax.

Google Developers
Research· Important· 🧪 Worth Testing

TraceQuery optimizes inference for diffusion language models

Researchers have developed TraceQuery, a sparse decoding framework that reduces the computational cost of diffusion large language models (dLLMs). By analyzing denoising trajectories, the method selectively recomputes only necessary token representations rather than performing redundant operations, maintaining high accuracy with fewer computations.

arXiv cs.CL
Models· Important· 🧪 Worth Testing

Boa Compressor Brings Online Long-Context Compression to LLMs

Researchers have introduced the Boa Compressor, an online tool utilizing a Mamba state-space model to compress long-form context into shorter representations in linear time. By distilling knowledge from full-context LLM interactions, this method allows for efficient per-query inference without the massive latency overhead of transformer-based compression.

arXiv cs.CL

Apple Researchers Introduce GHARP for Real-Time 3D Head Animation

Apple has unveiled GHARP, a new method for animating 3D human heads in real time using a two-stage Gaussian Splatting approach. By separating static identity reconstruction from lightweight, mobile-optimized expression prediction, the model achieves state-of-the-art results on the Ava-256 benchmark while remaining computationally efficient.

Apple Machine Learning
Models· Important· 🧪 Worth Testing

IBM Spyre accelerator gains native PyTorch and vLLM integration

IBM has updated its software ecosystem to treat the Spyre dataflow accelerator as a first-class PyTorch backend. By leveraging TorchDynamo and TorchInductor, developers can now run models on IBM Z, LinuxONE, and Power systems using standard PyTorch workflows and vLLM plugins.

IBM Research
Cyber Security· Important· 🧪 Worth Testing

Cohere launches confidential computing for Model Vault AI inference

Cohere has released a public beta of Confidential Computing on Model Vault, running on NVIDIA H100 GPUs via Microsoft Azure. The system encrypts data in use across both CPU and GPU, encrypts the request payload itself using Oblivious HTTP (OHTTP), and uses remote attestation so clients can verify the environment before sending sensitive data. If verification fails, the client refuses to send the protected request. Cohere plans to open-source the confidential inference stack and extend support to NVIDIA Blackwell (B200) GPUs toward general availability.

Cohere
Automation· Important

NVIDIA debuts Minerva, a monitoring platform for large-scale GPU clusters

NVIDIA has introduced Minerva, an operations platform that pulls together telemetry across GPUs, networking gear, storage and running jobs so operators can spot trouble in massive GPU clusters before it causes downtime. Machine learning models build a live map of the hardware and workloads, flag unusual behavior, and bundle related alerts into one incident with a likely cause rather than flooding teams with separate warnings. A built-in assistant lets staff query cluster health in plain language, and the system can link to automation tools to act on fixes. IREN, Aolani, Bitdeer AI, IBM, Jio, Lambda, Quake AI and Tesla are lined up to use it, while Dell, Lenovo and Supermicro will sell it as part of their infrastructure offerings.

NVIDIA

NVIDIA Adds Data, Training and Deployment Tools to Its Cosmos World-Model Platform

NVIDIA has broadened its Cosmos platform, aimed at building 'world models' for physical AI and robotics, with new data curation blueprints, model fine-tuning support and faster deployment pipelines. The update includes five Cosmos Data Factory blueprints for synthetic and augmented training data, expanded Cosmos Framework support for adapting open models such as GR00T, Alpamayo, QWEN and Pi0.5, and upgraded Cosmos 3 NIM microservices for inference. Companies including Porsche, ANYbotics, Ultralytics and ADNOC are already applying these tools to autonomous vehicle data generation, robot anomaly detection, defect inspection and safety monitoring.

NVIDIA
Agents· 🧪 Worth Testing

AWS shows hybrid local-plus-cloud agent framework to cut LLM costs

AWS published a walkthrough for the Adaptive Agentic Intelligence Framework (AAIF), built on Strands Agents, which routes simple requests to small local models (via Ollama) and only sends complex requests to cloud models through Amazon Bedrock. A lightweight 1B-parameter model (llama3.2:1b) acts as an orchestrator deciding where each request should go, with an example IoT monitoring agent that uses local telemetry retrieval, local vision analysis, and cloud-based expert insights. AWS states that in the hybrid setup, roughly 80 percent of traffic can be handled locally without hitting a paid API, and the architecture can run fully on edge hardware (e.g. Raspberry Pi with 8GB memory), fully in the cloud, or in a hybrid mode.

AWS Machine Learning
Voice AI· Important· 🧪 Worth Testing

IBM refreshes Granite Speech lineup: new ASR model, quicker edge transcription, sharper speaker attribution

IBM rolled out three updates to its Granite Speech family: a brand-new 2B speech-language model (granite-speech-5.0-2b), a revised 470M Turbo CTC model built for speed, and an improved 4.1-2B-Plus model tuned for multi-speaker recordings. Together they target high-accuracy transcription/translation, fast on-device recognition, and accurate speaker/timestamp labeling. All three remain under Apache 2.0.

Hugging Face

NVIDIA and University of Washington Launch RFD4-Proteina for Faster AI-Driven Protein Design

NVIDIA's BioNeMo team, working with the University of Washington's Institute for Protein Design, has introduced RFD4-Proteina, a model that generates both a protein's sequence and its full atomic structure in a single pass instead of handing off to a separate inverse-folding tool. By working with a residue-level representation and a streamlined diffusion process, the model needs only about 60 denoising steps instead of roughly 400, which translates into markedly quicker runs and a much smaller memory footprint than its predecessor RFD3 when generating a 200-residue protein. During generation, the model can also run a reward-guided search that scores and branches partial designs using GPU-accelerated evaluation tools such as OpenFold3 and TMol, steering computation toward the most promising candidates. It ships both as an open-source model on GitHub and as a NIM microservice reachable over HTTP, with toolkit skills available to let automated agents chain generation, scoring and evaluation steps together.

NVIDIA Developer