NVIDIA has launched CUDA 13.5, reaching general availability for its new Vera Rubin GPU architecture. This update introduces significant enhancements to multi-process service (MPS) resource scheduling, locality domain partitioning, and deeper integration with SIMT libraries via CUDA Tile.
NVIDIA has introduced Cosmos, a platform offering world foundation models and development tools specifically designed for physical AI applications, such as robotics and autonomous vehicles. The framework includes modular training recipes and NIM microservices for GPU-accelerated inference, enabling the generation of synchronized video and LiDAR observations.
Amazon has published a workflow for using Sequence-Level Knowledge Distillation (SeqKD) to fine-tune smaller models on trajectories generated by frontier models. By using a 'teacher' model to generate successful tool-use trajectories and filtering them with a programmatic scorer, teams can train compact 'student' models that perform complex reasoning at a fraction of the cost.
NVIDIA has released a new command-line interface for the TensorRT Edge-LLM runtime, allowing developers to quickly prototype and test AI models on the DGX Spark 64GB workstation. The tool chain facilitates local model serving and supports DSpark speculative decoding to boost generation performance on edge-constrained hardware.
NVIDIA has introduced Nemotron 3.5 Super VL, a 120B-parameter multimodal model featuring a 1M token context window. The model utilizes a hybrid Mamba-2 and MoE architecture to balance reasoning capabilities with efficient inference, and is now available for deployment on Amazon EKS and SageMaker HyperPod.
Mistral AI has released Mistral Large 4 (Le Chonk), a 1-trillion-parameter open-weight model optimized for coding, cybersecurity, and reasoning. The model introduces three distinct reasoning modes to balance performance with inference costs and features enhanced guardrails for secure agent deployments.
NVIDIA released CUDA 13.4, adding hardware-transparent memory boundary enforcement for GPU sharing. Paired with the new open-source GPU Fractioning Operator and KAI Scheduler v0.18.0, teams can now define specific memory requests, burst limits, and compute modes directly in Kubernetes Pod annotations.
NVIDIA has documented a threefold optimization approach to accelerate MoE inference, tested on the 124-billion-parameter Ling-3.0-flash model. By utilizing specialized Humming kernels, FP8 precision for the LM head, and Multi-Token Prediction, developers can achieve a 42.5% throughput increase and a 24.5% reduction in latency.
Google has updated its A2UI generative UI protocol to include swappable inference formats, addressing latency and reliability issues found in the previous Direct JSON approach. A new format, A2UI Express, reduces token usage by up to 3x and latency by up to 3x by using a concise Python-like syntax.
Researchers have introduced the Boa Compressor, an online tool utilizing a Mamba state-space model to compress long-form context into shorter representations in linear time. By distilling knowledge from full-context LLM interactions, this method allows for efficient per-query inference without the massive latency overhead of transformer-based compression.
IBM has updated its software ecosystem to treat the Spyre dataflow accelerator as a first-class PyTorch backend. By leveraging TorchDynamo and TorchInductor, developers can now run models on IBM Z, LinuxONE, and Power systems using standard PyTorch workflows and vLLM plugins.
NVIDIA has broadened its Cosmos platform, aimed at building 'world models' for physical AI and robotics, with new data curation blueprints, model fine-tuning support and faster deployment pipelines. The update includes five Cosmos Data Factory blueprints for synthetic and augmented training data, expanded Cosmos Framework support for adapting open models such as GR00T, Alpamayo, QWEN and Pi0.5, and upgraded Cosmos 3 NIM microservices for inference. Companies including Porsche, ANYbotics, Ultralytics and ADNOC are already applying these tools to autonomous vehicle data generation, robot anomaly detection, defect inspection and safety monitoring.
NVIDIA has added three new open, commercially licensed components to its Nemotron Retriever lineup: Embed models (8B and 1B, in BF16 and NVFP4) for searching text and images, a Rerank model (8B) that reorders retrieved evidence by relevance, and a compact 631M Parse model that converts document images into structured text. The goal is to feed AI agents with better, more relevant evidence so they need fewer search rounds and fewer inference tokens to answer a question. NVIDIA reports the new Embed and Rerank models top the ViDoRe v3.1 leaderboards for dense retrieval and reranking respectively.
AI research company i14 partnered with AWS to port its Laplace 3B model, a 24-layer architecture built on Gated Linear Attention (GLA) instead of standard transformer attention, onto AWS Trainium2 using native PyTorch (TorchNeuron Native). Rather than a KV cache that expands with every token, the model keeps a constant 29.5 MB recurrent state per sequence. On a single trn2.48xlarge instance, the team reached about 120,000 training tokens per second and served 11,544 output tokens per second across 1,280 simultaneous sequences through an unmodified vLLM engine.