NVIDIA has released the initial MLPerf Training v6.1 and Endpoints v1.0 results for its new Vera Rubin architecture. The system demonstrated significant leaps in both training speed and power efficiency for agent-based AI workloads.
Amazon has released a reference architecture that integrates the NVIDIA NeMo Switchyard library with LiteLLM to dynamically route coding agent requests between different models on Bedrock. By using stage or random routing, you can shift traffic between high-capability and high-efficiency models to balance performance and cost.
NVIDIA has released a technical workflow for serving the 753B parameter GLM-5.3 MoE model across two GB300-equipped DGX Stations. By utilizing tensor parallelism (TP=2) over RDMA connections and vLLM, you can overcome single-GPU memory constraints (252 GB HBM3e) for massive frontier-scale models.
The llama.cpp project has officially reached its 1.0.0 milestone, three and a half years after its inception. The release introduces strict semantic versioning to ensure better predictability for developers relying on its C/C++ libraries, libllama and libggml.
NVIDIA has integrated the open-source k8s-aibom project into its AI Cluster Runtime (AICR). This allows platform teams to automatically generate and audit an AI Bill of Materials (AIBOM) for workloads running on Kubernetes, identifying models, inference runtimes, and dependencies.
NVIDIA has released NVIDIA Kubernetes Engine (NVKE) and NVIDIA Kubernetes Recipes (NVKR) as open-source projects in public preview. These tools aim to help providers build, deploy, and manage Kubernetes clusters specifically optimized for GPU-accelerated workloads.
NVIDIA has released an open-source tool, NVIDIA Snapshot, for Kubernetes-based GPU workloads. By checkpointing fully initialized models and their states, the system allows inference replicas to resume in seconds rather than minutes, effectively eliminating repetitive initialization steps.
NVIDIA has contributed Agent Substrate to the CNCF Sandbox to solve the inefficiency of running AI agents on Kubernetes. The platform allows multiplexing many agent sessions onto a small pool of warm worker pods by checkpointing memory and filesystem states during idle periods.
NVIDIA has introduced deployment recipes for four prominent Korean open models, including A.X K2 and Solar Open 2, within the Dynamo framework. These recipes are specifically optimized for multi-node NVIDIA B200 GPU clusters, designed to handle large-scale inference efficiently.
NVIDIA has introduced the Open Agent Safety Platform, featuring the OpenShell runtime to provide sandboxed execution and policy-based governance for enterprise AI agents. Concurrently, LG AI Research has opened API access to K-EXAONE-2.5, a model supported by the new Nemotron-Personas-Korea dataset consisting of 7 million synthetic personas.
NVIDIA has introduced new optimization tools in TensorRT LLM VisualGen for the FLUX 3 Image Klein model. By utilizing NVFP4 quantization, kernel fusion, sparse attention, and step caching, developers can achieve up to 3.46x faster text-to-image generation on NVIDIA Blackwell hardware.
Amazon SageMaker Inference has introduced support for NVIDIA Multi-Instance GPU (MIG), allowing you to partition a single physical GPU into multiple independent units. This hardware-level division provides dedicated compute and memory to each partition, ensuring predictable latency and strong isolation between models hosted on the same accelerator.
NVIDIA has introduced the Vera CPU, specifically engineered to optimize the performance of AI agents that cycle between GPU reasoning and CPU-bound tasks. The architecture aims to solve the power and throughput limitations currently faced by AI factories using standard x86 infrastructure.
NVIDIA introduced four 16B parameter checkpoints within its Cosmos 3 Nano family, designed for interactive world simulation. These models enable developers to generate synthetic multi-sensor driving data, simulate robotic movements, and convert simulator controls into photorealistic video.
NVIDIA has introduced five co-design pillars for its Vera Rubin NVL72 platform, aimed at maximizing throughput in agentic AI workloads. By combining new precision formats like NVFP4, wide expert parallelism, and the Dynamo distributed serving layer, the system treats memory as a unified, tiered resource across GPUs, host CPUs, and specialized storage.
NVIDIA has introduced the AI Factory Operations Agent, a framework built on NemoClaw and OpenShell, designed to automate the management and troubleshooting of large-scale GPU clusters. By leveraging specialized sub-agents and Model Context Protocol (MCP), it enables SREs to perform root cause analysis across hardware, network, and software layers through a unified chat interface.
NVIDIA has launched CUDA 13.5, reaching general availability for its new Vera Rubin GPU architecture. This update introduces significant enhancements to multi-process service (MPS) resource scheduling, locality domain partitioning, and deeper integration with SIMT libraries via CUDA Tile.
NVIDIA introduced a workflow for turning production agent failures into structured test cases using NeMo Compass and NeMo Eval Author. This process analyzes trace logs to identify recurring issues, generates targeted evaluations, and validates harness improvements.
NVIDIA has open-sourced 'NVIDIA Attribution,' a tool designed to correlate logs, metrics, and scheduler states into structured diagnoses for distributed AI training jobs. By identifying root causes and distinguishing them from downstream symptoms (like NCCL errors), it helps infrastructure teams manage large-scale GPU deployments more effectively.
NVIDIA Recovery is a new open-source service built to maintain productive training by automating job-level fault handling across GPU clusters. It replaces manual oversight with a systematic loop that detects errors, identifies causes, and triggers restarts or node exclusions.
NVIDIA has released updated AI Cluster Runtime (AICR) recipes designed to streamline the deployment of Kubernetes on the new Vera Rubin NVL72 hardware. These recipes provide pre-validated configurations for drivers, kernels, and network components, ensuring production-ready clusters from day one.
NVIDIA has introduced Cosmos, a platform offering world foundation models and development tools specifically designed for physical AI applications, such as robotics and autonomous vehicles. The framework includes modular training recipes and NIM microservices for GPU-accelerated inference, enabling the generation of synchronized video and LiDAR observations.
NVIDIA has released Workload Map, a tool designed to provide a common way for Kubernetes schedulers and platforms to interpret diverse distributed AI workloads. By using declarative YAML definitions, it eliminates the need for developers to build custom parsing adapters for every new framework like JobSet, Ray, or KServe.
NVIDIA has released AI Cluster Runtime (AICR) to package Dynamo inference infrastructure into validated recipes. This update simplifies deployment by pinning versions and removing mandatory external dependencies like NATS, while introducing new KV-aware routing paths for Kubernetes.