The llama.cpp project has officially reached its 1.0.0 milestone, three and a half years after its inception. The release introduces strict semantic versioning to ensure better predictability for developers relying on its C/C++ libraries, libllama and libggml.
NVIDIA has released NVIDIA Kubernetes Engine (NVKE) and NVIDIA Kubernetes Recipes (NVKR) as open-source projects in public preview. These tools aim to help providers build, deploy, and manage Kubernetes clusters specifically optimized for GPU-accelerated workloads.
Nod is a new open-source, local-first agent application that executes tasks by operating within a sandboxed virtual machine on your computer. Unlike chat-based AI, Nod interacts directly with web interfaces and applications to perform multi-step jobs independently.
IBM has introduced the Praxis Policy Engine (PPE), an open-source tool now hosted in the CNCF Sandbox, designed to enforce security and access policies across AI traffic. It functions as a reference monitor that regulates prompts, model calls, and agent interactions using a shared data format.
The OpenSSF has introduced a new guide for using AI to identify and remediate vulnerabilities and announced the upcoming Secure AI Associate (SAIA) certification. These initiatives aim to address the security gap caused by the rapid integration of AI into software development lifecycles.
NVIDIA has introduced deployment recipes for four prominent Korean open models, including A.X K2 and Solar Open 2, within the Dynamo framework. These recipes are specifically optimized for multi-node NVIDIA B200 GPU clusters, designed to handle large-scale inference efficiently.
Gradium has released Phonon, a suite of compact (65-75 MB) text-to-speech models designed to run entirely on edge devices. It supports five languages, operates offline, and performs competitively against models significantly larger in size.
IBM rolled out three updates to its Granite Speech family: a brand-new 2B speech-language model (granite-speech-5.0-2b), a revised 470M Turbo CTC model built for speed, and an improved 4.1-2B-Plus model tuned for multi-speaker recordings. Together they target high-accuracy transcription/translation, fast on-device recognition, and accurate speaker/timestamp labeling. All three remain under Apache 2.0.
IBM Research released vLLM-Hook MIA, a plugin for the open-source vLLM inference engine that lets developers observe and intervene in a model's internal states at runtime — for example to decline malicious prompts, flag hallucinated facts, or adjust tone — without disabling vLLM's fast graph-replay execution mode. According to IBM's technical report, MIA is up to 20x faster than the earlier vLLM-Hook and up to 14x faster than methods that insert code directly into the model. In tests, steering an 8B Llama or 1.5B Qwen model added only about 2% latency, and a 70B Llama model added about 6%.
Red Hat, IBM, and Google announced llm-d 1.0, a Kubernetes-native distributed inference serving system that moves beyond proof-of-concept performance tricks into full production readiness. The release adds multi-tenant quality of service, zero-downtime operations, and agent-aware serving so the same endpoint can handle both human users and autonomous agents efficiently. In one test, llm-d's prefix-aware router captured 97% of available cache reuse on an agentic coding workload, and IBM/Red Hat ran 3,000 concurrent software-engineering agents for 5 to 10 times less cost than using the same model as a hosted service.
IBM has released Granite Switch 4.1 in 8B and 30B sizes, a modular system that runs chat, decision-making and speech transcription from one hosted endpoint instead of separate models. The update adds a general-purpose Decision adapter, a specialized Guardian adapter for safety-related classifications, and an embedded ASR model, Granite-Speech-5.0-Turbo-CTC, which tops the OpenASR Leaderboard. Despite bundling three capabilities, the combined model uses roughly the same VRAM as the base model alone.
NVIDIA has released the full Run:ai platform as open source, adding it as a foundation of the NVIDIA DSX AI Factory Platform for AI workload and GPU orchestration on Kubernetes. The release includes four modular components — KAI Scheduler, GPU Fractioning Operator, KAI Resource Management, and Workload Map — that can be adopted together or independently. Eight ecosystem partners, including Crusoe, Mirantis, Nebius, OpenNebula Systems, Rafay, Saturn Cloud, Spectro Cloud, and vCluster, are integrating the platform into their own offerings.
Amazon Web Services researchers published a method called 'priming' that converts pretrained Transformer models into hybrid architectures mixing attention and state space model (SSM) layers, reusing existing weights instead of retraining from scratch. The approach cuts pretraining costs by over 99% and lets a 32-billion-parameter hybrid model match its source Transformer's quality while decoding up to twice as fast. Amazon tested three SSM types and released an open-source model zoo, training code, and serving tools under Apache 2.0.
Mistral AI has released Voxtral Mini 4B Realtime Arabic, a streaming speech-to-text model fine-tuned for Arabic dialects, Modern Standard Arabic, and code-switching between languages. The 4.4-billion-parameter model processes 16 kHz audio in real time with a configurable transcription delay, achieving an 8.82% average Character Error Rate across seven Arabic benchmarks at a 480ms delay — close to the offline Voxtral Transcribe Arabic model's 7.91% CER. It was co-developed with Morocco's digital transition ministry as part of a sovereign AI partnership and is released under Apache 2.0.
Google released ML Drift, an open-source, cross-platform GPU compute engine for on-device ML inference, replacing the legacy TFLite GPU delegate. It unifies shader handling across OpenGL ES, OpenCL, Metal and WebGPU, adds 5D tensor support, and introduces stage-aware optimizations for LLM prefill/decode. It is already running in production in Chrome, YouTube Shorts, Google Photos, Meet and partner apps from Adobe and Snap.
The Technology Innovation Institute (TII) has released Falcon-ASR, a 1.6 billion parameter automatic speech recognition model built primarily for Arabic, with particular focus on the Emirati dialect. It also transcribes English, French, Spanish and Portuguese using the same model weights. TII reports it beats the best published leaderboard result on a standard Arabic benchmark and leads on an internal Emirati evaluation, while also including word-level timestamps.
GitHub Security Lab built an open-source 'Taskflow Agent' that uses custom AI prompt workflows to audit code for vulnerabilities, and used it to find and report over 24 vulnerabilities in Android applications. Two detailed examples show the agent finding a location-tracking flaw in the OsmAnd navigation app and an account-takeover chain in the Wikipedia Android app via a deeplink parsing bug. The team notes LLMs are strong at finding complex logic vulnerabilities and understand API behavior well, but struggle to correctly judge severity and often produce false positives, so findings still require human review. The taskflows are open source and can be run on any GitHub repository via a Copilot-powered codespace script, though running them consumes premium model requests and can take hours.