AWS announced general availability of training sessions within Amazon SageMaker AI serverless model customization. Teams can write their own post-training loops — including custom loss functions and reward signals for reinforcement learning — while SageMaker AI handles GPU allocation, distributed execution, fault recovery, and checkpoint storage. Supported techniques include SFT, DPO, RLVR, RLAIF, and multi-turn RL, and users can choose LoRA or full fine-tuning. The feature supports models including gpt-oss-20b, gemma-4-31B-it, Qwen3.6-27B, and Qwen3.8-27B, and billing is based on tokens processed, pausing while a user's reward function scores responses.
Google announced Public Preview of streaming support in API Gateway, covering Server-Sent Events, chunked/NDJSON responses, WebSockets and gRPC streaming. Streaming endpoints can now sit behind the same gateway as regular APIs, keeping existing authentication, logging and monitoring, without any changes to the API spec. Streams can stay open up to one hour, and gateways now get a predictable default hostname instead of a generated hash. The feature also ties into model routing, letting a single gateway stream responses from Gemini, Claude and OpenAI gpt-oss models.
AWS published a walkthrough for the Adaptive Agentic Intelligence Framework (AAIF), built on Strands Agents, which routes simple requests to small local models (via Ollama) and only sends complex requests to cloud models through Amazon Bedrock. A lightweight 1B-parameter model (llama3.2:1b) acts as an orchestrator deciding where each request should go, with an example IoT monitoring agent that uses local telemetry retrieval, local vision analysis, and cloud-based expert insights. AWS states that in the hybrid setup, roughly 80 percent of traffic can be handled locally without hitting a paid API, and the architecture can run fully on edge hardware (e.g. Raspberry Pi with 8GB memory), fully in the cloud, or in a hybrid mode.
Google has added an Extension System to its Agents CLI (from v1.5.0, with a first-party LangChain extension in v1.8.0) that lets developers override or add CLI commands without forking the tool. Previously, Agents CLI's deployment, evaluation and observability pipeline was tightly coupled to Google's own Agent Development Kit (ADK); now teams can scaffold, run, deploy and evaluate LangChain, LangGraph or Deep Agents projects through the same command surface. Extensions are distributed as directories in Git repositories, pinned to a commit SHA in a lockfile, and can be installed at project or user scope. The system also lets enterprise platform teams wrap commands like deploy with custom policies, such as compliance scans or SSO gates. The feature is marked experimental in v1.8.0.
IBM rolled out three updates to its Granite Speech family: a brand-new 2B speech-language model (granite-speech-5.0-2b), a revised 470M Turbo CTC model built for speed, and an improved 4.1-2B-Plus model tuned for multi-speaker recordings. Together they target high-accuracy transcription/translation, fast on-device recognition, and accurate speaker/timestamp labeling. All three remain under Apache 2.0.
Cohere launched Rerank 5, its newest enterprise reranking models, in Pro and Fast tiers. Rerank 5 Pro leads major public benchmarks including BRIGHT, ViDoRe V3, BEIR and nanoMTEB, while Rerank 5 Fast offers near-leading quality at lower cost and higher throughput. The release introduces Context Focus, which returns the most relevant text spans within documents rather than whole documents, and calibrated relevance scores that read as probabilities across models and domains.
Cloudflare has launched full transaction detection and response rules for its web application firewall, which evaluate both the HTTP request and the origin's response to confirm whether an attack actually succeeded. Previously the WAF only inspected incoming requests, which could miss exploits that only reveal themselves in the response (e.g. leaked file contents) or create false positives when blocking based on request signatures alone. The first managed protections target the recently disclosed php2shell vulnerability, and the feature is generally available to all customers, including Free plans.
IBM Research released vLLM-Hook MIA, a plugin for the open-source vLLM inference engine that lets developers observe and intervene in a model's internal states at runtime — for example to decline malicious prompts, flag hallucinated facts, or adjust tone — without disabling vLLM's fast graph-replay execution mode. According to IBM's technical report, MIA is up to 20x faster than the earlier vLLM-Hook and up to 14x faster than methods that insert code directly into the model. In tests, steering an 8B Llama or 1.5B Qwen model added only about 2% latency, and a 70B Llama model added about 6%.
NVIDIA has added three new open, commercially licensed components to its Nemotron Retriever lineup: Embed models (8B and 1B, in BF16 and NVFP4) for searching text and images, a Rerank model (8B) that reorders retrieved evidence by relevance, and a compact 631M Parse model that converts document images into structured text. The goal is to feed AI agents with better, more relevant evidence so they need fewer search rounds and fewer inference tokens to answer a question. NVIDIA reports the new Embed and Rerank models top the ViDoRe v3.1 leaderboards for dense retrieval and reranking respectively.
AI research company i14 partnered with AWS to port its Laplace 3B model, a 24-layer architecture built on Gated Linear Attention (GLA) instead of standard transformer attention, onto AWS Trainium2 using native PyTorch (TorchNeuron Native). Rather than a KV cache that expands with every token, the model keeps a constant 29.5 MB recurrent state per sequence. On a single trn2.48xlarge instance, the team reached about 120,000 training tokens per second and served 11,544 output tokens per second across 1,280 simultaneous sequences through an unmodified vLLM engine.
Google Cloud has made Anthropic's Claude Opus 5.5 and Claude Sonnet 5.5 selectable inside its Antigravity agentic coding platform, alongside Gemini 3.8 Flash and Gemini 4 Argon. Developers can switch between model families mid-conversation across Antigravity 2.0, the CLI, and IDE extensions for VS Code, JetBrains and Visual Studio, all under one enterprise billing and security boundary. Google also describes two engineering patterns: using a second model family to independently review code written by the first ('LLM-as-a-judge'), and assigning fast Gemini Flash subagents to handle high-volume mechanical tasks while a stronger model handles deep reasoning.
AWS introduced Agent Registry, a managed catalog service for Model Context Protocol (MCP) servers, agents, and agent skills. Instead of hardcoding every tool into an orchestrator, agents can now search an approved, versioned catalog at runtime and build a client from the returned descriptor. A reference sample demonstrates a Discover → Instantiate → Execute loop using Amazon Bedrock AgentCore Gateway and AgentCore runtime, with Anthropic Claude Sonnet as the foundation model.
Eon, a data protection ISV running on AWS, replaced per-tool credentials and local MCP servers with a single Amazon Bedrock AgentCore MCP Gateway. The gateway uses Cedar-based policies to enforce least-privilege access for both human engineers and an autonomous SRE agent running Claude on Amazon Bedrock, routing calls to MCP servers hosted on AgentCore Runtimes across multiple AWS accounts. Developer onboarding time dropped from about two days to about two minutes after rolling the system out to 100+ developers and roughly 10,000 sessions.
AWS and Keyfactor published a reference design that fixes a known weak point in Model Context Protocol (MCP) deployments: MCP has no built-in way to authenticate which agent is actually making a request. The architecture adds three trust layers — user-to-system, agent-to-agent, and agent-to-tool-server — built on signed request envelopes, delegation assertions signed by the user, RFC 8693 audience-restricted tokens, and live OCSP revocation checks. A working example runs on Amazon Bedrock and the AWS Strands Agents SDK with Keyfactor EJBCA issuing and validating the X.509 certificates, and the full code is on GitHub.
Mistral AI released Voxtral 3, a set of speech recognition and speech synthesis models paired with an SDK that lets a single real-time pipeline listen, reason, call tools and speak back. The recognition models (Small, Mini and Realtime Transcribe 3) cover 50+ languages with speaker diarization, keyword-biasing for technical vocabulary, and transcription latency under 500ms, which Mistral claims beats Whisper, Scribe v2 and Gemini 3.5 Transcribe on its benchmarks. The speech synthesis model, Mini TTS 3, handles 35+ languages and 50+ dialects with emotion, pacing and emphasis that adapt to context, priced at $13 per million characters, and a separate Pro Voice feature can reproduce a person's voice and emotional range from just 30 minutes of recorded audio. All components can run through Mistral's EU-hosted API or be deployed on a customer's own servers.
IBM Research's ALTK-Evolve extracts short behavioral rules from an agent's own training-task attempts and appends them to the system prompt of a basic ReAct agent running on GPT-4.1. On the τ²-bench customer-service benchmark (retail and airline domains), this learned-memory approach raised accuracy and reliability without updating model weights or using human annotation. The full 'lossless' guideline set pushed a GPT-4.1 agent to a retail Pass^4 score of 67.5%, topping the October 12, 2026 leaderboard snapshot, including beating a model fine-tuned specifically on synthetic retail data. The technique also improved a frontier GPT-5.5 agent on airline tasks, taking it to the top of that leaderboard as well.
Red Hat, IBM, and Google announced llm-d 1.0, a Kubernetes-native distributed inference serving system that moves beyond proof-of-concept performance tricks into full production readiness. The release adds multi-tenant quality of service, zero-downtime operations, and agent-aware serving so the same endpoint can handle both human users and autonomous agents efficiently. In one test, llm-d's prefix-aware router captured 97% of available cache reuse on an agentic coding workload, and IBM/Red Hat ran 3,000 concurrent software-engineering agents for 5 to 10 times less cost than using the same model as a hosted service.
AWS has rolled out structured outputs across every Anthropic Claude model hosted on Amazon Bedrock, reaching both the standard runtime endpoint and the newer Mantle endpoint built for agentic workloads. Instead of hoping the model returns valid JSON, developers can now lock responses to an exact schema through constrained decoding, which blocks the model from generating tokens that would break the format. A companion feature, strict tool use, applies the same guarantee to function-calling arguments so agent workflows don't fail on malformed inputs.
AWS released a push-based NIXL connector for vLLM that changes how key-value (KV) cache data moves between prefill and decode instances in disaggregated LLM serving. Instead of the decode instance pulling KV blocks after prefill finishes, the decode instance registers its target memory in advance and the prefill instance writes the KV data directly once ready, moving coordination work off the critical path. Benchmarked on a 1-prefill/1-decode Qwen3-32B deployment using EC2 P5en instances with EFA, push mode delivered up to 3x better time-to-first-token (TTFT) and up to 30% better time-per-output-token (TPOT) than the default pull mode, especially under high request rates.
Microsoft announced an expansion of its Project Perception security framework to address risks from autonomous AI agents, framing them as a new category of 'insider' that needs distinct identity, permission, and risk controls. The update introduces Dynamic Permissions (least-privilege, task-scoped, time-bound access), Dynamic Agent Risk (continuous risk scoring based on behavior drift and context), and Dynamic Conditional Access (graduated responses from narrowing access to full runtime isolation). Microsoft is also extending discovery and identity support (Microsoft Entra Agent ID) to 60 types of local agents, including Claude Code, OpenAI Codex, and GitHub Copilot, and introducing 'Guardian Agents' that monitor and respond to risky agent behavior at machine speed.
DeepSeek released DeepSeek-V4.1-Flash-1017, an upgraded version of its Flash model, accessible through the same API endpoint and pricing as before. The update brings notable gains across agentic, coding, and tool-use benchmarks while reducing task runtime and output length. DeepSeek also detailed its in-house search engine, which now powers both web/app search and a developer-facing Search & Fetch API.
OpenAI has rolled out prompt-to-voice access to all GPT-Live API customers, allowing a spoken voice to be designed simply by describing it in text, either through the API or the voice Playground, and then reused inside live voice sessions. Designing the voice costs nothing, but speaking with it still runs on normal GPT-Live session billing. Alongside this rollout, nine additional built-in voices covering German, French, Hindi, Japanese and Korean were added to the platform.
Google Cloud published a reference architecture for building AI agents that coordinate long-running, GPU-heavy scientific tasks like protein folding and compound screening, which break standard single-LLM-agent setups due to timeouts and context overflow. The design separates a cognitive orchestrator (Gemini on Agent Runtime) from a GPU execution layer (NVIDIA NIMs like ESMFold and GenMol on GKE) and wraps both in an enterprise governance plane with agent identity, a gateway, a registry and Model Armor. It also lays out best practices such as passing bulky data by reference instead of inline, streaming progress updates, and gating irreversible actions behind human approval.
IBM has released Granite Switch 4.1 in 8B and 30B sizes, a modular system that runs chat, decision-making and speech transcription from one hosted endpoint instead of separate models. The update adds a general-purpose Decision adapter, a specialized Guardian adapter for safety-related classifications, and an embedded ASR model, Granite-Speech-5.0-Turbo-CTC, which tops the OpenASR Leaderboard. Despite bundling three capabilities, the combined model uses roughly the same VRAM as the base model alone.