AI research company i14 partnered with AWS to port its Laplace 3B model, a 24-layer architecture built on Gated Linear Attention (GLA) instead of standard transformer attention, onto AWS Trainium2 using native PyTorch (TorchNeuron Native). Rather than a KV cache that expands with every token, the model keeps a constant 29.5 MB recurrent state per sequence. On a single trn2.48xlarge instance, the team reached about 120,000 training tokens per second and served 11,544 output tokens per second across 1,280 simultaneous sequences through an unmodified vLLM engine.
AWS introduced Agent Registry, a managed catalog service for Model Context Protocol (MCP) servers, agents, and agent skills. Instead of hardcoding every tool into an orchestrator, agents can now search an approved, versioned catalog at runtime and build a client from the returned descriptor. A reference sample demonstrates a Discover → Instantiate → Execute loop using Amazon Bedrock AgentCore Gateway and AgentCore runtime, with Anthropic Claude Sonnet as the foundation model.
Eon, a data protection ISV running on AWS, replaced per-tool credentials and local MCP servers with a single Amazon Bedrock AgentCore MCP Gateway. The gateway uses Cedar-based policies to enforce least-privilege access for both human engineers and an autonomous SRE agent running Claude on Amazon Bedrock, routing calls to MCP servers hosted on AgentCore Runtimes across multiple AWS accounts. Developer onboarding time dropped from about two days to about two minutes after rolling the system out to 100+ developers and roughly 10,000 sessions.
AWS and Keyfactor published a reference design that fixes a known weak point in Model Context Protocol (MCP) deployments: MCP has no built-in way to authenticate which agent is actually making a request. The architecture adds three trust layers — user-to-system, agent-to-agent, and agent-to-tool-server — built on signed request envelopes, delegation assertions signed by the user, RFC 8693 audience-restricted tokens, and live OCSP revocation checks. A working example runs on Amazon Bedrock and the AWS Strands Agents SDK with Keyfactor EJBCA issuing and validating the X.509 certificates, and the full code is on GitHub.
AWS has rolled out structured outputs across every Anthropic Claude model hosted on Amazon Bedrock, reaching both the standard runtime endpoint and the newer Mantle endpoint built for agentic workloads. Instead of hoping the model returns valid JSON, developers can now lock responses to an exact schema through constrained decoding, which blocks the model from generating tokens that would break the format. A companion feature, strict tool use, applies the same guarantee to function-calling arguments so agent workflows don't fail on malformed inputs.
AWS and Deepgram jointly rebuilt the Nova-3 speech-to-text model to run natively on Trainium2 chips instead of standard GPU-based PyTorch setups, targeting large offline transcription jobs. The work involved rewriting bottleneck layers as custom kernels and replacing Deepgram's Python serving layer with a Rust runtime calling Trainium directly. The result: a single trn2.3xlarge instance running eight parallel model copies delivers 11-70% more transcribed audio per dollar than Deepgram's prior infrastructure, with similar savings potentially available to customers self-hosting on SageMaker.
AWS released a push-based NIXL connector for vLLM that changes how key-value (KV) cache data moves between prefill and decode instances in disaggregated LLM serving. Instead of the decode instance pulling KV blocks after prefill finishes, the decode instance registers its target memory in advance and the prefill instance writes the KV data directly once ready, moving coordination work off the critical path. Benchmarked on a 1-prefill/1-decode Qwen3-32B deployment using EC2 P5en instances with EFA, push mode delivered up to 3x better time-to-first-token (TTFT) and up to 30% better time-per-output-token (TPOT) than the default pull mode, especially under high request rates.
AWS published a governance framework called Trust in Depth, designed to let AI agents operate with more autonomy while staying within enforceable limits. It combines scoping agent permissions before deployment, giving each agent a verifiable identity, enforcing boundaries at runtime via policy gateways, and observing agent behavior afterward. The framework leans on automated reasoning (mathematical proof) for rules that can be precisely defined, and human judgment for ambiguous cases, using AWS tools like Bedrock AgentCore, Guardrails, Kiro, Strands, Agent Registry and CloudWatch Omni.
Orca Security built Discovery AI, an Amazon Bedrock-based agent that turns plain-English security questions into structured queries against its Sonar system, which tracks over 900 cloud resource types. Rather than relying on one large model with the full schema stuffed into every prompt, the production version splits the work across nine specialized stages and multiple foundation models. The result: dramatically lower cost, faster responses, and better accuracy on complex queries, now running live since March 2026.
AWS detailed three kernel-level optimizations for Mixture-of-Experts (MoE) model serving on Hopper GPUs, implemented in collaboration with the vLLM community and shipped in Amazon SageMaker AI's LMI container. The changes target the inefficiencies of fused MoE computation during low-batch decode, which previously left Tensor Cores underutilized. Combined, the optimizations deliver up to 10% higher throughput and up to 9% faster time-to-first-token (TTFT).
AWS published a joint engineering write-up with Postman describing how Agent Mode, Postman's AI-native interface for API testing and documentation, was built to run in production at scale. Rather than model quality, the team found that tool sprawl and missing context were the biggest obstacles to reliable agent behavior. The post details architectural patterns including dynamic tool scoping, schema-based query access, dedicated context handlers, and how Amazon Bedrock provides model flexibility, cross-Region inference, data residency controls, and prompt caching.
AWS describes a new two-stage access control architecture for Retrieval Augmented Generation (RAG) in Amazon Quick and Amazon Bedrock Knowledge Bases. Instead of relying only on periodically synced access control lists (ACLs), the system now verifies permissions in real time against the authoritative data source (e.g. Google Drive, SharePoint, Confluence) at query time, before passing content to the LLM. AWS says this closes security gaps caused by stale or incorrectly mapped permissions between sync cycles. Mondelēz International, with over 35,000 employees, is cited as a customer using this approach.
Anthropic has launched Claude Haiku 5.5 (claude-haiku-5-5), positioned as its most capable Haiku model for high-volume, latency-sensitive work. It supports a 1 million token context window, up to 128k max output tokens, and adaptive thinking controlled via a new effort parameter. The model is available on the Claude API, Amazon Bedrock, Claude Platform on AWS, Google Cloud, and Microsoft Foundry. Anthropic warns that code written for the previous Haiku 4.5 model may break on the new version.
Z.ai's GLM 5.3, a 753-billion-parameter mixture-of-experts model optimized for coding and long-horizon agentic tasks, is now available as a fully managed model on Amazon Bedrock. It builds on GLM 5 with improved coding benchmark results, standout cybersecurity capabilities (CyberGym score of 84.5), and deeper Bedrock integration including OpenAI-compatible APIs, prompt caching, cross-Region inference and service tiers. AWS also demonstrates using GLM 5.3 with the open-source Strix agent for authorized penetration testing of your own applications.
GitHub has published ReviewBench, a research-preview benchmark designed to compare AI code review agents on a common, auditable standard. The dataset draws on 219 real public pull requests across 19 languages and 187 repositories, chosen to mirror the language and size distribution found across 103.9 million GitHub pull requests. Ground truth comes from a blended pool of human reviewers, static analysis tools, and multiple frontier LLMs, cross-checked by senior engineers who agreed with the benchmark's labels 96.6% of the time. GitHub says it has already used ReviewBench internally to track its own Copilot code review (CCR) product, and that offline benchmark movement reliably predicted the direction of later production A/B tests.
GitHub Security Lab built an open-source 'Taskflow Agent' that uses custom AI prompt workflows to audit code for vulnerabilities, and used it to find and report over 24 vulnerabilities in Android applications. Two detailed examples show the agent finding a location-tracking flaw in the OsmAnd navigation app and an account-takeover chain in the Wikipedia Android app via a deeplink parsing bug. The team notes LLMs are strong at finding complex logic vulnerabilities and understand API behavior well, but struggle to correctly judge severity and often produce false positives, so findings still require human review. The taskflows are open source and can be run on any GitHub repository via a Copilot-powered codespace script, though running them consumes premium model requests and can take hours.
AWS released a step-by-step walkthrough showing how to give Claude Desktop (on Amazon Bedrock) live internet access through Amazon Bedrock AgentCore Gateway, closing the gap left by the model's training cutoff. The setup wires AWS IAM Identity Center SSO through Amazon Cognito as a SAML-to-OAuth federation layer, issuing JWTs that the AgentCore Gateway validates on every request. The Web Search capability itself is MCP-compatible and runs on an internal Amazon web index covering tens of billions of documents, so no query traffic leaves AWS and no third-party API key is needed. The feature is currently limited to three AWS Regions: us-east-1 (N. Virginia), eu-west-1 (Ireland), and ap-northeast-1 (Tokyo).