Amazon SageMaker AI now monitors inference endpoints to identify latency drift, underutilized GPUs, and missing performance optimizations. The feature uses an agentic experience in SageMaker Studio to help you benchmark and resolve these issues without manual setup.
Amazon has introduced the Global Inference Gateway for SageMaker HyperPod, enabling a single endpoint to route requests across multiple clusters and AWS Regions. The gateway manages traffic based on real-time health and load signals like KV cache utilization and queue depth, automating cross-region failover.
Amazon Textract Insights now provides generative AI capabilities that extract structured data from documents while offering confidence scores and page-level grounding. It supports processing mixed-document packets, splitting them automatically, and matching them to predefined or custom blueprints.
AWS has launched the GetScopedCredentials API, allowing you to dynamically reduce the permissions of temporary credentials. This enables you to provide AI agents or sub-agents with only the access they strictly require for a specific task, rather than granting them the full scope of the parent session.
Google has expanded its Agent Anomaly Detection service to support agents running outside Google Cloud, such as those on AWS Lambda or Microsoft Foundry. The service uses telemetry and logs to identify behavioral drift, unauthorized tool usage, and resource exhaustion without needing access to the runtime environment.
Exa successfully scaled its vector embedding pipeline by optimizing performance on AWS Inferentia2 and Trainium2 chips. Through a combination of weight layout adjustments, custom NKI (Neuron Kernel Interface) kernels, and an agent-driven development methodology, they achieved a 47% improvement in cost-performance.
Amazon Bedrock AgentCore Identity now permits agents to authenticate against internal identity providers using private certificate authorities (CAs). This update removes the need for public endpoints when your organization self-hosts services like Keycloak or PingFederate within a VPC.
Moonshot AI has updated the Kimi Code CLI to v2.7.0, introducing a native Amazon Bedrock provider. This allows developers to authenticate using standard AWS credential chains and SSO instead of managing separate, long-lived API keys.
Amazon SageMaker AI now supports native cross-Region inference routing, allowing you to automatically shift traffic to healthy endpoints in different regions during capacity shortages. This feature manages failover, health monitoring, and traffic recovery without requiring custom code on the critical request path.
Amazon Polly has introduced a new voice cloning feature that builds synthetic voices from as little as two to ten minutes of speaker audio. This tool is now available through a gated release for seven languages and integrates directly with existing AWS cloud workflows for enhanced security and brand consistency.
Amazon Bedrock AgentCore has upgraded its underlying harness to the Strands framework. This update introduces built-in capabilities like web retrieval, task planning, and automatic context management, alongside new lifecycle hooks for deterministic control over agent execution.
Amazon has published a reference architecture for connecting Bedrock AgentCore to private REST APIs and MCP servers in restricted network environments. By utilizing a VPC Lattice resource gateway placed in a dedicated network-account VPC, traffic between agents and internal tools is forced through a central inspection firewall.
AWS details three strategies to manage massive JSON payloads (20MB+) in AI agents, achieving up to 71% fewer tokens and 80% lower latency. The approach involves offloading state to Amazon S3, paginating large tool results, and compacting dictionary outputs.
Amazon has published a security guide for exposing AI agents via Bedrock AgentCore Runtime, emphasizing the need for robust ingress controls. The guide details how to prevent cross-tenant leakage and privilege escalation using identity propagation and layered authorization.
Amazon has introduced new AI-driven data preparation capabilities within SageMaker to streamline the creation and maintenance of high-quality training and evaluation datasets. These features include auto-labeling, synthetic data generation, and automated data quality checks, controllable via Python SDK or AI-ML coding agents.
AWS now provides managed support for NVIDIA Dynamo within SageMaker HyperPod, simplifying the deployment of large-scale language models. This update introduces multi-node inference, allowing teams to run models that exceed single-GPU memory through disaggregated prefill and decode worker pools.
Amazon SageMaker AI has launched a new 'KV_AWARE' routing strategy that directs LLM inference requests to instances based on their actual KV cache state rather than just prompt headers. This optimizes multi-turn conversations and long-context workloads by ensuring requests land where relevant data is already cached.
Amazon Bedrock has launched a new memory-aware capability for AgenticRetrieveStream. This feature automates the storage, extraction, and retrieval of user-specific facts and conversation histories, allowing agents to provide context-aware responses while maintaining strict document-level access control.
Amazon has released a reference architecture that integrates the NVIDIA NeMo Switchyard library with LiteLLM to dynamically route coding agent requests between different models on Bedrock. By using stage or random routing, you can shift traffic between high-capability and high-efficiency models to balance performance and cost.
Amazon has released SGLang-Neuron, a plugin allowing developers to run large-scale AI models on AWS Trainium hardware using the standard SGLang interface. This integration enables native PyTorch execution and supports advanced features like PD disaggregation over EFA for optimized serving.
Amazon has released a next-generation runtime for Bedrock AgentCore designed to handle complex, long-running AI agent workloads. The update brings hardware-isolated environments with faster startup times, increased compute capacity, and improved cost efficiency through usage-based billing.
Amazon introduced a pattern for securing multi-tenant AI agents by ensuring that RAG search queries are scoped to specific users. The solution uses ACL-aware knowledge bases combined with a request interceptor in the AgentCore Gateway to deterministically inject verified user identities into tool calls, preventing LLMs or prompt injections from accessing unauthorized data.
Amazon now offers a direct Web Search API for Bedrock AgentCore, enabling developers to perform web searches via standard AWS SDKs and CLI. This bypasses the need for the AgentCore Gateway, simplifying the process for custom agent loops and non-agent applications that require real-time information.
AWS has published a guide on creating an event-driven architecture to automatically sync Model Context Protocol (MCP) tool schemas with the AWS Agent Registry. Using AWS CloudTrail, EventBridge, and Lambda, this pipeline detects runtime changes and updates the registry whenever an MCP server's tool schema evolves.