Amazon has introduced the Global Inference Gateway for SageMaker HyperPod, enabling a single endpoint to route requests across multiple clusters and AWS Regions. The gateway manages traffic based on real-time health and load signals like KV cache utilization and queue depth, automating cross-region failover.
Amazon Textract Insights now provides generative AI capabilities that extract structured data from documents while offering confidence scores and page-level grounding. It supports processing mixed-document packets, splitting them automatically, and matching them to predefined or custom blueprints.
Exa successfully scaled its vector embedding pipeline by optimizing performance on AWS Inferentia2 and Trainium2 chips. Through a combination of weight layout adjustments, custom NKI (Neuron Kernel Interface) kernels, and an agent-driven development methodology, they achieved a 47% improvement in cost-performance.
Amazon SageMaker AI now supports native cross-Region inference routing, allowing you to automatically shift traffic to healthy endpoints in different regions during capacity shortages. This feature manages failover, health monitoring, and traffic recovery without requiring custom code on the critical request path.
AWS details three strategies to manage massive JSON payloads (20MB+) in AI agents, achieving up to 71% fewer tokens and 80% lower latency. The approach involves offloading state to Amazon S3, paginating large tool results, and compacting dictionary outputs.
Amazon has introduced new AI-driven data preparation capabilities within SageMaker to streamline the creation and maintenance of high-quality training and evaluation datasets. These features include auto-labeling, synthetic data generation, and automated data quality checks, controllable via Python SDK or AI-ML coding agents.
AWS has published a guide on creating an event-driven architecture to automatically sync Model Context Protocol (MCP) tool schemas with the AWS Agent Registry. Using AWS CloudTrail, EventBridge, and Lambda, this pipeline detects runtime changes and updates the registry whenever an MCP server's tool schema evolves.
AWS introduced pre-built Ray Train Deep Learning Containers (DLCs) that consolidate PyTorch, CUDA, NCCL, and EFA networking libraries into a single tested image. This approach allows your team to deploy distributed training clusters on Amazon EKS without maintaining custom build pipelines or resolving complex driver compatibility issues.
AWS has updated its QA Studio reference solution, adding native support for multi-team organizational structures. The new capabilities include application-level ownership, automated health scoring based on CloudWatch metrics, and granular role-based access control.