Amazon SageMaker AI has launched a new 'KV_AWARE' routing strategy that directs LLM inference requests to instances based on their actual KV cache state rather than just prompt headers. This optimizes multi-turn conversations and long-context workloads by ensuring requests land where relevant data is already cached.
NVIDIA has released a technical primer on co-designing sparse attention for long-context models. By focusing on GPU-friendly indexer kernels—which identify relevant tokens before full attention—developers can significantly improve inference efficiency for context lengths surpassing 100K tokens.
Researchers introduced OmniTag-Bench, a new five-language framework for evaluating multimodal LLMs on simultaneous speech transcription and metadata tagging. Instead of using separate specialist models, the study demonstrates that a single, fine-tuned multimodal LLM can extract transcripts, emotion, gender, language, and named entities in a single inference pass.
Researchers have introduced Trident, a framework designed to improve LLM-based agents by solving three information bottlenecks: ineffective memory retrieval, redundant behavioral candidates, and risky whole-harness rewriting. By implementing boundary-aware memory reading, diversity-aware routing, and order-aware patching, Trident consistently outperforms existing reflection and memory-based methods.
Researchers have introduced EndoUT, an adversarial framework that identifies universal prompt triggers capable of significantly degrading LLM utility. By analyzing how a model's internal response-onset preferences are disrupted, attackers can create fixed strings that force performance drops across various benign tasks without needing target labels or feedback.
MoEngage has integrated AI agents into its customer engagement platform by using the Model Context Protocol (MCP) and Amazon Bedrock AgentCore to expose existing APIs as agent-callable tools. By treating its platform as an architecture of isolated sessions and standardized tool schemas, the company enabled teams to automate workflows and decision-making within a single conversation.
Hugging Face has partnered with OpenRouter to power its language model inference. All language model requests now use OpenRouter to automatically select the optimal provider based on real-time price, speed, and reliability metrics.
Researchers have introduced a 'full-block estimator' to optimize the alignment of Masked Diffusion Language Models (MDLMs). By evaluating every block exactly once during training, this approach removes the variance associated with random block sampling.
Researchers introduced ActGCG, a method to replicate the effects of activation steering—typically requiring internal model access—by optimizing input suffixes. The technique successfully reproduces specific behaviors like language steering through external text inputs alone.
Amazon has released a reference architecture that integrates the NVIDIA NeMo Switchyard library with LiteLLM to dynamically route coding agent requests between different models on Bedrock. By using stage or random routing, you can shift traffic between high-capability and high-efficiency models to balance performance and cost.
Amazon Threat Intelligence observed autonomous agents using the open-source ARTEX framework to conduct large-scale credential theft and deepfake-based identity fraud. These campaigns exploited unpatched AI infrastructure and public misconfigurations to automate multi-stage offensive operations.
NVIDIA has released a technical workflow for serving the 753B parameter GLM-5.3 MoE model across two GB300-equipped DGX Stations. By utilizing tensor parallelism (TP=2) over RDMA connections and vLLM, you can overcome single-GPU memory constraints (252 GB HBM3e) for massive frontier-scale models.
The llama.cpp project has officially reached its 1.0.0 milestone, three and a half years after its inception. The release introduces strict semantic versioning to ensure better predictability for developers relying on its C/C++ libraries, libllama and libggml.
Researchers have developed a method to adapt general-purpose, frozen LLMs into full-duplex voice systems using only 97 million trainable parameters and 30 GPU-hours of training. The system functions as a thinker-talker cascade with an 80 ms frame clock, allowing for realistic turn-taking, interruptions, and backchanneling without massive speech-data pretraining.
IBM has introduced LLMON (LLM Object Notation), an XML-like markup language that uses reserved special tokens to explicitly label components like instructions, data, and constraints. Two experimental models, Granite 4.1 3B and 8B LLMON-preview, are now available, demonstrating near-zero success rates for indirect prompt injection attacks.
Researchers have established the theoretical maximum reward gain possible during language model alignment under fixed KL-divergence constraints. The findings demonstrate that 'best-of-N' sampling effectively reaches these theoretical bounds, whereas common reinforcement learning approaches like PPO and GRPO currently fall short of optimal performance.
Amazon introduced a pattern for securing multi-tenant AI agents by ensuring that RAG search queries are scoped to specific users. The solution uses ACL-aware knowledge bases combined with a request interceptor in the AgentCore Gateway to deterministically inject verified user identities into tool calls, preventing LLMs or prompt injections from accessing unauthorized data.
Researchers have identified that the upstream construction of target hidden states significantly impacts the quality of LLM knowledge updates. By implementing an L1 regularization method to prune extraneous components of knowledge vectors, this approach reduces interference during parameter editing, allowing for more precise knowledge injection.
ElevenLabs introduced a major update to its platform at the NYC Summit, focusing on domain-specific model fine-tuning, an 'Interaction Engine' for intelligent model routing, and persistent agent memory. The company also launched 'Routines' for workflow automation and a new 'ElevenAI' app for enterprise meeting productivity.
Amazon has released the Universal Agent Evaluation Framework (UAEF), an open-source tool designed to standardize the evaluation of AI agents. It uses adapters to normalize execution traces from various frameworks like LangChain and LangGraph into a single format, enabling consistent measurement across 36 metrics and 7 quality dimensions.
Amazon Bedrock AgentCore now includes a comprehensive catalog of built-in, third-party, and custom evaluators to monitor agent behavior in production. The platform allows you to run metrics from DeepEval and AutoEval in a managed environment, supporting on-demand, online, and batch evaluation workflows.
Tether AI Research has introduced BotPsy, a 0.8B-parameter language model optimized for agentic workflows and function calling. Built on Qwen3.5-0.8B, the model is designed for on-device deployment, offering superior tool-use reliability and instruction following while maintaining data privacy.
Moonshot AI has introduced Kimi K3.1 Pro Highspeed, a new model on Amazon Bedrock optimized for long-horizon agentic tasks and coding. The model delivers 2–3 times faster output than the standard version and includes configurable reasoning, tool calling, and support for explicit prompt caching to optimize costs.