Hugging Face has introduced expert streaming for Mixture-of-Experts (MoE) models within the Transformers library. This technique allows running massive models that would typically exceed standard hardware memory limits by streaming active experts on demand.
PolyAI has introduced Dialog-RSN-1, a model specifically trained to handle live voice conversations by processing audio directly without intermediate transcription. It is now accessible in the AWS ecosystem for deployment as a private, secure inference endpoint.
NVIDIA has released a new line of compact desktop PCs from ASUS, Dell, and MSI designed specifically for local AI workloads. These systems feature a 20-core NVIDIA Grace CPU, Blackwell RTX GPU, and up to 128GB of unified memory to support large language models and persistent agents.
Researchers released Limite 1B - Violetto, a compact 1-billion-parameter model built for mathematical problem-solving. It supports a 131k token context window and is licensed under Apache 2.0.
A new causal method named RAW (Refusal Aversion in the Wild) uses LLM stochasticity to measure how often user refusals lead to platform abandonment. By analyzing existing interaction logs, you can identify the exact causal effect of refusals on subsequent user re-engagement.
Aionic Labs has launched OpenTSLM-Flamingo, a time-series language model (TSLM) now deployable via Amazon SageMaker JumpStart. It combines the Gemma 3 4B architecture with a time-series encoder to interpret operational metrics alongside text context.
AWS now provides managed support for NVIDIA Dynamo within SageMaker HyperPod, simplifying the deployment of large-scale language models. This update introduces multi-node inference, allowing teams to run models that exceed single-GPU memory through disaggregated prefill and decode worker pools.
Amazon SageMaker AI has launched a new 'KV_AWARE' routing strategy that directs LLM inference requests to instances based on their actual KV cache state rather than just prompt headers. This optimizes multi-turn conversations and long-context workloads by ensuring requests land where relevant data is already cached.
NVIDIA has released a technical primer on co-designing sparse attention for long-context models. By focusing on GPU-friendly indexer kernels—which identify relevant tokens before full attention—developers can significantly improve inference efficiency for context lengths surpassing 100K tokens.
NVIDIA unveiled 'Nemotron for Science', a toolkit combining the Nemotron 3.5 Super VL model with specialized chemistry datasets and post-training recipes. This platform allows researchers to deploy local agents that can autonomously plan experiments, execute code, and validate data findings.
Researchers introduced OmniTag-Bench, a new five-language framework for evaluating multimodal LLMs on simultaneous speech transcription and metadata tagging. Instead of using separate specialist models, the study demonstrates that a single, fine-tuned multimodal LLM can extract transcripts, emotion, gender, language, and named entities in a single inference pass.
TokenEconomizer is a new open-source plugin that allows AI coding agents to learn from past sessions within a specific repository. By tracking recurring patterns, pruning unnecessary tools, and preventing repetitive failure loops, it significantly reduces token consumption without altering the agent or its base model.
Hugging Face has partnered with OpenRouter to power its language model inference. All language model requests now use OpenRouter to automatically select the optimal provider based on real-time price, speed, and reliability metrics.
NVIDIA has released the initial MLPerf Training v6.1 and Endpoints v1.0 results for its new Vera Rubin architecture. The system demonstrated significant leaps in both training speed and power efficiency for agent-based AI workloads.
Researchers introduced ActGCG, a method to replicate the effects of activation steering—typically requiring internal model access—by optimizing input suffixes. The technique successfully reproduces specific behaviors like language steering through external text inputs alone.
Amazon has released a reference architecture that integrates the NVIDIA NeMo Switchyard library with LiteLLM to dynamically route coding agent requests between different models on Bedrock. By using stage or random routing, you can shift traffic between high-capability and high-efficiency models to balance performance and cost.
Amazon has added OpenFold3-preview to SageMaker JumpStart, allowing users to deploy models for predicting 3D structures of proteins, DNA, RNA, and ligands. The implementation utilizes asynchronous inference endpoints to handle long-running predictions while supporting cost-efficient autoscaling and scale-to-zero capabilities.
AI-powered penetration testing and orchestration frameworks have proliferated rapidly since early October, with thousands of unique instances currently active. These autonomous tools automate the entire cyber kill chain, chaining credentials to move laterally through infrastructure with unprecedented speed and scale.
NVIDIA has released the cuda.bindings.cudla module, enabling direct Python-based management of the Deep Learning Accelerator (DLA) on Jetson AGX Orin. Developers can now orchestrate hybrid workflows where GPU preprocessing and DLA inference share a single CUDA stream.
NVIDIA has released a technical workflow for serving the 753B parameter GLM-5.3 MoE model across two GB300-equipped DGX Stations. By utilizing tensor parallelism (TP=2) over RDMA connections and vLLM, you can overcome single-GPU memory constraints (252 GB HBM3e) for massive frontier-scale models.
The llama.cpp project has officially reached its 1.0.0 milestone, three and a half years after its inception. The release introduces strict semantic versioning to ensure better predictability for developers relying on its C/C++ libraries, libllama and libggml.
DeepSeek has moved beyond basic model inference by exposing its full application-level toolkit to API users. Developers can now enable parameters for web search, URL reading, file parsing, and safety filtering directly in chat completions, or utilize standalone ASR, TTS, and OCR endpoints.
Amazon has released SGLang-Neuron, a plugin allowing developers to run large-scale AI models on AWS Trainium hardware using the standard SGLang interface. This integration enables native PyTorch execution and supports advanced features like PD disaggregation over EFA for optimized serving.
Researchers have established the theoretical maximum reward gain possible during language model alignment under fixed KL-divergence constraints. The findings demonstrate that 'best-of-N' sampling effectively reaches these theoretical bounds, whereas common reinforcement learning approaches like PPO and GRPO currently fall short of optimal performance.