Reson8 has launched Rumble-2-Flash, an open-source 0.5B parameter speech-to-text model supporting over 40 languages. It achieves performance parity with models up to 30B parameters on noisy, real-world call-center benchmarks.
Aionic Labs has launched OpenTSLM-Flamingo, a time-series language model (TSLM) now deployable via Amazon SageMaker JumpStart. It combines the Gemma 3 4B architecture with a time-series encoder to interpret operational metrics alongside text context.
AWS now provides managed support for NVIDIA Dynamo within SageMaker HyperPod, simplifying the deployment of large-scale language models. This update introduces multi-node inference, allowing teams to run models that exceed single-GPU memory through disaggregated prefill and decode worker pools.
Amazon SageMaker AI has launched a new 'KV_AWARE' routing strategy that directs LLM inference requests to instances based on their actual KV cache state rather than just prompt headers. This optimizes multi-turn conversations and long-context workloads by ensuring requests land where relevant data is already cached.
NVIDIA has released a technical primer on co-designing sparse attention for long-context models. By focusing on GPU-friendly indexer kernels—which identify relevant tokens before full attention—developers can significantly improve inference efficiency for context lengths surpassing 100K tokens.
Researchers introduced OmniTag-Bench, a new five-language framework for evaluating multimodal LLMs on simultaneous speech transcription and metadata tagging. Instead of using separate specialist models, the study demonstrates that a single, fine-tuned multimodal LLM can extract transcripts, emotion, gender, language, and named entities in a single inference pass.
Hugging Face has partnered with OpenRouter to power its language model inference. All language model requests now use OpenRouter to automatically select the optimal provider based on real-time price, speed, and reliability metrics.
Amazon has released a reference architecture that integrates the NVIDIA NeMo Switchyard library with LiteLLM to dynamically route coding agent requests between different models on Bedrock. By using stage or random routing, you can shift traffic between high-capability and high-efficiency models to balance performance and cost.
NVIDIA has released a technical workflow for serving the 753B parameter GLM-5.3 MoE model across two GB300-equipped DGX Stations. By utilizing tensor parallelism (TP=2) over RDMA connections and vLLM, you can overcome single-GPU memory constraints (252 GB HBM3e) for massive frontier-scale models.
The llama.cpp project has officially reached its 1.0.0 milestone, three and a half years after its inception. The release introduces strict semantic versioning to ensure better predictability for developers relying on its C/C++ libraries, libllama and libggml.
IBM has introduced LLMON (LLM Object Notation), an XML-like markup language that uses reserved special tokens to explicitly label components like instructions, data, and constraints. Two experimental models, Granite 4.1 3B and 8B LLMON-preview, are now available, demonstrating near-zero success rates for indirect prompt injection attacks.