Skip to content
Friday, 20 November 2026
Tenesys AI News
Subscribe

All News

TENESYS AI NEWS tracks the most important AI news and explains what changed, why it matters and whether a technology is worth testing.

#MoE — 7 articles ✕

Models· Important· 🧪 Worth Testing

Scaling Large MoE Models Across Multiple NVIDIA DGX Stations

NVIDIA has released a technical workflow for serving the 753B parameter GLM-5.3 MoE model across two GB300-equipped DGX Stations. By utilizing tensor parallelism (TP=2) over RDMA connections and vLLM, you can overcome single-GPU memory constraints (252 GB HBM3e) for massive frontier-scale models.

NVIDIA Developer
Models· Important· 🧪 Worth Testing

Moonshot AI releases Kimi-K3.1-Pro multimodal agentic model

Moonshot AI has introduced Kimi-K3.1-Pro, an open-weight, natively multimodal model with 2.8 trillion parameters. It features a 1-million-token context window, uses a Mixture-of-Experts (MoE) architecture, and is specifically optimized for agentic workflows and long-horizon tasks.

Moonshot Kimi (Hugging Face)
Models· Important· 🧪 Worth Testing

NVIDIA releases multimodal Nemotron 3.5 Super VL for AWS EKS

NVIDIA has introduced Nemotron 3.5 Super VL, a 120B-parameter multimodal model featuring a 1M token context window. The model utilizes a hybrid Mamba-2 and MoE architecture to balance reasoning capabilities with efficient inference, and is now available for deployment on Amazon EKS and SageMaker HyperPod.

AWS Machine Learning
Models· Important· 🧪 Worth Testing

Boosting Large-Scale MoE Model Throughput on NVIDIA Hardware

NVIDIA has documented a threefold optimization approach to accelerate MoE inference, tested on the 124-billion-parameter Ling-3.0-flash model. By utilizing specialized Humming kernels, FP8 precision for the LM head, and Multi-Token Prediction, developers can achieve a 42.5% throughput increase and a 24.5% reduction in latency.

NVIDIA Developer
Models· Important· 🧪 Worth Testing

Z.ai GLM 5.3 Flash model now available with 1M-token context

Z.ai's GLM 5.3 Flash, a multimodal 320B-parameter Mixture-of-Experts model with 18B active parameters, is now available through Mistral's platform. It offers a 1M-token context window and is built for developer agent workflows, visual coding, and complex tool-use tasks. Pricing is set at $0.15 per million input tokens, $0.03 per million cached input tokens, and $0.50 per million output tokens.

Mistral Changelog
Research· Important

Google and Meta bring native TPU support to PyTorch's TorchTitan training framework

Google introduced TorchTPU, a native PyTorch backend for Cloud TPUs (torch.device("tpu")), and co-developed a fork of Meta's TorchTitan trainer adapted for TPU training. Previously, running PyTorch training on TPUs meant rewriting code in JAX or relying on the older torch_xla path; now only minimal device-initialization changes are needed. The fork provides tested recipes for pre-training large dense models (Llama 3 70B/405B, Qwen 3 8B/32B), Mixture-of-Experts models (Qwen 3 235B), and diffusion transformers (FLUX.1 12B), plus a 'Graph Trainer' that captures an entire training step as one XLA program for better compiler optimization.

Google Developers
Models· Important

Reflection releases Beam, a 501B open-weight MoE model with 1M-token context

Reflection, a new US AI lab, has launched Beam, its first model, as an open-weight mixture-of-experts system with 501B total parameters (23B active per token) and a 1M-token context window, released under Apache 2.0. The model is built for coding and agentic use cases, ships with a custom Rust-based inference harness, and has day-0 support in transformers, vLLM and SGLang. Reflection reports strong results on coding, reasoning and tool-calling benchmarks across BF16, FP8 and NVFP4 checkpoints.

Hugging Face