Google has updated its open-source JAX training framework, MaxText, to support training LLMs at extreme sequence lengths on Cloud TPU hardware. By introducing four distinct context-parallelism strategies, the update enables efficient training of models with up to 14 million tokens, overcoming memory and compute bottlenecks.
Google Cloud has launched dynamic slicing for TPUs on GKE, allowing developers to decouple logical hardware configurations from physical nodepools. This update eliminates the need to manually recreate nodepools when workload requirements change, enabling faster scaling and improved fault tolerance.
Google introduced TorchTPU, a native PyTorch backend for Cloud TPUs (torch.device("tpu")), and co-developed a fork of Meta's TorchTitan trainer adapted for TPU training. Previously, running PyTorch training on TPUs meant rewriting code in JAX or relying on the older torch_xla path; now only minimal device-initialization changes are needed. The fork provides tested recipes for pre-training large dense models (Llama 3 70B/405B, Qwen 3 8B/32B), Mixture-of-Experts models (Qwen 3 235B), and diffusion transformers (FLUX.1 12B), plus a 'Graph Trainer' that captures an entire training step as one XLA program for better compiler optimization.
Google announced new vLLM and SGLang inference engines built directly on PyTorch for TPU, developed with Inferact and RadixArk respectively. The engines run on a new TorchTPU layer that makes TPUs a native PyTorch device, replacing the previous JAX-based backend approach. Google reports a public benchmark result of 2.01 cents per million tokens in high-concurrency configurations via the SemiAnalysis InferenceX project, plus support for agentic workloads, long context, disaggregated serving and multimodal models.
A Google Cloud TPU engineering team worked with Ai2 to rebuild Olmo 3 7B's pre-training from scratch using MaxText, Google's JAX/XLA training framework, running on TPUs instead of Ai2's original PyTorch/GPU setup. They matched Ai2's published results not just on the training loss curve but on four independent held-out evaluation surfaces across the full ~5.93-trillion-token, 1.41-million-step stage-1 run plus the stage-2 annealing phase. Along the way they ported Olmo 3's unusual architecture (reordered-norm blocks, QK-norm, 3:1 sliding/global attention ratio) into MaxText and caught a data-loader bug that had been quietly inflating apparent performance through memorization rather than genuine learning.
Google engineers describe how they implemented sparse spatio-temporal attention for video diffusion models on TPU v6e chips, converting the theoretical sparsity of the Sparse VideoGen (SVG) approach into real hardware speedups. Through a progression of kernel optimizations (full/boundary tile specialization, tile-size tuning, and mask-tile alignment), they reduced attention kernel latency from 96.37ms (naive sparse) to 32.76ms, a 2.40x speedup over dense Splash Attention on a single TPU v6e chip, using 75.6K tokens, 10 heads, and head dimension 128.