Skip to content
Thursday, 19 November 2026
Tenesys AI News
Subscribe

AI Research

Research papers, benchmarks, architectures, reasoning and training methods.

#Training Infrastructure — 2 articles ✕

Research· Important

Google and Meta bring native TPU support to PyTorch's TorchTitan training framework

Google introduced TorchTPU, a native PyTorch backend for Cloud TPUs (torch.device("tpu")), and co-developed a fork of Meta's TorchTitan trainer adapted for TPU training. Previously, running PyTorch training on TPUs meant rewriting code in JAX or relying on the older torch_xla path; now only minimal device-initialization changes are needed. The fork provides tested recipes for pre-training large dense models (Llama 3 70B/405B, Qwen 3 8B/32B), Mixture-of-Experts models (Qwen 3 235B), and diffusion transformers (FLUX.1 12B), plus a 'Graph Trainer' that captures an entire training step as one XLA program for better compiler optimization.

Google Developers

Google's TPU Team Reproduces Ai2's Olmo 3 7B Training Run in MaxText

A Google Cloud TPU engineering team worked with Ai2 to rebuild Olmo 3 7B's pre-training from scratch using MaxText, Google's JAX/XLA training framework, running on TPUs instead of Ai2's original PyTorch/GPU setup. They matched Ai2's published results not just on the training loss curve but on four independent held-out evaluation surfaces across the full ~5.93-trillion-token, 1.41-million-step stage-1 run plus the stage-2 annealing phase. Along the way they ported Olmo 3's unusual architecture (reordered-norm blocks, QK-norm, 3:1 sliding/global attention ratio) into MaxText and caught a data-loader bug that had been quietly inflating apparent performance through memorization rather than genuine learning.

Google Developers