Skip to content
Thursday, 19 November 2026
Tenesys AI News
Subscribe

All News

TENESYS AI NEWS tracks the most important AI news and explains what changed, why it matters and whether a technology is worth testing.

#MaxText — 2 articles ✕

Models· Important· 🧪 Worth Testing

Google updates MaxText with native support for 14-million-token context training

Google has updated its open-source JAX training framework, MaxText, to support training LLMs at extreme sequence lengths on Cloud TPU hardware. By introducing four distinct context-parallelism strategies, the update enables efficient training of models with up to 14 million tokens, overcoming memory and compute bottlenecks.

Google Developers

Google's TPU Team Reproduces Ai2's Olmo 3 7B Training Run in MaxText

A Google Cloud TPU engineering team worked with Ai2 to rebuild Olmo 3 7B's pre-training from scratch using MaxText, Google's JAX/XLA training framework, running on TPUs instead of Ai2's original PyTorch/GPU setup. They matched Ai2's published results not just on the training loss curve but on four independent held-out evaluation surfaces across the full ~5.93-trillion-token, 1.41-million-step stage-1 run plus the stage-2 annealing phase. Along the way they ported Olmo 3's unusual architecture (reordered-norm blocks, QK-norm, 3:1 sliding/global attention ratio) into MaxText and caught a data-loader bug that had been quietly inflating apparent performance through memorization rather than genuine learning.

Google Developers