What changed
Ai2 released Olmo-core 3, the open training stack behind its large mixture-of-experts work. The code adds new parallelism and expert-routing infrastructure, MXFP8 support and checkpoint tooling, and the team reports tests from a 47B-parameter MoE through a 1.2T benchmark plus a short 2.38T capacity run.
Why it matters
Open training systems rarely expose infrastructure intended for trillion-parameter mixtures. The release gives research and infrastructure teams code they can inspect and adapt, while separating claimed systems throughput from evidence of a completed model training run.
What to do
Reproduce the smallest published configuration first. Pin hardware, topology, precision, token batch and expert routing; measure tokens per second per GPU, loss stability, checkpoint restart, expert balance and total utilization before extrapolating to a larger cluster.
Caveat
The performance results are Ai2's controlled benchmarks, including about 2.7 times the older FSDP throughput for one 47B MoE setup and a 21% MXFP8 uplift in a separate test. The 2.38T run was a short capacity test, not evidence of full training or model quality at that scale.