OPEN-1B: A Fully Auditable Training Run

TL;DR - OPEN-1B is a 1B-parameter language model whose distributed training trajectory can be independently replayed and verified bit-for-bit across heterogeneous commodity hardware. This enables auditors to check that published checkpoints genuinely came from the disclosed data, code, and training recipe.

• The method imposes fixed ordering on GPU reductions, data batches, and inter/intra-node collective communication to eliminate major sources of training nondeterminism.
• Any individual distributed-training step can be replayed on a single commodity device and compared with the published trajectory.
• A collective verification scheme distributes individual-step audits across many participants, avoiding the infeasible cost of replaying the entire run on one machine.
• The release includes the full pretraining dataset, all intermediate checkpoints, training code, and an audit harness.