Cross-Entropy Risk Estimation for Language Models: Inconsistency Must Be Dense, and the Holdout Method Is No Exception

TL;DR - This paper proves that language models’ held-out per-token cross-entropy risk cannot be consistently estimated across all possible data distributions and trained models. It identifies bounded probability flooring and thresholded risk reporting as two qualified remedies.

• Finite- and infinite-risk states can lie arbitrarily close, preventing any estimator—not only holdout averages—from being universally consistent.
• The impossibility persists with bounded expected sequence length and full-support models; under full support, inconsistent states are dense.
• With bounded context, flooring next-token probabilities makes finite risk equivalent to finite expected sequence length.
• Reporting risk only below a predetermined threshold restores consistency while retaining what model selection requires, but changes the estimation objective.