Entropy-Regularized Rank-Masked Policy Optimization for Test-Time Reinforcement Learning in Code Generation

TL;DR - ERPO enables test-time reinforcement learning for code generation by rewarding behavioral agreement among candidate programs on automatically constructed probe inputs. It also limits reward hacking and policy drift, improving pass@1 and pass@k across in-domain adaptation and zero-shot transfer benchmarks.

• Probe-driven TTRL generates output-free test inputs from problem statements and executes candidate programs on them.
• Probe Consensus Reward compares program behavior rather than source-code surface form, providing a training signal without canonical answers.
• Rank masking makes conservative negative updates when consensus is low, while an entropy ceiling constrains policy drift.
• Behavioral consensus remains an imperfect verifier, so ERPO explicitly addresses the risk of spurious agreement.