Towards a theory of inference-time alignment with unknown rewards
Ranking
Overall
82
Content
95
Popularity
N/A
No observed public metrics; popularity remains neutral/archived.
Merged summary
TL;DR - This paper casts inference-time alignment with unknown rewards as a PAC-style weak-to-strong learning problem. It introduces “alignment dimension,” a combinatorial measure that exactly characterizes whether a reward class is alignment-learnable.
- Learns entirely from data without assuming access to an accurate reward estimate.
- Allows each prompt to have multiple good responses.
- Proves learnability holds if and only if the reward class has finite alignment dimension.
- Uses the one-inclusion graph algorithm to conduct tournaments between incomparable label sets.
Sources (1)
Towards a theory of inference-time alignment with unknown rewards
Public signals
N/A
TL;DR - This paper casts inference-time alignment with unknown rewards as a PAC-style weak-to-strong learning problem. It introduces “alignment dimension,” a combinatorial measure that exactly characterizes whether a reward class is alignment-learnable.
- Learns entirely from data without assuming access to an accurate reward estimate.
- Allows each prompt to have multiple good responses.
- Proves learnability holds if and only if the reward class has finite alignment dimension.
- Uses the one-inclusion graph algorithm to conduct tournaments between incomparable label sets.