EnigmaForge: The Question Is Hidden in the Story
Ranking
Overall
82
Content
95
Popularity
N/A
No observed public metrics; popularity remains neutral/archived.
Merged summary
TL;DR - EnigmaForge is a renewable benchmark that asks models to infer both a hidden logic question and its answer from document-like stories. Its results suggest this “intuition” capability differs substantially from fact retrieval and can expose effects from model content filters.
- Each generated puzzle has a SAT-verified unique solution and an ablation certificate showing every clue is necessary.
- Twenty-five frontier models were evaluated on more than 600 instances, producing 17,400 scored records across three matched conditions.
- Intuition scores showed a 22Ă— performance spread, versus 1.6Ă— for fact recovery, substantially reshuffling model rankings.
- Some models performed as well or better without being told the question, while refusals showed that benchmark scores may partly measure content-filter behavior.
Sources (1)
EnigmaForge: The Question Is Hidden in the Story
Public signals
N/A
TL;DR - EnigmaForge is a renewable benchmark that asks models to infer both a hidden logic question and its answer from document-like stories. Its results suggest this “intuition” capability differs substantially from fact retrieval and can expose effects from model content filters.
- Each generated puzzle has a SAT-verified unique solution and an ablation certificate showing every clue is necessary.
- Twenty-five frontier models were evaluated on more than 600 instances, producing 17,400 scored records across three matched conditions.
- Intuition scores showed a 22Ă— performance spread, versus 1.6Ă— for fact recovery, substantially reshuffling model rankings.
- Some models performed as well or better without being told the question, while refusals showed that benchmark scores may partly measure content-filter behavior.