Can Agents Deceive? Evaluating Reasoning and Deception in ParliamentBench using a Social Deduction Game

TL;DR - ParliamentBench is an open-source benchmark using a social deduction game to measure LLM agents’ reasoning, persuasion, and deception. Frontier models perform strongly, but most struggle to sustain a consistent deceptive persona.

• Evaluates 16 LLMs across 1,600 simulated matches involving model-model, model-human, and online-game comparisons.
• Introduces metrics for social deduction, reasoning, and deceptive consistency.
• GPT-5.4, Kimi K2.5, Grok 4.1 Fast, and DeepSeek 3.1 Terminus form the strongest-performing cluster.
• Weaker models underperform random and simple algorithmic baselines; deception retention falls below 50% for most models.