TRACE: Training Reasoning Agents for Causal Exploration with Synthesized Rewards

TL;DR - TRACE trains tool-using reasoning agents to diagnose simulated digital-advertising anomalies using objective rewards derived from hidden interventions. Combining supervised fine-tuning with reinforcement learning improved accuracy beyond prompted frontier models while reducing tool use.

• The simulator injects one of 12 root causes, producing noisy and confounded observations while retaining an oracle label for verifiable rewards.
• Agents use Python and SQL to identify both the root cause and, when relevant, the affected segment.
• On 235 held-out episodes, Qwen3.5-35B-A3B improved from 0.159 FullAttr@1 to 0.637 after supervised fine-tuning and 0.757 after reinforcement learning.
• The RL-trained model outperformed the strongest prompted baseline, Claude Opus 5 at 0.686, suggesting scalable verification can matter more than model size alone.