FinEvo-Bench: A Longitudinal Benchmark for Self-Evolving Agents in Professional Financial Workflows
Ranking
Observed public metrics from 1 member.
Merged summary
TL;DR - FinEvo-Bench is a longitudinal benchmark of 120 real-case-grounded financial tasks (20 business scenes, six domains) designed to measure whether self-evolving agents actually convert experience from earlier tasks into better later performance. It matters because most agent benchmarks score tasks independently and cannot detect cross-task learning in professional, open-ended workflows.
- Design: institution-provided procedures define required operations/constraints; each scene has six distinct cases sharing a procedure and a manually reviewed rubric covering task quality and financial compliance; tasks are delivered as three independently shuffled, globally interleaved streams.
- Setup: four self-evolving scaffolds on a shared Qwen3.7-Max backbone, with paired non-evolving controls isolating self-evolution gain; outputs scored by an independent Claude Code agent backed by Claude Opus 4.6.
- Results: Letta scores highest evolved (91.65) with fewest compliance issues (0.09/task); Codex shows the largest gain (+19.37). Evolving conditions add 9.33–19.37 points and cut compliance issues by 0.12–0.44 per task.
- Ablations: gains are larger at within-scene ranks 4–6 than 1–3 (by 6.10–8.70 points), indicating cumulative learning; in Claude Code, skill-only evolution beats memory-only and combined memory-skill, and rubric feedback outperforms reference-answer feedback across all scaffolds.
Sources (1)
FinEvo-Bench: A Longitudinal Benchmark for Self-Evolving Agents in Professional Financial Workflows
TL;DR - FinEvo-Bench is a longitudinal benchmark of 120 real-case-grounded financial tasks (20 business scenes, six domains) designed to measure whether self-evolving agents actually convert experience from earlier tasks into better later performance. It matters because most agent benchmarks score tasks independently and cannot detect cross-task learning in professional, open-ended workflows.
- Design: institution-provided procedures define required operations/constraints; each scene has six distinct cases sharing a procedure and a manually reviewed rubric covering task quality and financial compliance; tasks are delivered as three independently shuffled, globally interleaved streams.
- Setup: four self-evolving scaffolds on a shared Qwen3.7-Max backbone, with paired non-evolving controls isolating self-evolution gain; outputs scored by an independent Claude Code agent backed by Claude Opus 4.6.
- Results: Letta scores highest evolved (91.65) with fewest compliance issues (0.09/task); Codex shows the largest gain (+19.37). Evolving conditions add 9.33–19.37 points and cut compliance issues by 0.12–0.44 per task.
- Ablations: gains are larger at within-scene ranks 4–6 than 1–3 (by 6.10–8.70 points), indicating cumulative learning; in Claude Code, skill-only evolution beats memory-only and combined memory-skill, and rubric feedback outperforms reference-answer feedback across all scaffolds.