DocOps: A Verifiable Benchmark for Autonomous Agents in Complex Document Operations

TL;DR - DocOps is a deterministically verifiable benchmark for evaluating autonomous agents on increasingly complex document operations. It finds that frontier agent configurations still struggle with coupled, long-range workflows, highlighting barriers to reliable workspace automation.

• Uses a hierarchical taxonomy to decompose realistic document operations into atomic capabilities and escalating workflow complexities.
• Evaluates representative open- and closed-source models across multiple agentic harnesses.
• Identifies three major failure modes: long-term state-tracking collapse, shallow semantic verification, and destructive editing of structural metadata.
• Emphasizes global consistency and non-destructive editing as priorities for more robust document agents.