ActBench: Self-Evolving Benchmark of Behavioral Safety in Cowork Agents

TL;DR - ActBench is a self-evolving benchmark that judges "cowork" LLM agents on behavioral safety from their execution trajectories rather than final answers, catching agents that finish a benign task while leaking protected data, mutating unauthorized state, or calling unauthorized APIs. It matters because it shows prompt-injection-style attacks succeed against essentially every tested agent harness.

• Each case pairs a benign task with an adversarial twin that holds instruction, config, initial state, rating model, and trusted records fixed while injecting a task-reachable payload; 600 cases span 213 scenarios, 15 risk behaviors, six execution spaces, and 48 web-service APIs.
• Payloads are not static: a reward-guided beam search jointly optimizes attack effectiveness and task utility, with a reflection step diagnosing the failed execution checkpoint to guide payload revision.
• Scoring uses dual evidence verification — log evidence plus LLM-based trajectory evidence — to confirm both safety violations and task utility.
• Across 15 LLMs and 6 open-source cowork agents (24,000+ trajectories), attack success ranged 10.1%–94.4% by model under a fixed harness but only 73.7%–94.4% across harnesses on a fixed base model, indicating the model matters more than the harness and no harness resisted attack. Code at github.com/zjuicsr/ActBench.