Rethinking Inference-Time Scaling in Local Computer-Use Agents: Failure Modes and Compute Tradeoffs

TL;DR - A systematic OSWorld study finds that inference-time scaling for resource-constrained local computer-use agents often yields diminishing returns and shifts failure modes rather than reliably improving task success. Selective compute allocation and failure-aware controls may be more effective than uniformly increasing computation.

• More context improves trajectory stability and accuracy, but gains saturate as token costs rise.
• Longer execution horizons reduce max-step stalls without substantially improving success, often prolonging erroneous trajectories.
• Structural decomposition adds planning and formatting overhead to local two-stage agents.
• Parallel scaling partially mitigates failures, but at substantial computational cost.