OSWorld-Pro: Process-based Evaluation for Computer Use Agents
Ranking
Overall
82
Content
95
Popularity
N/A
No observed public metrics; popularity remains neutral/archived.
Merged summary
TL;DR - OSWorld-Pro is a process-based benchmark for evaluating computer-use agents across intermediate subgoals rather than only final task outcomes. It exposes specific failure modes that end-state evaluation can obscure, helping target improvements in agent reliability and efficiency.
- Includes 300+ tasks, 2,800+ subgoals, and over 67,000 human annotations.
- Uses human-aligned LLM judges to assess progress through sequentially dependent subgoals.
- Claude Opus 5, the top reported performer, scored 75.7% on OSWorld-Pro versus 83.4% on OSWorld.
- Process-level analysis identifies failures such as subgoal-irrelevant actions, keyboard-input errors, and imprecise GUI clicks.
Sources (1)
OSWorld-Pro: Process-based Evaluation for Computer Use Agents
Public signals
N/A
TL;DR - OSWorld-Pro is a process-based benchmark for evaluating computer-use agents across intermediate subgoals rather than only final task outcomes. It exposes specific failure modes that end-state evaluation can obscure, helping target improvements in agent reliability and efficiency.
- Includes 300+ tasks, 2,800+ subgoals, and over 67,000 human annotations.
- Uses human-aligned LLM judges to assess progress through sequentially dependent subgoals.
- Claude Opus 5, the top reported performer, scored 75.7% on OSWorld-Pro versus 83.4% on OSWorld.
- Process-level analysis identifies failures such as subgoal-irrelevant actions, keyboard-input errors, and imprecise GUI clicks.