Why Are GUI Agents Correct but Late? Decode on the Decision-Time Critical Path, Tested with Pre-Compiled Policy Trees
Ranking
Overall
78
Content
95
Popularity
37
Observed public metrics from 1 member.
Merged summary
TL;DR - AAPT pre-computes bounded GUI action trees during idle periods, moving costly multimodal-model decoding off the decision-time critical path. This raises success on transient events from 0.50 to 0.79 without incorrect actions.
- A lightweight observer routes screen changes to pre-authorized actions for immediate execution.
- Baselines that decoded during execution had zero success within the contested decision window.
- Ablations identify observer speed, valid planning, and especially accurate branch routing as critical.
- Results replicated on another multimodal model across 126 paired trials; reactive methods remain better when actions cannot be enumerated beforehand.
Sources (1)
Why Are GUI Agents Correct but Late? Decode on the Decision-Time Critical Path, Tested with Pre-Compiled Policy Trees
Public signals
Semantic Scholar citations 0 · Semantic Scholar influential citations 0
TL;DR - AAPT pre-computes bounded GUI action trees during idle periods, moving costly multimodal-model decoding off the decision-time critical path. This raises success on transient events from 0.50 to 0.79 without incorrect actions.
- A lightweight observer routes screen changes to pre-authorized actions for immediate execution.
- Baselines that decoded during execution had zero success within the contested decision window.
- Ablations identify observer speed, valid planning, and especially accurate branch routing as critical.
- Results replicated on another multimodal model across 126 paired trials; reactive methods remain better when actions cannot be enumerated beforehand.