LookAgain: Closed-Loop GUI Grounding with Visually Grounded Reflection
Ranking
Observed public metrics from 1 member.
Merged summary
TL;DR - LookAgain is a closed-loop GUI grounding model that treats a predicted click coordinate as a revisable hypothesis, re-examining it with fresh visual evidence before committing. It targets the failure modes that break single-shot grounders — small targets, dense controls, and out-of-distribution UIs — which are the bottleneck for reliable GUI agents.
- Reframes grounding as a multi-turn predict → look-again → refine loop with two primitives:
locate(emit a coordinate, render a marker on the image, append a local patch of the predicted region) andconfirm(accept or reject the hypothesis and terminate). - The key distinction from prior iterative zoom-in methods: refinement operates over the previous prediction as a spatial prior, not merely over the inspected view, and visual evidence is gathered to scrutinize an already-committed coordinate rather than to support the next one.
- Training is two-stage: SFT on constructed reflective trajectories as a cold start, then GRPO using terminal grounding correctness as the sole reward — no dense process supervision.
- Reported state-of-the-art on both refusal-aware and general GUI grounding benchmarks, with ablations supporting the framework's components (specific numbers not provided in the abstract).
Sources (1)
LookAgain: Closed-Loop GUI Grounding with Visually Grounded Reflection
TL;DR - LookAgain is a closed-loop GUI grounding model that treats a predicted click coordinate as a revisable hypothesis, re-examining it with fresh visual evidence before committing. It targets the failure modes that break single-shot grounders — small targets, dense controls, and out-of-distribution UIs — which are the bottleneck for reliable GUI agents.
- Reframes grounding as a multi-turn predict → look-again → refine loop with two primitives:
locate(emit a coordinate, render a marker on the image, append a local patch of the predicted region) andconfirm(accept or reject the hypothesis and terminate). - The key distinction from prior iterative zoom-in methods: refinement operates over the previous prediction as a spatial prior, not merely over the inspected view, and visual evidence is gathered to scrutinize an already-committed coordinate rather than to support the next one.
- Training is two-stage: SFT on constructed reflective trajectories as a cold start, then GRPO using terminal grounding correctness as the sole reward — no dense process supervision.
- Reported state-of-the-art on both refusal-aware and general GUI grounding benchmarks, with ablations supporting the framework's components (specific numbers not provided in the abstract).