The Next Screenshot Knows: Gated Hindsight Distillation for Mobile GUI Agents

TL;DR - An arXiv cs.CV preprint introducing Gated Hindsight Distillation (GHD), a training method for mobile GUI agents that exploits the next screenshot as privileged supervision signal during offline training. It matters because standard imitation learning discards the very evidence that justifies an action, leaving agents unable to learn correct reasoning for non-obvious UI paths.

• Problem framing: decomposing trajectories into prefix-action pairs throws away the subsequent observation, where the rationale for an action usually appears (e.g., a menu must open before "Soft Wrap" is visible), so the model rarely samples the correct reasoning.
• Method: a student predicts from the observable trajectory prefix while a parameter-sharing teacher additionally sees the next screenshot and re-scores the student's on-policy responses.
• Gating: distillation is applied selectively — only when the student fails and the hindsight-conditioned teacher recovers the demonstrated action.
• Results: reported improvements in task success over GRPO on AndroidWorld and AndroidLab across two vision-language models; code and checkpoints promised. No numeric figures given in the abstract.