SEE: Structure-aware Exploring \& Exploiting for Long-horizon GUI Agent Trajectory Synthesis
Merged summary
TL;DR - SEE synthesizes diverse, long-horizon training trajectories for GUI agents by mapping app interfaces into transition graphs and planning paths through them. This reduces dependence on human demonstrations and improves agents’ success and generalization to unseen screens.
- Builds explicit transition graphs spanning screens and UI elements.
- Uses graph planning and controlled sampling to compose reproducible, explainable trajectories.
- Generates 14.8-step trajectories on average while preventing spurious cycles.
- Fine-tuning on SEE improves performance across multiple real-world apps.
Sources (1)
SEE: Structure-aware Exploring \& Exploiting for Long-horizon GUI Agent Trajectory Synthesis
TL;DR - SEE synthesizes diverse, long-horizon training trajectories for GUI agents by mapping app interfaces into transition graphs and planning paths through them. This reduces dependence on human demonstrations and improves agents’ success and generalization to unseen screens.
- Builds explicit transition graphs spanning screens and UI elements.
- Uses graph planning and controlled sampling to compose reproducible, explainable trajectories.
- Generates 14.8-step trajectories on average while preventing spurious cycles.
- Fine-tuning on SEE improves performance across multiple real-world apps.