🛰️ Daily AI Frontier
‹ back to 2026-08-07

AppDeltaWorld: Transition-Grounded Delta Code World Model for Mobile GUI Agents

arXiv cs.AI LLM Agents Weikai Xu, Yunren Feng, Haoxiang Lei, Kun Huang, Yuxuan Liu, Kang Zhao, Xiaolin Hu, Shuo Shang, Bo An 2026-08-06
Representative image for AppDeltaWorld: Transition-Grounded Delta Code World Model for Mobile GUI Agents

TL;DR - AppDeltaWorld is a GUI world model that predicts the next mobile screen as a constrained, executable HTML "delta code update" rather than a raw image or free-text description, giving agents a scalable synthetic environment when real app trajectories are unavailable due to privacy or cost.

  • Two-level pipeline: retrieves app-specific Level-1 HTML references under an action-transition constraint, then generates Level-2 executable HTML conditioned on current screen, action, predicted next-screen text, and retrieved structure; generated visual assets are inserted into image slots before browser rendering.
  • As a world model, it reports the highest fidelity on CMGUIBench-500 under Code2World evaluation, with gains in structural layout and UI element reconstruction over image-only and code-only baselines.
  • As a training environment, it supports filtered closed-loop SFT data construction; combined with public supervision, AppDeltaAgent reaches state-of-the-art on AndroidLens and consistent gains on MobileGym and MobileWorld.
  • World-model-based test-time reinforcement learning further improves policy adaptation without any additional interaction with real apps.

view merged work →