🛰️ Daily AI Frontier
‹ back to 2026-08-20

用过一次就能学会!哈工大开源自进化GUI Agent,跨App连续干活

WeChat: PaperWeekly LLM Agents 2026-08-18
Representative image for 用过一次就能学会!哈工大开源自进化GUI Agent,跨App连续干活

TL;DR - Harbin Institute of Technology has open-sourced KnowAct-GUIClaw, a self-improving GUI-agent framework for long-running workflows across apps and platforms. It matters because it turns execution traces into reusable memories and validated skills, improving reliability and efficiency rather than treating every task as new.

  • A Host Agent manages context, memory, tools, and workflow orchestration, while the pluggable GUIClaw handles real-time interface interaction across Android, iOS, HarmonyOS, and Windows.
  • Its Know–Route–Act–Reflect pipeline uses a typed Blackboard to pass structured data between apps and stops when required inputs or outputs are missing instead of guessing.
  • The system extracts textual experience and parameterized, state-validated skills from successful and failed trajectories, including verified deep links and targeted skill repair.
  • On MobileWorld GUI-Only tasks with Kimi-K2.6, success rose from 55.6% to 64.1% with Host, memory, and self-evolving skills; transferred capabilities improved Qwen3.5-35B-A3B by 16.2 percentage points, while skill reuse reduced token use by about 6%.

view merged work →