CLI agents beat GUI baselines on mobile tasks without phone-screen access
A study by Li Gu, Zihuan Jiang, and Linqiang Guo shows that coding agents using a command-line interface outperform reproducible GUI baselines on AndroidWorld and MobileWorld benchmarks, with Claude Code (Opus 4.7) reaching 71.8% and 51.9% respectively — surpassing all GUI competitors.
Score breakdown
The results show that CLI-based mobile agents, without any mobile-specific training, already surpass GUI-based agents on established benchmarks while completing tasks in nearly half the steps, establishing CLI as a viable and more efficient paradigm for mobile automation research.
- 01Claude Code (Opus 4.7) scores 71.8% on AndroidWorld and 51.9% on MobileWorld, outperforming all reproducible GUI baselines.
- 02Three GUI baselines tested: GUI-Owl-1.5-32B (69.3% / 43.2%), MAI-UI (68.1% / 26.3%), and Qwen3-VL-32B (57.8% / 13.3%) on AndroidWorld / MobileWorld.
- 03Oracle CLI solutions reach 88.8% on AndroidWorld (103/116 tasks CLI-solvable) and 86.3% on MobileWorld (101/117 tasks CLI-solvable).
Li Gu, Zihuan Jiang, and Linqiang Guo challenge the prevailing GUI-centric view of mobile agents, arguing that the command-line interface (CLI) available on mobile platforms deserves equal consideration. Their study evaluates three coding agents — Claude Code, Terminus-2, and mini-swe-agent — across four model APIs on the AndroidWorld and MobileWorld benchmarks, with no mobile-specific post-training applied. Claude Code (Opus 4.7) leads all tested configurations, scoring 71.8% on AndroidWorld and 51.9% on MobileWorld, exceeding every reproducible GUI baseline: GUI-Owl-1.5-32B (69.3% / 43.2%), MAI-UI (68.1% / 26.3%), and Qwen3-VL-32B (57.8% / 13.3%). The remaining CLI configurations also remain competitive against those GUI baselines.
Every CLI agent outperforms every GUI baseline across all five categories, and CLI agents complete tasks in an average of 10.7 steps compared to 18.6 for GUI agents.
To establish an upper bound for the CLI paradigm, the authors provide oracle CLI solutions that reach 88.8% on AndroidWorld (103 of 116 tasks deemed CLI-solvable) and 86.3% on MobileWorld (101 of 117 tasks CLI-solvable), indicating significant headroom for future improvement. They also introduce the CLI-Advantage Task Suite — 45 task templates across five categories (bulk operations, multi-condition filtering, aggregation, cross-app workflows, and hidden device state) — designed to capture everyday user intents that fall outside the GUI paradigm's scope. Every CLI agent outperforms every GUI baseline across all five categories, and CLI agents complete tasks in an average of 10.7 steps compared to 18.6 for GUI agents. The authors plan to open-source agent implementations, oracle solutions, the CLI-Advantage suite, and evaluation infrastructure to support future research.
Key facts
- 01Claude Code (Opus 4.7) scores 71.8% on AndroidWorld and 51.9% on MobileWorld, outperforming all reproducible GUI baselines.
- 02Three GUI baselines tested: GUI-Owl-1.5-32B (69.3% / 43.2%), MAI-UI (68.1% / 26.3%), and Qwen3-VL-32B (57.8% / 13.3%) on AndroidWorld / MobileWorld.
- 03Oracle CLI solutions reach 88.8% on AndroidWorld (103/116 tasks CLI-solvable) and 86.3% on MobileWorld (101/117 tasks CLI-solvable).
- 04CLI agents complete tasks in an average of 10.7 steps vs. 18.6 steps for GUI agents.
- 05The new CLI-Advantage Task Suite contains 45 templates across five categories: bulk operations, multi-condition filtering, aggregation, cross-app workflows, and hidden device state.
- 06No mobile-specific post-training was applied to any of the three CLI coding agents evaluated.
- 07The authors plan to open-source agent implementations, oracle solutions, the CLI-Advantage suite, and evaluation infrastructure.
Topics
Summary and scoring are generated automatically from the original article. We always link back to the publisher and never republish images or paywalled content. Last processed Jun 19, 2026 · 10:25 UTC. How this works →