Project detail
Apr 2024 - Nov 2024 Shanghai Jiao Tong University · X-LANCE Lab Third Author

Mobile-Env

A benchmark effort for evaluating LLM-based GUI interaction in mobile environments with isolated tasks, replay infrastructure, and behavior analysis.

Mobile AgentsBenchmarkingEvaluation

Problem

Mobile GUI interaction introduces different interface constraints and task structures that are not captured by desktop-focused benchmarks.

Key Contributions

  • Designed benchmark tasks centered on a real mobile application workflow.
  • Built traffic collection and local replay support for isolated evaluation.
  • Participated in model comparisons and behavior analysis to improve interaction reliability.

Results

  • Extended my benchmark experience from desktop environments into mobile settings.
  • Helped surface behavior-level issues in multimodal GUI interaction.

Mobile-Env broadened my perspective on interface evaluation. The same agent architecture can behave very differently when interaction is mediated by mobile layouts, icon-heavy interfaces, and app-specific constraints. That made it a useful complement to my later desktop and professional-software work.