Findings of EMNLP 2026 (Accepted) ASIL: Replacing Screenshot-and-Click with Structured State and Semantic Actions
First Author 2026
An agent-native interface that replaces screenshot-and-click control with structured software state and code-executable semantic actions, reaching 81.6 on GPT-5.4 under a 15-action budget versus 6.6 for screenshot GUI control under 50 steps.
Rui Xie, Lu Chen
arXiv Technical Report, 2026 Qwen-CUA: Native Computer Use for (almost) Everything
Core Contributor 2026
A technical report presenting a native computer-use agent that operates software from screenshots through keyboard and mouse actions, supported by large-scale verifiable tasks and interactive rollouts.
Dunjie Lu, Shuai Bai, Tianyi Bai, …, Rui Xie, …, Fan Zhou
arXiv Technical Report, 2026 RecreationWorld: Scalable and Verifiable Environments for Hybrid Computer-Use Agents
Contributing Author 2026
A framework for training and evaluating hybrid agents that explore interfaces, write code, and verify running applications across Ubuntu, macOS, Windows, Android, and Web, with 250 held-out RecreationBench tasks.
Shuai Bai, Jiayong Deng, Sicheng Fan, …, Rui Xie, …, Bowen Zhou
Qwen Release Announcement, 2026 Qwen3.8-Max: A New Bar for Coding and Cowork
Team Contributor 2026
The Qwen team's August 2026 flagship model release, advancing autonomous coding, professional workflows, long-horizon planning, and native multimodal agent capabilities with a 2.4-trillion-parameter MoE model.
Qwen Team
ECCV 2026 (Accepted) GUIDE: Resolving Domain Bias in GUI Agents through Real-Time Web Video Retrieval and Plug-and-Play Annotation
First Author 2026
A training-free, plug-and-play framework that retrieves task-relevant tutorial videos and distills transferable planning and grounding knowledge for domain-specific GUI agents, improving OSWorld average score by 4.47 to 7.48 percentage points without modifying agent parameters.
Rui Xie, Zhi Gao, Chenrui Shi, Zirui Shang, Lu Chen, Qing Li
arXiv Preprint, 2026 MatToolBench: Benchmarking Multimodal Agents in Real-World Materials Science Workflows
Co-first Author 2026
A real-environment benchmark with 204 tasks across 10 materials science tools, covering GUI operation, OriginPro scripting, and database queries, with expert-defined partial-credit scoring.
Mei Wu, Rui Xie, Runyu Zhang, Yuqiang Li, Tianfan Fu, Bo Chen, Kai Yu, Xin Chen, Lu Chen
AAAI 2026 TongUI: Internet-Scale Trajectories from Multimodal Web Tutorials for Generalized GUI Agents
Contributing Author 2026
A framework that transforms multimodal web tutorials into GUI-Net-1M, a dataset of one million trajectories across five operating systems and over 280 applications, supporting generalized GUI-agent training and evaluation.
Bofei Zhang, Zirui Shang, Zhi Gao, Wang Zhang, Rui Xie, Xiaojian Ma, Tao Yuan, Xinxiao Wu, Song-Chun Zhu, Qing Li
arXiv Preprint, 2023 Mobile-Env: Building Qualified Evaluation Benchmarks for LLM-GUI Interaction
Third Author 2023
A benchmark effort for evaluating LLM-based GUI interaction in mobile environments with isolated tasks, simulator support, and behavior analysis.
Danyang Zhang, Zhennan Shen, Rui Xie, Situo Zhang, Tianbao Xie, Zihan Zhao, Siyuan Chen, Lu Chen, Hongshen Xu, Ruisheng Cao, Kai Yu
