Explore my work across agent interfaces, computer use, training, and real-environment evaluation, with links to papers, project pages, code, models, and datasets.
Rui Xie, Lu Chen
An agent-native interface that replaces screenshot-and-click control with structured software state and code-executable semantic actions, reaching 81.6 on GPT-5.4 under a 15-action budget versus 6.6 for screenshot GUI control under 50 steps.
Dunjie Lu, Shuai Bai, Tianyi Bai, …, Rui Xie, …, Fan Zhou
A technical report presenting a native computer-use agent that operates software from screenshots through keyboard and mouse actions, supported by large-scale verifiable tasks and interactive rollouts.
Shuai Bai, Jiayong Deng, Sicheng Fan, …, Rui Xie, …, Bowen Zhou
A framework for training and evaluating hybrid agents that explore interfaces, write code, and verify running applications across Ubuntu, macOS, Windows, Android, and Web, with 250 held-out RecreationBench tasks.
Qwen Team
The Qwen team's August 2026 flagship model release, advancing autonomous coding, professional workflows, long-horizon planning, and native multimodal agent capabilities with a 2.4-trillion-parameter MoE model.
Rui Xie, Zhi Gao, Chenrui Shi, Zirui Shang, Lu Chen, Qing Li
A training-free, plug-and-play framework that retrieves task-relevant tutorial videos and distills transferable planning and grounding knowledge for domain-specific GUI agents, improving OSWorld average score by 4.47 to 7.48 percentage points without modifying agent parameters.
Mei Wu, Rui Xie, Runyu Zhang, Yuqiang Li, Tianfan Fu, Bo Chen, Kai Yu, Xin Chen, Lu Chen
A real-environment benchmark with 204 tasks across 10 materials science tools, covering GUI operation, OriginPro scripting, and database queries, with expert-defined partial-credit scoring.
Bofei Zhang, Zirui Shang, Zhi Gao, Wang Zhang, Rui Xie, Xiaojian Ma, Tao Yuan, Xinxiao Wu, Song-Chun Zhu, Qing Li
A framework that transforms multimodal web tutorials into GUI-Net-1M, a dataset of one million trajectories across five operating systems and over 280 applications, supporting generalized GUI-agent training and evaluation.
Danyang Zhang, Zhennan Shen, Rui Xie, Situo Zhang, Tianbao Xie, Zihan Zhao, Siyuan Chen, Lu Chen, Hongshen Xu, Ruisheng Cao, Kai Yu
A benchmark effort for evaluating LLM-based GUI interaction in mobile environments with isolated tasks, simulator support, and behavior analysis.