RX Rui Xie Research profile
Research profile

Building agents for software and everyday work.

Rui Xie PhD Student in Computer Science · Shanghai Jiao Tong University

I am a PhD student in Computer Science at Shanghai Jiao Tong University, working toward generally capable agents that can take on work across software, devices, and everyday environments.

I care about agent systems that can complete real work, from computer operation to broader everyday tasks, with an emphasis on grounded interaction, reliable training pipelines, and evaluation in real settings.

Email Download CV
Research Directions
GUI agentsLLM agentsRLAGI
Selected publications

Recent papers and ongoing research threads.

View all publications
Under Review

GUIDE: Resolving Domain Bias in GUI Agents through Real-Time Web Video Retrieval and Plug-and-Play Annotation

First Author 2026

A plug-and-play framework that retrieves task-relevant tutorial videos and distills transferable planning and grounding knowledge for domain-specific GUI agents.

Rui Xie, Zhi Gao, Chenrui Shi, Zirui Shang, Lu Chen, Qing Li
In Preparation

MatToolBench: A Real-Environment Benchmark for Evaluating Multimodal Agents on Professional Materials Science Software

Co-first Author 2026

A real-environment benchmark that evaluates multimodal agents on professional materials science workflows spanning GUI tools, code execution, and cross-tool coordination.

Mei Wu, Rui Xie, Runyu Zhang, Lu Chen, Bo Chen, Kai Yu, Xin Chen
AAAI 2026

TongUI: Internet-Scale Trajectories from Multimodal Web Tutorials for Generalized GUI Agents

Contributing Author 2026

A large-scale data construction effort that turns multimodal web tutorials into GUI trajectories for generalized agent training and evaluation.

Bofei Zhang, Zirui Shang, Zhi Gao, Wang Zhang, Rui Xie, Xiaojian Ma, Tao Yuan, Xinxiao Wu, Song-Chun Zhu, Qing Li
Featured projects

Flagship work on agents, evaluation, and real-environment systems.

Browse all projects
GUIDE framework overview
Mar 2025 - Present First Author

GUIDE

A plug-and-play framework for reducing planning and grounding bias in domain-specific GUI agents by retrieving and distilling task-relevant tutorial knowledge.

GUI AgentsRetrieval-Augmented SystemsGroundingPlanning
Open project page
MatToolBench execution architecture
Oct 2025 - Present Co-first Author

MatToolBench

A real-environment benchmark for multimodal agents working with professional materials science software, covering GUI operation, code execution, and cross-tool workflows.

BenchmarkingReal-Environment EvaluationScientific SoftwareAgent Infrastructure
Open project page
Research timeline

From systems foundations to multimodal agent research.

2025 - Present

PhD Student

Shanghai Jiao Tong University · X-LANCE Lab

Researching multimodal agents with an emphasis on GUI interaction, domain adaptation, and real-environment evaluation.

2025

Research Intern

Beijing Institute for General Artificial Intelligence

Worked on multi-app desktop agents, early-stage reinforcement learning pipelines, and practical software interaction systems.

2024 - 2025

Research Collaborator

Shanghai Jiao Tong University / BIGAI

Contributed to large-scale trajectory data construction and mobile GUI evaluation benchmarks.

2021 - 2025

B.Eng. in Computer Science

Shanghai Jiao Tong University

Built a strong foundation in systems, algorithms, architecture, and machine learning while moving into agent research.

CV and contact

Available for research conversations, collaborations, and project discussions.

The easiest way to reach me is by email. My current CV is available as a direct download, and this site will continue to grow as a durable record of publications, systems work, and research notes.

Email Download CV