RX Rui Xie Research profile
Research profile
Portrait of Rui Xie
Institution
Shanghai Jiao Tong University (SJTU)
Lab
X-LANCE Lab
Experience timeline
  1. May 2026 - Present Research Intern Alibaba · Qwen Agent Post-Training Group
  2. 2025 - Present PhD Student Shanghai Jiao Tong University · X-LANCE Lab
  3. Mar 2025 - Aug 2025 Research Intern Beijing Institute for General Artificial Intelligence (BIGAI)
  4. 2021 - 2025 B.Eng. in Computer Science Shanghai Jiao Tong University

Building agents for software and everyday work.

Rui Xie PhD Student in Computer Science · Shanghai Jiao Tong University

I am a PhD student in Computer Science at Shanghai Jiao Tong University, working toward generally capable agents that can take on work across software, devices, and everyday environments.

I care about agent systems that can complete real work, from computer operation to broader everyday tasks, with an emphasis on grounded interaction, reliable training pipelines, and evaluation in real settings.

Since May 2026, I have been interning with Alibaba's Qwen team, working on post-training for agent capabilities in Qwen mainline models and task testing for computer-use domains.

Research Directions
GUI agentsLLM agentsAgentic RLAGI
Email Download CV
Selected publications

Recent papers and ongoing research threads.

View all publications
Under Review

ASIL: Replacing Screenshot-and-Click with Structured State and Semantic Actions

First Author 2026

An agent-native interface for software-operating agents that replaces screenshot observation and low-level GUI events with structured software state and code-executable semantic actions.

Rui Xie, Lu Chen
Under Review

GUIDE: Resolving Domain Bias in GUI Agents through Real-Time Web Video Retrieval and Plug-and-Play Annotation

First Author 2026

A plug-and-play framework that retrieves task-relevant tutorial videos and distills transferable planning and grounding knowledge for domain-specific GUI agents.

Rui Xie, Zhi Gao, Chenrui Shi, Zirui Shang, Lu Chen, Qing Li
Under Review

MatToolBench: A Real-Environment Benchmark for Evaluating Multimodal Agents on Professional Materials Science Software

Co-first Author 2026

A real-environment benchmark that evaluates multimodal agents on professional materials science workflows spanning GUI tools, code execution, and cross-tool coordination.

Mei Wu, Rui Xie, Runyu Zhang, Lu Chen, Bo Chen, Kai Yu, Xin Chen
AAAI 2026

TongUI: Internet-Scale Trajectories from Multimodal Web Tutorials for Generalized GUI Agents

Contributing Author 2026

A large-scale data construction effort that turns multimodal web tutorials into GUI trajectories for generalized agent training and evaluation.

Bofei Zhang, Zirui Shang, Zhi Gao, Wang Zhang, Rui Xie, Xiaojian Ma, Tao Yuan, Xinxiao Wu, Song-Chun Zhu, Qing Li
Featured projects

Flagship work on agents, evaluation, and real-environment systems.

Browse all projects
ASIL benchmark coverage and interface effect
May 2026 - Present First Author

ASIL

An agent-native interface for software-operating agents that replaces screenshot-and-click control with structured state and semantic actions.

GUI AgentsSoftware InteractionAgentic RLBenchmarking
Open project page
GUIDE framework overview
Mar 2025 - Present First Author

GUIDE

A plug-and-play framework for reducing planning and grounding bias in domain-specific GUI agents by retrieving and distilling task-relevant tutorial knowledge.

GUI AgentsRetrieval-Augmented SystemsGroundingPlanning
Open project page
MatToolBench execution architecture
Oct 2025 - Present Co-first Author

MatToolBench

A real-environment benchmark for multimodal agents working with professional materials science software, covering GUI operation, code execution, and cross-tool workflows.

BenchmarkingReal-Environment EvaluationScientific SoftwareAgent Infrastructure
Open project page
Research timeline

From systems foundations to multimodal agent research.

May 2026 - Present

Research Intern

Alibaba · Qwen Agent Post-Training Group

Working on post-training for agent capabilities in Qwen mainline models, with a focus on computer-use task testing and software-use workflows.

2025 - Present

PhD Student

Shanghai Jiao Tong University · X-LANCE Lab

Researching multimodal agents with an emphasis on GUI interaction, domain adaptation, and real-environment evaluation.

Mar 2025 - Aug 2025

Research Intern

Beijing Institute for General Artificial Intelligence (BIGAI)

Worked on multi-app macOS agents and early GUI-agent reinforcement learning pipelines, including desktop automation, task generation, and DAPO-based training adaptation.

2024 - 2025

Research Collaborator

Shanghai Jiao Tong University / BIGAI

Contributed to large-scale trajectory data construction and mobile GUI evaluation benchmarks.

2021 - 2025

B.Eng. in Computer Science

Shanghai Jiao Tong University

Built a strong foundation in systems, algorithms, architecture, and machine learning while moving into agent research.

CV and contact

Available for research conversations, collaborations, and project discussions.

The easiest way to reach me is by email. My current CV is available as a direct download, and this site will continue to grow as a durable record of publications, systems work, and research notes.

Email Download CV