I am a Humboldt Research Fellow and postdoctoral researcher in the Collaborative Artificial Intelligence Lab at the University of Stuttgart, working with Prof. Andreas Bulling. Before that, I was a visiting researcher and later completed a brief postdoctoral stint in Prof. Antti Oulasvirta’s Computational Behavior Lab at Aalto University. I received my Ph.D. from the Hong Kong University of Science and Technology. Before my Ph.D., I worked as a machine learning engineer at Baidu and as a deep learning researcher at Deepir, led by Prof. Steven Hoi.

My research combines HCI, cognitive science, and AI. I focus on human understanding and cognitive modeling, and build on it toward adaptive assistance and human–AI collaboration. My broader vision is to use AI to empower people toward true inclusion and equality.

Human Understanding & Modeling

  • Human State Inference ESWA’26 AIED’26 ETRA’25 MHCI’24
  • Computational Cognitive & Behavior Modeling Nature Hum. Behav.’26 ICCM’26 IJHCS’25
  • Human Needs & Practices CSCW’25 ASSETS’23, ’22 CHI’22 DIS’21

AI for Human Empowerment

  • Accessible Interaction & Exploration UIST’26🏆 UIST’25, ’24 CHI’25🏅
  • Interactive Guidance & Decision Support UIST’26 TVCG’26, ’25 CHI’24 IMWUT’22
  • Human-Centered AI Collaboration IUI’26 Ongoing
🏆 Best Paper Award · 🏅 Honorable Mention Award

More broadly, I am interested in the intersections of science, technology, art, and philosophy. Beyond research, I explore computational media art; see our artwork at hidden-incidents-of-pet-food.com. I see my own pursuit of truth, authenticity, and equity as a long-running performance art piece, and I also practice performance art in my everyday life. I find flow, freedom, and energy in sports—especially tennis, skiing, and windsurfing. Deep in my heart, one thing I really want to do is to build a utopia.

Please feel free to reach out!

Publications

2026

RemiAssist: A Therapist-Supporting System for Photo-Based Reminiscence Therapy in Dementia Care — author teaser

RemiAssist: A Therapist-Supporting System for Photo-Based Reminiscence Therapy in Dementia Care

Shuchang Xu, Minglong Tang, Junyan Mao, Xiaofu Jin, Wazeer Zulfikar, Yasith Samaradivakara, Jiayi Zhou, Huamin Qu, Yuling Sun, Pattie Maes
In ACM Symposium on User Interface Software and Technology (UIST '26)
Paper
Sonic Stage: Automatically Generating Interactive Spatial Soundscapes to Facilitate Dialogue Video Comprehension for Blind Viewers — author teaser

Sonic Stage: Automatically Generating Interactive Spatial Soundscapes to Facilitate Dialogue Video Comprehension for Blind Viewers

Shuchang Xu, Xiaofu Jin, Gaurav Jain, Wenshuo Zhang, Huamin Qu, Brian A. Smith, Yukang Yan
In ACM Symposium on User Interface Software and Technology (UIST '26)
Best Paper Award
Paper Project Video
Task-sensitive fNIRS–eye-tracking complementarity for adaptive intelligent systems: An attribution–sufficiency diagnostic framework — Figure 2 (top): synchronous fNIRS and eye-tracking acquisition across five task contexts, preprocessing, and feature-level fusion

Task-sensitive fNIRS–eye-tracking complementarity for adaptive intelligent systems: An attribution–sufficiency diagnostic framework

Lina Xu, Bojing Hou, Yuqi Liang, Xiaofu Jin, Haotian Bai, Yuyang Wang, Luwen Yu
In Expert Systems with Applications, 333, 134296 (ESWA '26)
Paper
Hierarchical resource rationality explains human reading behaviour — Figure 1

Hierarchical resource rationality explains human reading behaviour

Yunpeng Bai, Xiaofu Jin, Shengdong Zhao, Antti Oulasvirta
In Nature Human Behaviour (2026)
Paper
Beyond Correctness: A Stage-Aware Framework for Decoding Student Problem-Solving Processes from Handwriting Trajectories — Figure 1

Beyond Correctness: A Stage-Aware Framework for Decoding Student Problem-Solving Processes from Handwriting Trajectories

Zhonghua Sheng, Shuyu Shen, Qiqi Duan, Leixian Shen, Xiaofu Jin, Pan Hui, Huamin Qu, Yuyu Luo
In International Conference on Artificial Intelligence in Education (AIED '26)
Paper
A Resource-Rational Account of Information Scent in Navigation — Figure 1: Model architecture

A Resource-Rational Account of Information Scent in Navigation

Xiaofu Jin, Yunpeng Bai, Antti Oulasvirta
In International Conference on Cognitive Modeling (ICCM '26) Accepted as Talk
Paper
Adaptive Prompt Elicitation for Text-to-Image Generation — Figure 1

Adaptive Prompt Elicitation for Text-to-Image Generation

Xinyi Wen, Lena Hegemann, Xiaofu Jin, Shuai Ma, Antti Oulasvirta
In ACM International Conference on Intelligent User Interfaces (IUI '26)
Paper
TrialCompass: Visual Analytics for Enhancing the Eligibility Criteria Design of Clinical Trials — Figure 1

TrialCompass: Visual Analytics for Enhancing the Eligibility Criteria Design of Clinical Trials

Rui Sheng, Xingbo Wang, Jiachen Wang, Xiaofu Jin, Zhonghua Sheng, Zhenxing Xu, Suraj Rajendran, Huamin Qu, Fei Wang
In IEEE Transactions on Visualization and Computer Graphics, 32(1), 1230–1240 (TVCG '26)
Paper

2025

Pick the Right Thing: GenAI-Supported Design and Production Workflows of Curators — Figure1: Posters, space design and view from virtual exhibition for three case studies exhibitions

Pick the Right Thing: GenAI-Supported Design and Production Workflows of Curators

Jingling Zhang, Yan Xiang, Yuanjin Zhao, Xiaofu Jin, Jia Sun, Xin Tong, Mingming Fan, RAY LC
In International Symposium on Visual Information Communication and Interaction (VINCI '25)
Paper
Toward AI-driven UI transition intuitiveness inspection for smartphone apps — Publisher graphical abstract

Toward AI-driven UI transition intuitiveness inspection for smartphone apps

Xiaozhu Hu, Xiaoyu Mo, Xiaofu Jin, Yuan Chai, Yongquan Hu, Mingming Fan, Tristan Braud
In International Journal of Human-Computer Studies, 206, 103661 (IJHCS '25)
Paper
When Traditional Medicine Meets AI: Critical Considerations for AI-Empowered Clinical Support in Traditional Medicine — Figure 1: The system structure of Shuzhiqihuang

When Traditional Medicine Meets AI: Critical Considerations for AI-Empowered Clinical Support in Traditional Medicine

Yuling Sun, Wenjing Yue, Xiaofu Jin, Shuai Ma, Xiaojuan Ma, Xiaoling Wang
In Proceedings of the ACM on Human-Computer Interaction, 9(7) (CSCW '25)
Paper
SynthLens: Visual Analytics for Facilitating Multi-Step Synthetic Route Design — Figure 3: SynthLens main interface

SynthLens: Visual Analytics for Facilitating Multi-Step Synthetic Route Design

Qipeng Wang, Rui Sheng, Shaolun Ruan, Xiaofu Jin, Chuhan Shi, Min Zhu
In IEEE Transactions on Visualization and Computer Graphics, 31(10), 7647–7660 (TVCG '25)
Paper
Branch Explorer: Leveraging Branching Narratives to Support Interactive 360° Video Viewing for Blind and Low Vision Users — Figure 1: Branch Explorer teaser

Branch Explorer: Leveraging Branching Narratives to Support Interactive 360° Video Viewing for Blind and Low Vision Users

Shuchang Xu, Xiaofu Jin, Wenshuo Zhang, Huamin Qu, Yukang Yan
In ACM Symposium on User Interface Software and Technology (UIST '25)
Paper
Decoding Cognitive Load: Eye-Tracking Insights into Working Memory and Visual Attention — Figure 1: Working memory and visual attention experiment timeline

Decoding Cognitive Load: Eye-Tracking Insights into Working Memory and Visual Attention

Xiaofu Jin, Yunpeng Bai, Lina Xu, Shuai Ma, Danqing Shi, Luwen Yu, Mingming Fan
In ACM Symposium on Eye Tracking Research and Applications (ETRA '25)
Paper
Modeling the Intuitiveness of Mobile GUI Navigation by Understanding and Simulating Users' Attention Distribution — Figure 5: The model structure of UILT

Modeling the Intuitiveness of Mobile GUI Navigation by Understanding and Simulating Users' Attention Distribution

Xiaozhu Hu, Xiaoyu Mo, Xiaofu Jin, Yongquan Owen Hu, Mingming Fan, Tristan Braud
In Companion Proceedings of the ACM Web Conference (WWW Companion '25)
Paper
Systematic Literature Review of Using Virtual Reality as a Social Platform in HCI Community — Figure 3: Review framework and taxonomy

Systematic Literature Review of Using Virtual Reality as a Social Platform in HCI Community

Xiaoying Wei, Xiaofu Jin, Ge Lin Kan, Yukang Yan, Mingming Fan
In Proceedings of the ACM on Human-Computer Interaction, 9(2) (CSCW '25)
Paper
DanmuA11y: Making Time-Synced On-Screen Video Comments (Danmu) Accessible to Blind and Low Vision Users via Multi-Viewer Audio Discussions — Figure 1: DanmuA11y teaser

DanmuA11y: Making Time-Synced On-Screen Video Comments (Danmu) Accessible to Blind and Low Vision Users via Multi-Viewer Audio Discussions

Shuchang Xu, Xiaofu Jin, Huamin Qu, Yukang Yan
In ACM CHI Conference on Human Factors in Computing Systems (CHI '25)
Honorable Mention Award
Paper

2024

EarMonitor: Non-clinical Assessment of Ear Health Conditions Using a Low-cost Endoscope Camera on Smartphones

EarMonitor: Non-clinical Assessment of Ear Health Conditions Using a Low-cost Endoscope Camera on Smartphones

Xiaofu Jin, Mingming Fan
In Proceedings of the ACM on Human-Computer Interaction, Volume 8, Issue MHCI (MHCI'24)
Paper
Exploring the Opportunity of Augmented Reality (AR) in Supporting Older Adults Explore and Learn Smartphone Applications

Exploring the Opportunity of Augmented Reality (AR) in Supporting Older Adults Explore and Learn Smartphone Applications

Xiaofu Jin, Wai Tong, Xiaoying Wei, Xian Wang, Emily Kuang, Xiaoyu Mo, Huamin Qu, Mingming Fan
In ACM CHI Conference on Human Factors in Computing Systems (CHI'24)
Paper
Empowering Autonomous Digital Learning for Older Adults

Empowering Autonomous Digital Learning for Older Adults

Xiaofu Jin
In ACM CHI Conference on Human Factors in Computing Systems (CHI'24 Extended Abstract)
Paper
Memory Reviver: Supporting Photo-Collection Reminiscence for People with Visual Impairment via a Proactive Chatbot

Memory Reviver: Supporting Photo-Collection Reminiscence for People with Visual Impairment via a Proactive Chatbot

Shuchang Xu, Chang Chen, Zichen Liu, Xiaofu Jin, Lin-Ping Yuan, Yukang Yan, Huamin Qu
In Proceedings of ACM Symposium on User Interface Software and Technology (UIST'24)
Paper
Augmented Library: Toward Enriching Physical Library Experience Using HMD-Based Augmented Reality

Augmented Library: Toward Enriching Physical Library Experience Using HMD-Based Augmented Reality

Qianjie Wei, Jingling Zhang, Pengqi Wang, Xiaofu Jin, Mingming Fan
In Proceedings of the 17th International Symposium on Visual Information Communication and Interaction (VINCI '24)
Paper
“Hidden Incidents of Pet Food”: Portraying Taurine Deficiency in Cats

“Hidden Incidents of Pet Food”: Portraying Taurine Deficiency in Cats

Ziwei Wu, Xiaofu Jin, Danlu Fei, Yifang Wang, Kang Zhang
In International Symposium on Electronic Art (ISEA'24)
Paper

2023

Designing Loving-Kindness Meditation in Virtual Reality for Long-Distance Romantic Relationships

Designing Loving-Kindness Meditation in Virtual Reality for Long-Distance Romantic Relationships

Xian Wang#,Xiaoyu Mo#,Lik-Hang Lee,Xiaoying Wei, Xiaofu Jin, Mingming Fan, Pan Hui
In ACM Multimedia (MM'23)
Paper
Understanding Curators' Practices and Challenges of Making Exhibitions More Accessible for Blind and Low Vision People

Understanding Curators' Practices and Challenges of Making Exhibitions More Accessible for Blind and Low Vision People

Yuru Huang#, Jingling Zhang#, Xiaofu Jin, Mingming Fan
In ACM Computers and Accessibility (ASSETS'23)
Paper
Bridging the Generational Gap: Exploring How Virtual Reality Supports Remote Communication Between Grandparents and Grandchildren

Bridging the Generational Gap: Exploring How Virtual Reality Supports Remote Communication Between Grandparents and Grandchildren

Xiaoying Wei, Yizheng Gu, Emily Kuang, Xian Wang, Beiyan Cao, Xiaofu Jin, Mingming Fan
In ACM CHI Conference on Human Factors in Computing Systems (CHI'23)
Paper

2022

Synapse: Interactive Guidance by Demonstration with Trial-and-Error Support for Older Adults to Use Smartphone Apps

Synapse: Interactive Guidance by Demonstration with Trial-and-Error Support for Older Adults to Use Smartphone Apps

Xiaofu Jin, Xiaozhu Hu, Xiaoying Wei, Mingming Fan
In ACM Interactive, Mobile, Wearable and Ubiquitous Technologies (IMWUT'22)
Paper Video
Communication in Immersive Social Virtual Reality: A Systematic Review of 10 Years' Studies

Communication in Immersive Social Virtual Reality: A Systematic Review of 10 Years' Studies

Xiaoying Wei#, Xiaofu Jin#, Mingming Fan
In ACM Chinese CHI (ChineseCHI'22)
Honorable Mention Award
Paper
"I Used To Carry A Wallet, Now I Just Need To Carry My Phone": Understanding Current Banking Practices and Challenges Among Older Adults in China

"I Used To Carry A Wallet, Now I Just Need To Carry My Phone": Understanding Current Banking Practices and Challenges Among Older Adults in China

Xiaofu Jin, Mingming Fan
In ACM Computers and Accessibility (ASSETS' 22)
Paper
"Merging Results Is No Easy Task": An International Survey Study of Collaborative Data Analysis Practices Among UX Practitioners

"Merging Results Is No Easy Task": An International Survey Study of Collaborative Data Analysis Practices Among UX Practitioners

Emily Kuang, Xiaofu Jin, Mingming Fan
In ACM Human Factors in Computing Systems (CHI'22)
Paper

2021

"Too Old to Bank Digitally?": A Survey of Banking Practices and Challenges among Older Adults in China

"Too Old to Bank Digitally?": A Survey of Banking Practices and Challenges among Older Adults in China

Xiaofu Jin, Emily Kuang, Mingming Fan
In ACM Designing Interactive Systems (DIS'21)
Paper Survey Questions

Earlier

Learning the Relation between Interested Objects and Aesthetic Region for Image Cropping

Learning the Relation between Interested Objects and Aesthetic Region for Image Cropping

Peng Lu, Hao Zhang, Xujun Peng, Xiaofu Jin
In IEEE Transactions on Multimedia
Paper Code

Industry Experience

  • Baidu Financial

    Beijing, China

    Machine Learning Algorithm Engineer 2018-2019

  • Deepir Inc

    Beijing, China

    Deep Learning Researcher 2017-2018

Education

  • Hong Kong University of Science and Technology

    Hong Kong, China

    Ph.D., Academy of Interdisciplinary Studies 2022-2025
    Advisor: Prof. Huamin Qu and Prof. Mingming Fan

  • Rochester Institute of Technology (RIT)

    Rochester, NY, USA

    Doctoral studies, Computational and Information Science 2020-2021
    Advisor: Prof. Mingming Fan

  • Beijing University of Posts and Telecommunications (BUPT)

    Beijing, China

    Master of Computer Science and Technology 2017-2020

  • Beijing University of Posts and Telecommunications (BUPT)

    Beijing, China

    Bachelor of Artificial Intelligence and Technology 2013-2017