I am a fourth-year Ph.D. candidate in the Academy of Interdisciplinary Studies at The Hong Kong University of Science and Technology (HKUST), advised by Prof. Fan Mingming and Prof. Qu Huamin. My current research focuses on developing and applying computational methods—such as cognitive modeling and machine learning—to understand and enhance human adaptive learning behaviors.

My work advances adaptive learning systems in two key directions: (1) leveraging human-centered design and emerging technologies—such as augmented reality (AR)—to create interactive support mechanisms that provide multi-modal, context-aware assistance, and (2) applying cognitive science and machine learning to differentiate cognitive load types and model trial-and-error learning strategies through reinforcement learning, enabling systems to provide human-like adaptive assistance. By integrating cognitive modeling, machine learning, and interactive system design, I aim to bridge the gap between theoretical insights and practical applications, creating technologies that adapt seamlessly to users’ cognitive states and learning needs. Beyond my core research, I am actively involved in AI-assisted healthcare, large language model (LLM)-supported accessibility for blind and low-vision (BLV) users, and the intersection of AI with computational media arts.

Currently, I am a visiting Ph.D. student at Prof. Antti Oulasvirta’s Computational Behavior Lab at Aalto University, Finland. Prior to this, I have been worked as a machine learning engienner in Baidu and deep learning researcher in Deepir, led by Prof. Steven Hoi. My work is driven by a passion for leveraging AI to uncover human nature and address pressing social challenges and contribute to a healthier, more inclusive society.

If my research interests align with yours or you’d like to collaborate, feel free to reach out via email! I am actively seeking academic or industry research positions!

Research


DanmuA11y: Making Time-Synced On-Screen Video Comments (Danmu) Accessible to Blind and Low Vision Users via Multi-Viewer Audio Discussions

Shuchang Xu, Xiaofu Jin, Huamin Qu, Yukang Yan
In ACM CHI Conference on Human Factors in Computing Systems (CHI'25)

EarMonitor: Non-clinical Assessment of Ear Health Conditions Using a Low-cost Endoscope Camera on Smartphones

Xiaofu Jin, Mingming Fan
In Proceedings of the ACM on Human-Computer Interaction, Volume 8, Issue MHCI (MHCI'24)

Exploring the Opportunity of Augmented Reality (AR) in Supporting Older Adults Explore and Learn Smartphone Applications

Xiaofu Jin, Wai Tong, Xiaoying Wei, Xian Wang, Emily Kuang, Xiaoyu Mo, Huamin Qu, Mingming Fan
In ACM CHI Conference on Human Factors in Computing Systems (CHI'24)

Empowering Autonomous Digital Learning for Older Adults

Xiaofu Jin
In ACM CHI Conference on Human Factors in Computing Systems (CHI'24 Extended Abstract)

Memory Reviver: Supporting Photo-Collection Reminiscence for People with Visual Impairment via a Proactive Chatbot

Shuchang Xu, Chang Chen, Zichen Liu, Xiaofu Jin, Lin-Ping Yuan, Yukang Yan, Huamin Qu
In Proceedings of ACM Symposium on User Interface Software and Technology (UIST'24)

SynthLens: Visual Analytics for Facilitating Multi-step Synthetic Route Design

Qipeng Wang, Rui Sheng, Shaolun Ruan, Xiaofu Jin, Chuhan Shi, Min Zhu
In IEEE Transactions on Visualization and Computer Graphics (TVCG'24)

Systematic Literature Review of Using Virtual Reality as a Social Platform in HCI Community

Xiaoying Wei, Xiaofu Jin, Ge Lin Kan, Yukang Yan, Mingming Fan
In ACM conference on Computer-Supported Cooperative Work and Social Computing (CSCW'24)

Augmented Library: Toward Enriching Physical Library Experience Using HMD-Based Augmented Reality

Qianjie Wei, Jingling Zhang, Pengqi Wang, Xiaofu Jin, Mingming Fan
In Proceedings of the 17th International Symposium on Visual Information Communication and Interaction (VINCI '24)

“Hidden Incidents of Pet Food”: Portraying Taurine Deficiency in Cats

Ziwei Wu, Xiaofu Jin, Danlu Fei, Yifang Wang, Kang Zhang
In International Symposium on Electronic Art (ISEA'24)

Designing Loving-Kindness Meditation in Virtual Reality for Long-Distance Romantic Relationships

Xian Wang#,Xiaoyu Mo#,Lik-Hang Lee,Xiaoying Wei, Xiaofu Jin, Mingming Fan, Pan Hui
In ACM Multimedia (MM'23)

Understanding Curators' Practices and Challenges of Making Exhibitions More Accessible for Blind and Low Vision People

Yuru Huang#, Jingling Zhang#, Xiaofu Jin, Mingming Fan
In ACM Computers and Accessibility (ASSESTS'23)

Bridging the Generational Gap: Exploring How Virtual Reality Supports Remote Communication Between Grandparents and Grandchildren

Xiaoying Wei, Yizheng Gu, Emily Kuang, Xian Wang, Beiyan Cao, Xiaofu Jin, Mingming Fan
In ACM CHI Conference on Human Factors in Computing Systems (CHI'23)

Synapse: Interactive Guidance by Demonstration with Trial-and-Error Support for Older Adults to Use Smartphone Apps

Xiaofu Jin, Xiaozhu Hu, Xiaoying Wei, Mingming Fan
In ACM Interactive, Mobile, Wearable and Ubiquitous Technologies (IMWUT'22)

Communication in Immersive Social Virtual Reality: A Systematic Review of 10 Years' Studies

Xiaoying Wei#, Xiaofu Jin#, Mingming Fan
In ACM Chinese CHI (ChineseCHI'22)

"I Used To Carry A Wallet, Now I Just Need To Carry My Phone": Understanding Current Banking Practices and Challenges Among Older Adults in China

Xiaofu Jin, Mingming Fan
In ACM Computers and Accessibility (ASSETS' 22)

"Merging Results Is No Easy Task": An International Survey Study of Collaborative Data Analysis Practices Among UX Practitioners

Emily Kuang, Xiaofu Jin, Mingming Fan
In ACM Human Factors in Computing Systems (CHI'22)

"Too Old to Bank Digitally?": A Survey of Banking Practices and Challenges among Older Adults in China

Xiaofu Jin, Emily Kuang, Mingming Fan
In ACM Designing Interactive Systems (DIS'21)

Learning the Relation between Interested Objects and Aesthetic Region for Image Cropping

Peng Lu, Hao Zhang, Xujun Peng, Xiaofu Jin
In IEEE Transactions on Multimedia

Work Experience


  • Baidu Financial

    Beijing, China

    Machine Learning Algorithm Engineer 2018-2019
  • Deepir Inc

    Beijing, China

    Deep Learning Researcher 2017-2018

Education Background


  • Hong Kong University of Science and Technology

    Hong Kong, China

    PhD candidate of Academy Of Interdisciplinary Studies 2022-current
    Advisor: Prof. Mingming Fan Co-advisor: Prof. Huamin Qu
  • Rochester Institute of Technology (RIT)

    Rochester, NY, USA

    PhD candidate of Computational and Information Science 2020-2021
    Advisor: Prof. Mingming Fan
  • Beijing University of Posts and Telecommunications(BUPT)

    Beijing, China

    Master of Computer Science and Technology 2017-2020
  • Beijing University of Posts and Telecommunications(BUPT)

    Beijing, China

    Bachelor of Artifical Intelligence and Technology 2013-2017