Contact Information
Matthew Keith Xi-Jie Pan
Assistant Professor
Department of Electrical and Computer Engineering
Queen’s University
| Mailing Address: | Physical Address: |
| Walter Light Hall, Room 416 Queen’s University Kingston ON K7L 3N9 CANADA | Walter Light Hall, Rm. 702 19 Union St. Kingston, ON K7L 3N9 CANADA |
Research Summary
I build interactive experiences using robots that detect, interpret, and react to human behaviours. My projects focus on providing social and physical assistance or augmentation to people during human-robot interactions in both real and virtual environments. Examples of my work towards this aim include animatronic mutual gaze behavior, human-robot handover of objects, and a robotic system for introducing dynamic object interactions in virtual reality. My research draws from the areas of robotics,virtual and augmented reality, machine learning, computer vision, human psychomotor behaviour, and cognitive science.
Key Words: Human-robot interaction, collaborative manipulation, virtual/augmented/mixed reality, nonverbal behaviour, intent recognition,haptics
Education
Ph.D.
Mechanical Engineering
The University of British Columbia, Vancouver, British Columbia, Canada
2018
Advisor: Elizabeth A. Croft
Thesis: Towards Enabling Human-Robot Handovers:Exploring Nonverbal Cues for Fluent Human-Robot Handovers
M.A.Sc.
Mechanical Engineering
The University of British Columbia, Vancouver, British Columbia, Canada
2012
Advisors: Elizabeth A. Croft, Karon E. MacLean
Thesis: An Exploration of a Haptic Affect Loop Through Use Cases
B.A.Sc. (Honors) With Distinction
Mechatronics Engineering with Biomechanics Option
The University of Waterloo, Waterloo, Ontario Canada
2009
Thesis: Design of a 3D Image-Guided Tele-Operated Laparoscopic Instrument
Professional Appointments
Assistant Professor
Queen’s University, Department of Electrical and Computer Engineering
2021 – Current
Human-Robot Interaction Senior Engineer
UBTECH North American Research and Development Center
2021
Postdoctoral Associate
Disney Research Los Angeles
2018 – 2020
Advisors: Günter Niemeyer, Douglas Fidaleo
Lab Associate
Disney Research Los Angeles
2016 – 2017
Advisors: Günter Niemeyer, Lanny Smoot
Awards & Honors
Patents & Publications
Patents
Human Abnormal Behavior Response Method and Mobility Aid Robot Using the Same.
2021
Pan, M.K.X.J., Shao, D. Xiu, Z., Tan, H.
US Patent Pending
Eye contact sensing and control for robotic characters.
2020
Christensen, D., Wieland, A., Smoot, L., Ayala, A., Smithwick, Q., McIntosh, K., Kennedy, J., Pan, M.K.X.J., Niemeyer, G., Zamora, D.C., Choi S.
U.S. Patent US11072075
System for introducing physical experiences into virtual reality (VR) worlds.
2020
Niemeyer, G., Smoot, L., Chawda, V., Pan, M.K.X.J., Knoop, E., Bächer, M.
U.S. Patent US10362299B1
System for introducing physical experiences into virtual reality (VR) worlds.
2019
Niemeyer, G., Smoot, L., Chawda, V., Pan, M.K.X.J.
U.S. Patent US10362299
Hypercatching in virtual reality (VR) system.
2018
Niemeyer, G., Pan, M.K.X.J.
U.S. Patent US10139899
Refereed Journal Articles
An aff ordance and distance minimization based method for computing object
orientations for robot-human handovers.
2018
Chan, W.P., Pan, M.K.X.J., Croft, E.A., Inaba, M.
International Journal of Social Robotics (SORO)
Automated detection of handovers using kinematic features.
2017
Pan, M.K.X.J., Skjervøy, V., Chan, W.P., Inaba, M., Croft, E.A.
International Journal of Robotics Research (IJRR)
Exploring the Role of Haptic Feedback in an Implicit HCI-Based Bookmarking Application.
2014
Pan, M.K.X.J., McGrenere, J., Croft, E. A., MacLean, K.E.
IEEE Transactions on Haptics
Refereed Conference Papers
Realistic and Interactive Robot Gaze
2020
Pan, M.K.X.J., Choi, S., Kennedy, J., McIntosh, K., Campos Zamora, D., Niemeyer, G., Kim, J., Wieland, A., Christensen, D.
IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS’20)
Nonparametric Motion Retargeting for Humanoid Robots on Shared Latent Space
2020
Choi, S., Pan, M.K.X.J., Kim, J.
Robotics: Science and Systems (RSS’20)
Fast Handovers with a Robot Character: Small Sensorimotor Delays Improve Perceived Qualities
2019
Pan, M.K.X.J., Knoop, E., Bächer, M., Niemeyer, G.
IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS’19)
A Multimodal System using Augmented Reality, Gestures, and Tactile Feedback for Robot Trajectory Programming and Execution.
2018
Chan, W.P., Quintero, C.P., Pan, M.K.X.J., Sakr, M., Van der Loos, H.F.M., Croft, E.A.
IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS’18)
Exploration of Geometry and Forces Occurring within Human-to-Robot Handovers.
2018
Pan, M.K.X.J., Croft, E.A., Niemeyer, G.
IEEE Haptics Symposium (HAPTICS’18),
Evaluating Social Perception of Human-to-Robot Handovers Using the Robot Social Attributes Scale (RoSAS)
2018
Pan, M.K.X.J., Croft, E.A., Niemeyer, G.
ACM/IEEE International Conference on Human-Robot Interaction (HRI’18)
Catching a Real Ball in Virtual Reality
2017
Pan, M.K.X.J., Niemeyer, G.
IEEE Conference on Virtual Reality (IEEE VR 2017)
Characterization of Handover Orientations Used by Humans for Efficient Robot to Human Handovers.
2015
Chan, W.P., Pan, M.K.X.J., Croft, E.A., Inaba, M.
IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS’15)
Predictions of Human Task Performance and Handover Trajectories for Human-Robot Interaction.
2015
Hart, J.W., Sheikholeslami, S., Pan, M.K.X.J., Chan, W.P., Croft, E.A.
ACM/IEEE International Conference on Human-Robot Interaction (HRI’15)
Human Behavioural Responses to Robot Head Gaze during Human-Robot Handovers.
2014
Zheng, M., Moon, A., Gleeson, B., Troniak, D., Pan, M.K.X.J., Blumer, B.A., Croft, E.A., Meng, M.Q.H.
IEEE International Conference on Robotics and Biomimetics (ROBIO’14)
Meet Me Where I’m Gazing: How Shared Attention Gaze Affects Human-Robot Handover Timing.
2014
Moon, A., Troniak, D.M., Gleeson, B., Pan, M.K.X.J., Zheng, M., Blumer, B.A., MacLean, K.E., Croft, E.A.
ACM/IEEE International Conference on Human-Robot Interaction (HRI’14)
Emotional Communication and Implicit Control through Touch.
2012
MacLean, K.E., Yohanan, S., Sefidgar, Y., Pan, M.K.X.J., Croft, E.A., McGrenere, J.
IEEE Haptics Symposium (HAPTICS’12)
Now, Where Was I? Physiologically Triggered Bookmarks for Audio Books.
2011
Pan, M.K.X.J., Chang, J.S., Himmetoglu, G.H., Moon, A., Hazelton, T.W., MacLean, K.E., and Croft, E.A.
ACM Conference on Human Factors in Computing Systems (CHI’11)
Galvanic skin response-derived bookmarking of an audio stream.
2011
Pan, M.K.X.J., Chang, J.S., Himmetoglu, G.H., Moon, A., Hazelton, T.W., MacLean, K.E., Croft, E.A.
ACM Conference on Human Factors in Computing Systems (CHI’11)
Expressive Wearable Haptic Devices
2010
Pan, M.K.X.J., Baumann, M.A., Hazelton, T.W., MacLean, K.E., Croft, E.A.
IEEE Haptics Symposium (HAPTICS’10)
Referred Workshop Papers
A Multimodal System using Augmented Reality, Gestures, and Tactile Feedback for Robot Trajectory Programming and Execution
2018
Chan, W.P., Quintero, C., Pan, M.K.X.J., Sakr, M., Van der Loos, M., Croft, E.A.
IEEE International Conference on Robotics and Automation (ICRA’18)
Workshop on Robotics in Virtual Reality
Validation of the Robot Social Attributes Scale (RoSAS) for Human-Robot Interaction through a Human-to-Robot Handover Use Case
2017
Pan, M.K.X.J., Croft, E.A., Niemeyer, G.
IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS’17)
Human-Robot Interaction in Collaborative Manufacturing Environments
Gesture, Gaze, Touch, and Hesitation: Timing Cues for Collaborative Work.
Hart, J., Gleeson, B., Pan, M.K.X.J., Moon, A., MacLean, K.E., Croft, E.A.
ACM/IEEE International Conference on Human-Robot Interaction (HRI’14)
Timing Workshop
Presenting a Biometrically-Driven Haptic Interaction Loop.
Karuei, I., Hazelton, T. W., MacLean, K. E., Baumann, M., Pan, M.K.X.J.
ACM Conference on Human Factors in Computing Systems (CHI’10)
Workshop on Whole Body Interaction
Other Contributions
Robotic Laparoscopy: Design of an Image-Guided Tool for Teleoperated Minimally Invasive Surgery
2009
Pan, M.K.X.J., Yip, M., Pardasani, U.
Technical Report for Robarts Imaging and Canadian Surgical Technologies and Advanced Robotics at the London Health Sciences Centre
MotionStation3D – Development and Deployment of a Tool for 3D Image-Guided Installation of Orthopedic Implants.
2008
Pan, M.K.X.J.
Technical Report for The University of Western Ontario
Talks
Invited Talks
| Ingenuity Labs Research Institute and IEEE Kingston Invited Talk | 2021 |
| Queen’s University, Electrical and Computer Engineering Seminar | 2021 |
| UBTECH North American Research and Development Center | 2021 |
| IROS Workshop on Shared Autonomy | 2017 |
| Walt Disney Technology Symposium | 2017 |
| Disney Research Los Angeles, Tech Talk | 2016 |
| Universitié Laval, General Motors Plugfest | 2012, 2013 |
Campus Talks
| Ingenuity Labs Seminar | 2021 |
| Walt Disney Imagineering, Software Soiree | 2019 |
| University of British Columbia, Dept. Seminar Series | 2013, 2014 |
Teaching
Lecturer
| MREN 178 | Data Structures and Algorithms Queen’s University | 2022 |
Teaching Assistantships
| ME/CS/EE 134 | Robotic Systems California Institute of Technology | 2020 |
| MECH 223 | Mechanical Design University of British Columbia | 2011-2015 |
| MECH 221 | Electrical Circuits University of British Columbia | 2010-2014 |
| MECH 224 | Community Service Learning University of British Columbia | 2010-2014 |
Curriculum Development
| APSC 100 | Introduction to Engineering University of British Columbia | 2015 |
Certification
Selected Service & Outreach
Conference and Journal Refereeing
| ACM Transactions on Human-Robot Interaction | 2021 |
| IEEE International Conference on Robotics and Automation (ICRA) | 2020 |
| IEEE-RAS International Conference on Humanoid Robots (Humanoids) | 2020 |
| IEEE Robotics and Automation Letters (RA-L) | 2020 |
| ACM/IEEE International Conference on Human-Robot Interaction (HRI) | 2020 |
| Royal Society Open Science | 2019 |
| Conference on Graphics, Visualization & HCI (GI) | 2019 |
| ACM User Interface Software and Technology Symposium (UIST) | 2018 |
| IEEE Symposium on Robot and Human Interactive Communication (RO-MAN) | 2016 |
| IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) | 2016 |
| International Journal of Robotics Research (IJRR) | 2016 |
| International Journal of Soft Robotics (SORO) | 2013 |
| IEEE Haptics Symposium (HAPTICS) | 2012 |
| ACM Conference on Human Factors in Computing Systems (CHI) | 2012 |
Organizing Committee
| Session Chair IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) | 2020 |
| Local Arrangements IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) | 2017 |
Other Service
| Judge Greater Vancouver Regional Science Fair | 2012-2015 |
| Executive Mechanical Engineering Graduate Association The University of British Columbia | 2009-2012 |
| Consultant/Judge International Autonomous Robot Racing Challenge The University of British Columbia | 2011-2012 |
| Senior Care Volunteer Yaletown House Society, Vancouver | 2010 |
| Executive IEEE Engineering in Medicine and Biology Society, Kitchener-Waterloo Chapter | 2006 |