Isaac Ngui Robotics · Human-Robot Interaction · Extended Reality

About Me

I am a Computer Science Ph.D. student in the Parasol Lab at the University of Illinois Urbana-Champaign, advised by Prof. Nancy M. Amato. I build extended reality (XR) systems that let people specify and teach robot behavior by editing the robot's environment rather than the robot itself, and I develop the planning and learning methods behind them, from preference learning to scalable multi-robot motion planning. I received my B.S. in Computer Science, with a minor in Mathematics, from the University of Wisconsin–Milwaukee.

My work sits where human-robot interaction, motion planning, and robot learning meet. Recently I presented CIPHER at WAFR 2026 and ERUPT at ICAT-EGVE 2026, and I am running a pre-registered user study comparing immersive and desktop interfaces for robot motion planning.

Research Interests

XR for Human-Robot Interaction

Immersive interfaces for programming, previewing, and supervising robots. Users reshape the robot's workspace and the planner responds in real time.

Learning from Human Feedback

Learning robot preferences from demonstrations and corrections, including environment edits that disambiguate what a person actually cares about.

Multi-Robot Motion Planning

Scalable planners that coordinate robot teams through workspace-level guidance, for geometric and kinodynamic robots.

Tools & Skills

Research

ICAT-EGVE 2026

ERUPT: Extended Reality Universal Planning Toolkit

  • Unity
  • ROS 2
  • MoveIt
  • Meta Quest 3

An open-source XR toolkit for interactive motion planning. Users create, scale, and move obstacles with their hands. Edits sync to the MoveIt planning scene in about 40 ms, and planned trajectories can be previewed in VR/AR before running on a physical UR5e. A pre-registered user study comparing ERUPT with the 2D MoveIt/RViz interface is underway.

Code
WAFR 2026

CIPHER: Scalable Multi-Robot Motion Planning

  • C++
  • OMPL
  • MAPF
  • Kinodynamic planning

A hierarchical multi-robot planner that coordinates robots through a workspace decomposition and refines it wherever conflicts arise, so it rarely has to plan in the expensive joint space of all robots. It solved 100% of benchmark problems with up to 32 robots and was up to an order of magnitude faster than state-of-the-art baselines.

Project Page Code
CoRL 2026 Workshop

FERRET: Environment Edits as a Corrective Signal

  • PyTorch
  • Graph neural networks
  • Reward learning

When a person drags a robot arm away from a laptop, the robot can't tell which of many changes they meant. FERRET lets people edit the scene instead while the robot's motion stays fixed. It learns preferences from both scene and robot edits using relational and graph neural network models, and recovers features that robot corrections alone cannot.

IEEE VR 2025 Workshops

RADER: Robot Action Demonstration in XR

  • Unity
  • ROS 2
  • Learning from demonstration

An XR interface for teaching robots by demonstration with a virtual arm. Users can safely show the high-cost states that are dangerous to demonstrate on real hardware. Features learned from XR demonstrations matched those learned from demonstrations on a physical UR5e.

Paper Code

Publications

* denotes equal contribution

  1. Scalable Multi-robot Motion Planning via Hierarchical Subproblem Expansion and Workspace Decomposition Refinement
    I. Ngui*, C. McBeth*, J. D. Motes, M. Morales, N. M. Amato
    Workshop on the Algorithmic Foundations of Robotics (WAFR), 2026. [project] [code]
  2. ERUPT: An Open Toolkit for Interfacing with Robot Motion Planners in Extended Reality
    I. Ngui, C. McBeth, A. Santos, G. He, K. Mimnaugh, J. D. Motes, L. Soares, M. Morales, N. M. Amato
    ICAT-EGVE, 2026. [code]
  3. Environment Edits as a Corrective Signal for Reward Learning
    I. Ngui*, C. McBeth*, N. M. Amato
    CoRL Workshop, 2026.
  4. Scalable Multi-robot Motion Planning Using Workspace Guidance-Informed Hypergraphs
    C. McBeth, J. D. Motes, I. Ngui, M. Morales, N. M. Amato
    IEEE Robotics and Automation Letters (RA-L), 11(4), 2026. [paper]
  5. Extended Reality System for Robotic Learning from Human Demonstration
    I. Ngui, C. McBeth, G. He, A. C. Santos, L. Soares, M. Morales, N. M. Amato
    IEEE Conference on Virtual Reality and 3D User Interfaces Abstracts and Workshops (VRW), 2025. [paper] [arXiv] [code]