AIGuide: Augmented Reality Hand Guidance in a Visual Prosthetic

Sooyeon Lee, Nelson Daniel Troncoso Aldas, Chonghan Lee, Mary Beth Rosson, John M. Carroll, Vijaykrishnan Narayanan

Research output: Contribution to journalArticlepeer-review

5 Scopus citations


Locating and grasping objects is a critical task in people's daily lives. For people with visual impairments, this task can be a daily struggle. The support of augmented reality frameworks in smartphones can overcome the limitations of current object detection applications designed for people with visual impairments. We present AIGuide, a self-contained smartphone application that leverages augmented reality technology to help users locate and pick up objects around them. We conducted a user study to investigate the effectiveness of AIGuide in a visual prosthetic for providing guidance; compare it to other assistive technology form factors; investigate the use of multimodal feedback, and provide feedback about the overall experience. We gathered performance data and participants' reactions and analyzed videos to understand users' interactions with the nonvisual smartphone user interface. Our results show that AIGuide is a promising technology to help people with visual impairments locate and acquire objects in their daily routine. The benefits of AIGuide may be enhanced with appropriate interaction design.

Original languageEnglish (US)
Article number12
JournalACM Transactions on Accessible Computing
Issue number2
StatePublished - Jun 2022
Externally publishedYes

All Science Journal Classification (ASJC) codes

  • Human-Computer Interaction
  • Computer Science Applications


  • Mobile assistive technology
  • augmented reality
  • nonvisual guidance interaction
  • people with visual impairments


Dive into the research topics of 'AIGuide: Augmented Reality Hand Guidance in a Visual Prosthetic'. Together they form a unique fingerprint.

Cite this