Helping Helpers: Supporting Volunteers in Remote Sighted Assistance with Augmented Reality Maps

Jingyi Xie, Rui Yu, Sooyeon Lee, Yao Lyu, Syed Masum Billah, John M. Carroll

Research output: Chapter in Book/Report/Conference proceedingConference contribution

2 Scopus citations

Abstract

Remote sighted assistance (RSA) has emerged as a conversational assistive service, where remote sighted workers, i.e., agents, provide real-time assistance to blind users via video-chat-like communication. Prior work identified several challenges for the agents to provide navigational assistance to users and proposed computer vision-mediated RSA service to address those challenges. We present an interactive system implementing a high-fidelity prototype of RSA service using augmented reality (AR) maps with localization and virtual elements placement capabilities. The paper also presents a confederate-based study design to evaluate the effects of AR maps with 13 untrained agents. The study revealed that, compared to baseline RSA, agents were significantly faster in providing indoor navigational assistance to a confederate playing the role of users, and agents' mental workload was significantly reduced - all indicate the feasibility and scalability of AR maps in RSA services.

Original languageEnglish (US)
Title of host publicationDIS 2022 - Proceedings of the 2022 ACM Designing Interactive Systems Conference
Subtitle of host publicationDigital Wellbeing
PublisherAssociation for Computing Machinery, Inc
Pages881-897
Number of pages17
ISBN (Electronic)9781450393584
DOIs
StatePublished - Jun 13 2022
Externally publishedYes
Event2022 ACM Designing Interactive Systems Conference: Digital Wellbeing, DIS 2022 - Virtual, Online, Australia
Duration: Jun 13 2022Jun 17 2022

Publication series

NameDIS 2022 - Proceedings of the 2022 ACM Designing Interactive Systems Conference: Digital Wellbeing

Conference

Conference2022 ACM Designing Interactive Systems Conference: Digital Wellbeing, DIS 2022
Country/TerritoryAustralia
CityVirtual, Online
Period6/13/226/17/22

All Science Journal Classification (ASJC) codes

  • Computer Graphics and Computer-Aided Design
  • Human-Computer Interaction
  • Software

Keywords

  • 3D map
  • People with visual impairments
  • augmented reality
  • blind
  • camera
  • computer vision
  • navigation
  • remote sighted assistance
  • smartphone

Fingerprint

Dive into the research topics of 'Helping Helpers: Supporting Volunteers in Remote Sighted Assistance with Augmented Reality Maps'. Together they form a unique fingerprint.

Cite this