Analyzing Deaf and Hard-of-Hearing Users' Behavior, Usage, and Interaction with a Personal Assistant Device that Understands Sign-Language Input

Abraham Glasser, Matthew Watkins, Kira Hart, Sooyeon Lee, Matt Huenerfauth

Research output: Chapter in Book/Report/Conference proceedingConference contribution

10 Scopus citations

Abstract

As voice-based personal assistant technologies proliferate, e.g., smart speakers in homes, and more generally as voice-control of technology becomes increasingly ubiquitous, new accessibility barriers are emerging for many Deaf and Hard of Hearing (DHH) users. Progress in sign-language recognition may enable devices to respond to sign-language commands and potentially mitigate these barriers, but research is needed to understand how DHH users would interact with these devices and what commands they would issue. In this work, we directly engage with the DHH community, using a Wizard-of-Oz prototype that appears to understand American Sign Language (ASL) commands. Our analysis of video recordings of DHH participants revealed how they woke-up the device to initiate commands, structured commands in ASL, and responded to device errors, providing guidance to future designers and researchers. We share our dataset of over 1400 commands, which may be of interest to sign-language-recognition researchers.

Original languageEnglish (US)
Title of host publicationCHI 2022 - Proceedings of the 2022 CHI Conference on Human Factors in Computing Systems
PublisherAssociation for Computing Machinery
ISBN (Electronic)9781450391573
DOIs
StatePublished - Apr 29 2022
Externally publishedYes
Event2022 CHI Conference on Human Factors in Computing Systems, CHI 2022 - Virtual, Online, United States
Duration: Apr 30 2022May 5 2022

Publication series

NameConference on Human Factors in Computing Systems - Proceedings

Conference

Conference2022 CHI Conference on Human Factors in Computing Systems, CHI 2022
Country/TerritoryUnited States
CityVirtual, Online
Period4/30/225/5/22

All Science Journal Classification (ASJC) codes

  • Software
  • Human-Computer Interaction
  • Computer Graphics and Computer-Aided Design

Keywords

  • Accessibility
  • Deaf and Hard of Hearing
  • Personal Assistants
  • Sign Language

Fingerprint

Dive into the research topics of 'Analyzing Deaf and Hard-of-Hearing Users' Behavior, Usage, and Interaction with a Personal Assistant Device that Understands Sign-Language Input'. Together they form a unique fingerprint.

Cite this