VocalSketch: Vocally imitating audio concepts

Mark Cartwright, Bryan Pardo

Research output: Chapter in Book/Report/Conference proceedingConference contribution

31 Scopus citations


A natural way of communicating an audio concept is to imitate it with one's voice. This creates an approximation of the imagined sound (e.g. a particular owl's hoot), much like how a visual sketch approximates a visual concept (e.g a drawing of the owl). If a machine could understand vocal imitations, users could communicate with software in this natural way, enabling new interactions (e.g. programming a music synthesizer by imitating the desired sound with one's voice). In this work, we collect thousands of crowd-sourced vocal imitations of a large set of diverse sounds, along with data on the crowd's ability to correctly label these vocal imitations. The resulting data set will help the research community understand which audio concepts can be effectively communicated with this approach. We have released the data set so the community can study the related issues and build systems that leverage vocal imitation as an interaction modality.

Original languageEnglish (US)
Title of host publicationCHI 2015 - Proceedings of the 33rd Annual CHI Conference on Human Factors in Computing Systems
Subtitle of host publicationCrossings
PublisherAssociation for Computing Machinery
Number of pages4
ISBN (Electronic)9781450331456
StatePublished - Apr 18 2015
Externally publishedYes
Event33rd Annual CHI Conference on Human Factors in Computing Systems, CHI 2015 - Seoul, Korea, Republic of
Duration: Apr 18 2015Apr 23 2015

Publication series

NameConference on Human Factors in Computing Systems - Proceedings


Other33rd Annual CHI Conference on Human Factors in Computing Systems, CHI 2015
Country/TerritoryKorea, Republic of

All Science Journal Classification (ASJC) codes

  • Software
  • Human-Computer Interaction
  • Computer Graphics and Computer-Aided Design


  • Audio software
  • Data set
  • User interaction
  • Vocal imitation


Dive into the research topics of 'VocalSketch: Vocally imitating audio concepts'. Together they form a unique fingerprint.

Cite this