Optical touch sensing on nonparametric rear-projection surfaces for interactive physical-virtual experiences

Jason Hochreiter, Salam Daher, Arjun Nagendran, Laura Gonzalez, Greg Welch

Research output: Contribution to journalArticlepeer-review

11 Scopus citations


We demonstrate a generalizable method for unified multitouch detection and response on various nonparametric and parametric surfaces to support interactive physical-virtual experiences. The method employs multiple infrared (IR) cameras, one or more projectors, IR light sources, and a rear-projection surface. IR light reflected off human fingers is captured by cameras with matched IR pass filters, allowing for the detection and localization of multiple simultaneous finger-touch events. The processing of these events is tightly coupled with the rendering system to produce auditory and visual responses displayed on the surface using the projector(s) to achieve a responsive, interactive, physical-virtual experience. We demonstrate the method on two nonparametric face-shaped surfaces and a planar surface. We also illustrate the approach’s applicability in an interactive medical training scenario using one of the head surfaces to support hands-on, touch-sensitive medical training with dynamic physical-virtual patient behavior.

Original languageEnglish (US)
Pages (from-to)33-46
Number of pages14
JournalPresence: Teleoperators and Virtual Environments
Issue number1
StatePublished - Jul 1 2016
Externally publishedYes

All Science Journal Classification (ASJC) codes

  • Software
  • Control and Systems Engineering
  • Human-Computer Interaction
  • Computer Vision and Pattern Recognition


Dive into the research topics of 'Optical touch sensing on nonparametric rear-projection surfaces for interactive physical-virtual experiences'. Together they form a unique fingerprint.

Cite this