Passive Vital Sign Monitoring via Facial Vibrations Extracted from AR/VR Vibration Sensing Based Testbed

The adoption of augmented reality/virtual reality (AR/VR) has dramatically risen over the past few years across various application sectors, including immersive gaming, social communication, education, and tourism. The emerging use of AR/VR headsets has also created an excellent opportunity to promote pervasive health monitoring service as most AR/VR devices are already equipped with enriched sensing paradigm and will interact with users for a long time. In this talk, we aim to explore innovative technologies that enable fine-grained and personalized health status monitoring (e.g., vital signs and user identities) leveraging facial vibrations captured by the in-built motion sensor testbed on commodity AR/VR headsets. On one hand, it provides real-time health information required in virtual healthcare applications. For instance, a doctor can continuously monitor a patient’s vital signs during the tele-medicine session at home, which helps the doctor to realize timely and precise diagnoses [2]. On the other hand, as people are spending increasing time in cyberspace (e.g., Metaverse), exposure to virtual and immersive contents requires high concentration on users’ mind. Such usage cases may significantly increase the visual and psychological burden and induce potential health issues (e.g., anxiety, hypertension, sleep disorders) [1, 3, 5].