Data-Driven Classification of Human Movements in Virtual Reality-Based Serious Games: Preclinical Rehabilitation Study in Citizen Science

Roni Barak Ventura, Kora Stewart Hughes, Oded Nov, Preeti Raghavan, Manuel Ruiz Marín, Maurizio Porfiri

Research output: Contribution to journalArticlepeer-review

4 Scopus citations

Abstract

Background: Sustained engagement is essential for the success of telerehabilitation programs. However, patients' lack of motivation and adherence could undermine these goals. To overcome this challenge, physical exercises have often been gamified. Building on the advantages of serious games, we propose a citizen science-based approach in which patients perform scientific tasks by using interactive interfaces and help advance scientific causes of their choice. This approach capitalizes on human intellect and benevolence while promoting learning. To further enhance engagement, we propose performing citizen science activities in immersive media, such as virtual reality (VR). Objective: This study aims to present a novel methodology to facilitate the remote identification and classification of human movements for the automatic assessment of motor performance in telerehabilitation. The data-driven approach is presented in the context of a citizen science software dedicated to bimanual training in VR. Specifically, users interact with the interface and make contributions to an environmental citizen science project while moving both arms in concert. Methods: In all, 9 healthy individuals interacted with the citizen science software by using a commercial VR gaming device. The software included a calibration phase to evaluate the users' range of motion along the 3 anatomical planes of motion and to adapt the sensitivity of the software's response to their movements. During calibration, the time series of the users' movements were recorded by the sensors embedded in the device. We performed principal component analysis to identify salient features of movements and then applied a bagged trees ensemble classifier to classify the movements. Results: The classification achieved high performance, reaching 99.9% accuracy. Among the movements, elbow flexion was the most accurately classified movement (99.2%), and horizontal shoulder abduction to the right side of the body was the most misclassified movement (98.8%). Conclusions: Coordinated bimanual movements in VR can be classified with high accuracy. Our findings lay the foundation for the development of motion analysis algorithms in VR-mediated telerehabilitation.

Original languageEnglish (US)
Article numbere27597
JournalJMIR Serious Games
Volume10
Issue number1
DOIs
StatePublished - Jan 1 2022
Externally publishedYes

All Science Journal Classification (ASJC) codes

  • Physical Therapy, Sports Therapy and Rehabilitation
  • Biomedical Engineering
  • Rehabilitation
  • Psychiatry and Mental health

Keywords

  • Motion analysis
  • Principal component analysis
  • Telerehabilitation
  • Virtual reality

Fingerprint

Dive into the research topics of 'Data-Driven Classification of Human Movements in Virtual Reality-Based Serious Games: Preclinical Rehabilitation Study in Citizen Science'. Together they form a unique fingerprint.

Cite this