MRCoach: A Real-Time IoT-Enabled Mixed Reality System With Semantic-Aware Transmission for Smart Sports and Personalized Coaching

  • Mingrui Yin
  • , Sohom Sen
  • , Yongjie Guan
  • , Dhananjay Jagdish Dubey
  • , Xinran Chen
  • , Xueyu Hou
  • , Tao Han
  • , Nirwan Ansari

Research output: Contribution to journalArticlepeer-review

Abstract

Real-time transmission of large-scale data, high computational demands, and resource limitations on edge devices poses significant challenges for intelligent sports systems. The proliferation of the Internet of Things (IoT) technologies has catalyzed the rise of smart sport, where wearable sensors, cameras, and intelligent algorithms are integrated to revolutionize athletic training. Despite this transformation, access to professional coaching remains constrained by factors such as time, cost, and scalability. A critical limitation of existing remote coaching approaches is their inability to perform effective spatiotemporal analysis, hindering comprehensive evaluation and refinement of athletic performance. This article introduces mixed reality coaching (MRCoach), a mixed reality-based, immersive, and interactive sports coaching system that enables data-driven training without requiring in-person supervision. MRCoach reconstructs 3-D volumetric avatars of both learners and expert athletes, allowing users to visualize and compare their movements side-by-side in a mixed reality environment for intuitive skill refinement. To ensure responsive and efficient feedback, we propose an adaptive semantic transmission strategy that prioritizes sport-relevant joints, thereby reducing latency and bandwidth requirements without sacrificing accuracy. Furthermore, a 3-D sports analysis framework is developed to evaluate motion based on normalized joint positions, velocities, and accelerations. This framework computes real-time similarity scores and delivers actionable guidance to learners. Experimental results across four sports - tennis, soccer, basketball, and baseball - demonstrate MRCoach's effectiveness in providing personalized, real-time training experiences. Compared with state-of-the-art baselines such as MagicStream, ExPose, and PIXIE, MRCoach achieves significantly lower end-to-end (E2E) latency (79.9 ms) and higher frame rates (≥ 56 frames/s) while maintaining accurate pose tracking and high avatar fidelity.

Original languageEnglish (US)
Pages (from-to)52466-52473
Number of pages8
JournalIEEE Internet of Things Journal
Volume12
Issue number24
DOIs
StatePublished - 2025

All Science Journal Classification (ASJC) codes

  • Signal Processing
  • Information Systems
  • Hardware and Architecture
  • Computer Science Applications
  • Computer Networks and Communications

Keywords

  • 3-D human model
  • 3-D sports analysis
  • mixed reality
  • sports coaching system

Fingerprint

Dive into the research topics of 'MRCoach: A Real-Time IoT-Enabled Mixed Reality System With Semantic-Aware Transmission for Smart Sports and Personalized Coaching'. Together they form a unique fingerprint.

Cite this