ICE-T: RC: Millimeter Wave Communications and Edge Computing for Next Generation Tetherless Mobile Virtual Reality

Virtual and augmented reality (VR/AR) technologies hold tremendous potential to advance our society, having impact on quality of life, environmental and energy conservation, and the world economy. However, two main challenges stand in the way of realizing this vision. These applications are hyper-data-intensive and require ultra-low latency, neither of which can be met by current and upcoming conventional networking methods and systems. These presently limit VR/AR applications to an offline operation, synthetic content, high-end wired equipment, and gaming/entertainment settings. This project envision a novel system at the intersection of millimeter-wave communication (mmWave) and edge computing that aims to overcome these challenges to bring us closer to the next generation tetherless VR/AR societal applications. The project will make notable contributions to the emerging area of networked VR/AR application systems and communications, leading to advances in numerous socially relevant applications, e.g., search and rescue, and disaster response. It will also facilitate fundamental research in the general application area of high-volume high-speed/low-latency data transfer in emerging settings. Beyond the direct scientific and technology impacts and their broader effects on society, educational, outreach, international collaboration, and scientific leadership activities will be pursued as an integral part of the project.

Overcoming the broad performance gap between present and upcoming networked systems capabilities and anticipated requirements of next generation applications will require novel holistic approaches to capture, coding, networking, and reconstruction/navigation of VR/AR data. Towards this objective, the project will investigate a futuristic 5G heterogeneous cellular network system that integrates radio frequency (RF) and millimeter wave communication, and viewport-adaptive space-time scalable VR signal tiling, for multi-path streaming of 360-degree tetherless mobile VR applications. In this setting, the project will pursue the following synergistic investigations: (1) Navigation-aware scalable VR signal tiling to enable interactive streaming of only the data truly needed by the user during navigation; (2) Deep machine learning for user navigation prediction to assist the envisioned resource allocation methods. (3) Space-time scalable rateless code construction for effective source-channel VR signal representation to protect against prospective transmission errors. (4) Dynamic rate-distortion optimized strategies for hybrid RF-mmWave multi-path VR streaming and analysis of the foundations of the interdependencies between the VR signal tiling design and the characteristics of the two network paths. (5) Analysis of the fundamental trade-offs between edge computing and communication that arise here and pursuit of optimization methods that will leverage them to maximize the system efficiency. (6) Graph-theoretic analysis of the problem of dynamic mmWave transmitter to VR user assignment. (7) Network slicing for parallel operation with other applications. Extensive integration and experimentation will be carried out to assess, validate, and prototype the enabled research advances in practical settings.

This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.