Multi-Connectivity and Edge Computing for Ultra-Low-Latency Lifelike Virtual Reality

Jacob Chakareski, Sabyasachi Gupta

Research output: Chapter in Book/Report/Conference proceedingConference contribution

14 Scopus citations

Abstract

We explore a novel multi-user mobile VR system for streaming scalable 8K360^° video at high reliability and immersion fidelity, and low interactive latency, via a synergistic integration of scalable 360^° tiling, dual-band millimeter wave (mmWave) and Wi-Fi transmission, and edge computing. High rate directed mmWave links are studied to send VR viewport-specific high-quality enhancement layers of the 360^° content to the individual users, while Wi-Fi broadcast of the base layer of the entire 360^° panorama is sent to all users, to augment the system's reliability. The viewport-specific enhancement layers can comprise compressed and raw 360^° tiles, decoded first at the edge server. We explore the joint optimization of the mmWave access point to user association, the choice of 360^° tiles to be transmitted decompressed, the allocation of mmWave data rate across the compressed tiles in a viewport-specific enhancement layer, and the allocation of computing resources at the edge server and user devices. Our objective is to maximize the minimum delivered VR immersion fidelity across all users, given transmission, latency, and computing constraints. We demonstrate that our framework can enable a significant improvement in immersion fidelity (8dB to 10 dB) and spatial resolution (8Kvs. 4K), over MPEG-DASH that uses Wi-Fi transmission only. We also show that an increasing number of raw 360^° tiles are sent, as the mmWave link rate or the edge server/user computing power increase, exploring rigorously here the fundamental interplay between computing and communication capabilities, end-to-end system latency, and delivered VR immersion fidelity.

Original languageEnglish (US)
Title of host publication2020 IEEE International Conference on Multimedia and Expo, ICME 2020
PublisherIEEE Computer Society
ISBN (Electronic)9781728113319
DOIs
StatePublished - Jul 2020
Event2020 IEEE International Conference on Multimedia and Expo, ICME 2020 - London, United Kingdom
Duration: Jul 6 2020Jul 10 2020

Publication series

NameProceedings - IEEE International Conference on Multimedia and Expo
Volume2020-July
ISSN (Print)1945-7871
ISSN (Electronic)1945-788X

Conference

Conference2020 IEEE International Conference on Multimedia and Expo, ICME 2020
Country/TerritoryUnited Kingdom
CityLondon
Period7/6/207/10/20

All Science Journal Classification (ASJC) codes

  • Computer Networks and Communications
  • Computer Science Applications

Keywords

  • DualbandWiFi/millimeter wave wireless streaming
  • Edge computing
  • Geometric programming
  • Mobile virtual reality
  • Mobile virtual reality, scalable 360°
  • Rate-computing-delay interplay
  • Resource allocation
  • Scalable 360° video tiling

Fingerprint

Dive into the research topics of 'Multi-Connectivity and Edge Computing for Ultra-Low-Latency Lifelike Virtual Reality'. Together they form a unique fingerprint.

Cite this