Aerial 360-Degree Video Delivery for Immersive First Person View UAV Navigation

Simran Singh, Jacob Chakareski

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Abstract

Adaptive transmission of conventional video from a UAV to the ground has been researched for various applications, but the research topic of 360° video transmission from a UAV for the specific application of first-person view (FPV) based navigation is still nascent. In this work, we present adaptive 360° video compression and streaming methods to optimize the perceptual quality of experience of a pilot, who navigates the UAV in real time by viewing this immersive FPV feed, which is sent wirelessly from the UAV to the pilot. This adaptation of the 360° FPV feed is performed in response to the wireless channel conditions and the pilot's viewport, wherein each 360° frame is split into two regions of variable size, one meant to be within the pilot's viewport and the other outside. Each region is encoded using different H. 265 quantization parameters (QP) and modulation orders. We model the scenario realistically by generating probability distributions of the variation in frame size and quality with QP, for aerial 360° videos. These models are expressed using a two-term exponential function, whose parameters are also provided. This model achieves lower prediction errors than the single-term exponential and power law functions. Simulations on a set of aerial 360-degree videos demonstrate that the adaptive approach achieves 9.73 dB (21.77 %) greater QoE than a baseline approach that utilizes throughput-based adaptive bit rate algorithm (ABR) to tune QP per GoP, and a 5G new radio adaptive modulation scheme (AMS) to tune modulation order: Additionally, we present a deep reinforcement learning approach to adapt FPV, which achieves an expected pilot QoE just 2.07 dB lower than the adaptive approach, while being significantly faster and requiring no prior knowledge of the environment.

Original languageEnglish (US)
Title of host publicationProceedings - 2023 IEEE International Symposium on Multimedia, ISM 2023
PublisherInstitute of Electrical and Electronics Engineers Inc.
Pages139-146
Number of pages8
ISBN (Electronic)9798350395761
DOIs
StatePublished - 2023
Event2023 IEEE International Symposium on Multimedia, ISM 2023 - Laguna Hills, United States
Duration: Dec 11 2023Dec 13 2023

Publication series

NameProceedings - 2023 IEEE International Symposium on Multimedia, ISM 2023

Conference

Conference2023 IEEE International Symposium on Multimedia, ISM 2023
Country/TerritoryUnited States
CityLaguna Hills
Period12/11/2312/13/23

All Science Journal Classification (ASJC) codes

  • Signal Processing
  • Media Technology
  • Modeling and Simulation

Keywords

  • 360-degree video
  • first-person view
  • quality of experience
  • UAVs
  • wireless communications

Fingerprint

Dive into the research topics of 'Aerial 360-Degree Video Delivery for Immersive First Person View UAV Navigation'. Together they form a unique fingerprint.

Cite this