Multiple description coding and recovery of free viewpoint video for wireless multi-path streaming

Zhi Liu, Gene Cheung, Jacob Chakareski, Yusheng Ji

Research output: Contribution to journalArticlepeer-review

48 Scopus citations


By transmitting texture and depth videos captured from two nearby camera viewpoints, a client can synthesize via depth-image-based rendering (DIBR) any freely chosen intermediate virtual view of the 3D scene, enhancing the user's perception of depth. During wireless network transmission, burst packet losses can corrupt the transmitted texture and depth videos and degrade the synthesized view quality at the client. In this paper, we propose a multiple description coding system for multi-path transmission of free-viewpoint video, with joint interview and temporal description recovery capability. In particular, we encode separately the even frames of the left view and the odd frames of the right view, and transmit them as one description on one path. The second description comprises the remaining frames in the two views and is transmitted over a second path. If the receiver receives only one description due to burst loss in the other path, the missing frames in the other description are partially reconstructed using our frame recovery procedure. First, we construct two recovery candidates for each lost pixel in a frame. The first candidate is generated via temporal super-resolution from its predecessor and successor frames in the same view. The second candidate is generated via DIBR fromthe received frame of the same time instance in the other view. Next, we select the best pixel candidates one patch at a time, where an image patch corresponds to a neighborhood of pixels with similar depth values in the 3D scene. Near-optimal source and channel coding rates for each description are selected using a branch-and-bound method, for given transmission bandwidth on each path. Experimental results show that our systemcan outperform a traditional single-description/single-path transmission scheme by up to 5.5 dB in Peak Signal-to-Noise Ratio (PSNR) of the synthesized intermediate view at the client.

Original languageEnglish (US)
Article number6832458
Pages (from-to)151-164
Number of pages14
JournalIEEE Journal on Selected Topics in Signal Processing
Issue number1
StatePublished - Feb 1 2015
Externally publishedYes

All Science Journal Classification (ASJC) codes

  • Signal Processing
  • Electrical and Electronic Engineering


  • Depth-image-based rendering
  • Free-viewpoints video
  • Multiple description coding
  • Wireless multi-path streaming


Dive into the research topics of 'Multiple description coding and recovery of free viewpoint video for wireless multi-path streaming'. Together they form a unique fingerprint.

Cite this