ZFP-X: Efficient Embedded Coding for Accelerating Lossy Floating Point Compression

Bing Lu, Yida Li, Junqi Wang, Huizhang Luo, Kenli Li

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Abstract

Today's scientific simulations are confronting seriously limited I/O bandwidth, network bandwidth, and storage capacity because of immense volumes of data generated in high-performance computing systems. Data compression has emerged as one of the most effective approaches to resolve the issue of the exponential increase of scientific data. However, existing state-of-the-art compressors also are confronting the issue of low throughput, especially under the trend of growing disparities between the compute and I/O rates. Among them, embedded coding is widely applied, which contributes to the dominant running time for the corresponding compressors. In this work, we propose a new kind of embedded coding algorithm, and apply it as the backend embedded coding of ZFP, one of the most successful lossy compressors. Our embedded coding algorithm uses bit groups instead of bit planes to store the compressed data, avoiding the time overhead of generating bit planes and group tests of bit planes, which significantly reduces the running time of ZFP. Our embedded coding algorithm can also accelerate the decompression of ZFP, because the costly procedures of the reverse of group tests and reconstructing bit planes are also avoided. Moreover, we provide theoretical proof that the proposed coding algorithm has the same compression ratio as the baseline ZFP. Experiments with four representative real-world scientific simulation datasets show that the compression and decompression throughput of our solution is up to 2.5× (2.1× on average), and up to 2.1× (1.5× on average) as those of ZFP, respectively.

Original languageEnglish (US)
Title of host publicationProceedings - 2023 IEEE International Parallel and Distributed Processing Symposium, IPDPS 2023
PublisherInstitute of Electrical and Electronics Engineers Inc.
Pages1041-1050
Number of pages10
ISBN (Electronic)9798350337662
DOIs
StatePublished - 2023
Externally publishedYes
Event37th IEEE International Parallel and Distributed Processing Symposium, IPDPS 2023 - St. Petersburg, United States
Duration: May 15 2023May 19 2023

Publication series

NameProceedings - 2023 IEEE International Parallel and Distributed Processing Symposium, IPDPS 2023

Conference

Conference37th IEEE International Parallel and Distributed Processing Symposium, IPDPS 2023
Country/TerritoryUnited States
CitySt. Petersburg
Period5/15/235/19/23

All Science Journal Classification (ASJC) codes

  • Artificial Intelligence
  • Computer Networks and Communications
  • Hardware and Architecture
  • Information Systems

Keywords

  • Lossy compression
  • bit plane
  • compression throughput
  • embedded coding

Fingerprint

Dive into the research topics of 'ZFP-X: Efficient Embedded Coding for Accelerating Lossy Floating Point Compression'. Together they form a unique fingerprint.

Cite this