Region-adaptive, Error-controlled Scientific Data Compression using Multilevel Decomposition

Qian Gong, Ben Whitney, Chengzhu Zhang, Xin Liang, Anand Rangarajan, Jieyang Chen, Lipeng Wan, Paul Ullrich, Qing Liu, Robert Jacob, Sanjay Ranka, Scott Klasky

Research output: Chapter in Book/Report/Conference proceedingConference contribution

3 Scopus citations

Abstract

The increase of computer processing speed is significantly outpacing improvements in network and storage bandwidth, leading to the big data challenge in modern science, where scientific applications can quickly generate much more data than that can be transferred and stored. As a result, big scientific data must be reduced by a few orders of magnitude while the accuracy of the reduced data needs to be guaranteed for further scientific explorations. Moreover, scientists are often interested in some specific spatial/temporal regions in their data, where higher accuracy is required. The locations of the regions requiring high accuracy can sometimes be prescribed based on application knowledge, while other times they must be estimated based on general spatial/temporal variation. In this paper, we develop a novel multilevel approach which allows users to impose region-wise compression error bounds. Our method utilizes the byproduct of a multilevel compressor to detect regions where details are rich and we provide the theoretical underpinning for region-wise error control. With spatially varying precision preservation, our approach can achieve significantly higher compression ratios than single-error bounded compression approaches and control errors in the regions of interest. We conduct the evaluations on two climate use cases-one targeting small-scale, node features and the other focusing on long, areal features. For both use cases, the locations of the features were unknown ahead of the compression. By selecting approximately 16% of the data based on multi-scale spatial variations and compressing those regions with smaller error tolerances than the rest, our approach improves the accuracy of post-analysis by approximately 2 × compared to single-error-bounded compression at the same compression ratio. Using the same error bound for the region of interest, our approach can achieve an increase of more than 50% in overall compression ratio.

Original languageEnglish (US)
Title of host publicationScientific and Statistical Database Management - 34th International Conference, SSDBM 2022 - Proceedings
EditorsElaheh Pourabbas, Yongluan Zhou, Yuchen Li, Bin Yang
PublisherAssociation for Computing Machinery
ISBN (Electronic)9781450396677
DOIs
StatePublished - Jul 6 2022
Event34th International Conference on Scientific and Statistical Database Management, SSDBM 2022 - Copenhagen, Denmark
Duration: Jul 6 2022Jul 8 2022

Publication series

NameACM International Conference Proceeding Series

Conference

Conference34th International Conference on Scientific and Statistical Database Management, SSDBM 2022
Country/TerritoryDenmark
CityCopenhagen
Period7/6/227/8/22

All Science Journal Classification (ASJC) codes

  • Human-Computer Interaction
  • Computer Networks and Communications
  • Computer Vision and Pattern Recognition
  • Software

Keywords

  • Climate Data Compression
  • Error Control
  • Region-adaptive Lossy Compression

Fingerprint

Dive into the research topics of 'Region-adaptive, Error-controlled Scientific Data Compression using Multilevel Decomposition'. Together they form a unique fingerprint.

Cite this