MGARD+: Optimizing Multilevel Methods for Error-bounded Scientific Data Reduction

Xin Liang, Ben Whitney, Jieyang Chen, Lipeng Wan, Qing Liu, Dingwen Tao, James Kress, David R. Pugmire, Matthew Wolf, Norbert Podhorszki, Scott Klasky

Research output: Contribution to journalArticlepeer-review

Abstract

Data management is becoming increasingly important in dealing with the large amounts of data produced by todays large-scale scientific simulations and instruments. Existing multilevel compression algorithms offer a promising way to manage scientific data at scale, but may suffer from relatively low performance and reduction quality. In this paper, we propose MGARD+, a multilevel data reduction and refactoring framework drawing on previous multilevel methods, to achieve high-performance data decomposition and high-quality error-bounded lossy compression. Our contributions are four-fold: 1) We propose a level-wise coefficient quantization method, which uses different error tolerances to quantize the multilevel coefficients. 2) We propose an adaptive decomposition method which treats the multilevel decomposition as a preconditioner and terminates the decomposition process at an appropriate level. 3) We leverage a set of algorithmic optimization strategies to significantly improve the performance of multilevel decomposition/recomposition. 4) We evaluate our proposed method using four real-world scientific datasets and compare with several state-of-the-art lossy compressors. Experiments demonstrate that our optimizations improve the decomposition/recomposition performance of the existing multilevel method by up to 70X, and the proposed compression method can improve compression ratio by up to 2X compared with other state-of-the-art error-bounded lossy compressors under the same level of data distortion.

Original languageEnglish (US)
JournalIEEE Transactions on Computers
DOIs
StateAccepted/In press - 2021

All Science Journal Classification (ASJC) codes

  • Software
  • Theoretical Computer Science
  • Hardware and Architecture
  • Computational Theory and Mathematics

Keywords

  • Arrays
  • Compressors
  • Computers
  • Data models
  • Distortion
  • Error Control
  • High-performance Computing
  • Lossy Compression
  • Multilevel Decomposition
  • Optimization
  • Quantization (signal)
  • Scientific Data

Fingerprint

Dive into the research topics of 'MGARD+: Optimizing Multilevel Methods for Error-bounded Scientific Data Reduction'. Together they form a unique fingerprint.

Cite this