Load-aware Elastic Data Reduction and Re-computation for Adaptive Mesh Refinement

Mengxiao Wang, Huizhang Luo, Qing Liu, Hong Jiang

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Abstract

The increasing performance gap between computation and I/O creates huge data management challenges for simulation-based scientific discovery. Data reduction, among others, is deemed to be a promising technique to bridge the gap through reducing the amount of data migrated to persistent storage. However, the reduction performance is still far from what is being demanded from production applications. To this end, we propose a new methodology that aggressively reduces data despite the substantial loss of information, and re-computes the original accuracy on-demand. As a result, our scheme creates an illusion of a fast and large storage medium with the availability of high-accuracy data. We further design a load-aware data reduction strategy that monitors the I/O overhead at runtime, and dynamically adjusts the reduction ratio. We verify the efficacy of our methodology through adaptive mesh refinement, a popular numerical technique for solving partial differential equations. We evaluate data reduction and selective data re-computation on Titan, using a real application in FLASH and mini-applications in Chombo. To clearly demonstrate the benefits of re-computation, we compare it with other state-of-the-art data reduction methods including SZ, ZFP, FPC and deduplication, and it is shown to be superior in both write and read speeds, particularly when a small amount of data (e.g., 1%) need to be retrieved, as well as reduction ratio. Our results confirm that data reduction and selective data re-computation can 1) reduce the performance gap between I/O and compute via aggressively reducing AMR levels, and more importantly 2) can recover the target accuracy efficiently for AMR through re-computation.

Original languageEnglish (US)
Title of host publication2019 IEEE International Conference on Networking, Architecture and Storage, NAS 2019 - Proceedings
PublisherInstitute of Electrical and Electronics Engineers Inc.
ISBN (Electronic)9781728144092
DOIs
StatePublished - Aug 2019
Event14th IEEE International Conference on Networking, Architecture and Storage, NAS 2019 - Enshi, China
Duration: Aug 15 2019Aug 17 2019

Publication series

Name2019 IEEE International Conference on Networking, Architecture and Storage, NAS 2019 - Proceedings

Conference

Conference14th IEEE International Conference on Networking, Architecture and Storage, NAS 2019
CountryChina
CityEnshi
Period8/15/198/17/19

All Science Journal Classification (ASJC) codes

  • Information Systems and Management
  • Computer Networks and Communications
  • Hardware and Architecture

Fingerprint Dive into the research topics of 'Load-aware Elastic Data Reduction and Re-computation for Adaptive Mesh Refinement'. Together they form a unique fingerprint.

Cite this