Six degrees of scientific data: Reading patterns for extreme scale science IO

Jay Lofstead, Milo Polte, Garth Gibson, Scott Klasky, Karsten Schwan, Ron Oldfield, Matthew Wolf, Qing Liu

Research output: Chapter in Book/Report/Conference proceedingConference contribution

78 Scopus citations

Abstract

Petascale science simulations generate 10s of TBs of application data per day, much of it devoted to their checkpoint/restart fault tolerance mechanisms. Previous work demonstrated the importance of carefully managing such output to prevent application slowdown due to IO blocking, resource contention negatively impacting simulation performance and to fully exploit the IO bandwidth available to the petascale machine. This paper takes a further step in understanding and managing extreme-scale IO. Specifically, its evaluations seek to understand how to efficiently read data for subsequent data analysis, visualization, checkpoint restart after a failure, and other read-intensive operations. In their entirety, these actions support the 'end-to-end' needs of scientists enabling the scientific processes being undertaken. Contributions include the following. First, working with application scientists, we define 'read' benchmarks that capture the common read patterns used by analysis codes. Second, these read patterns are used to evaluate different IO techniques at scale to understand the effects of alternative data sizes and organizations in relation to the performance seen by end users. Third, defining the novel notion of a 'data district' to characterize how data is organized for reads, we experimentally compare the read performance seen with the ADIOS middleware's log-based BP format to that seen by the logically contiguous NetCDF or HDF5 formats commonly used by analysis tools. Measurements assess the performance seen across patterns and with different data sizes, organizations, and read process counts. Outcomes demonstrate that high end-to-end IO performance requires data organizations that offer flexibility in data layout and placement on parallel storage targets, including in ways that can make tradeoffs in the performance of data writes vs. reads.

Original languageEnglish (US)
Title of host publicationHPDC'11 - Proceedings of the 20th International Symposium on High Performance Distributed Computing
Pages49-60
Number of pages12
DOIs
StatePublished - 2011
Externally publishedYes
Event20th ACM International Symposium on High-Performance Parallel and Distributed Computing, HPDC'11 - San Jose, CA, United States
Duration: Jun 8 2011Jun 11 2011

Publication series

NameProceedings of the IEEE International Symposium on High Performance Distributed Computing
ISSN (Print)1082-8907

Other

Other20th ACM International Symposium on High-Performance Parallel and Distributed Computing, HPDC'11
Country/TerritoryUnited States
CitySan Jose, CA
Period6/8/116/11/11

All Science Journal Classification (ASJC) codes

  • Software
  • Computer Networks and Communications

Keywords

  • adios
  • analysis
  • hdf5
  • log-based
  • logically contiguous
  • netcdf
  • pnetcdf
  • visualization

Fingerprint

Dive into the research topics of 'Six degrees of scientific data: Reading patterns for extreme scale science IO'. Together they form a unique fingerprint.

Cite this