Combining phase identification and statistic modeling for automated parallel benchmark generation

Ye Jin, Xiaosong Ma, Mingliang Liu, Qing Liu, Jeremy Logan, Norbert Podhorszki, Jong Youl Choi, Scott Klasky

Research output: Contribution to journalConference articlepeer-review

10 Scopus citations


Parallel application benchmarks are indispensable for evaluating/optimizing HPC software and hardware. However, it is very challenging and costly to obtain high-fidelity benchmarks reecting the scale and complexity of state-of-the-art parallel applications. Hand-extracted synthetic benchmarks are time- and labor-intensive to create. Real applications themselves, while offering most accurate performance evaluation, are expensive to compile, port, reconfigure, and often plainly inaccessible due to security or ownership concerns. This work contributes APPrime, a novel tool for tracebased automatic parallel benchmark generation. Taking as input standard communication-I/O traces of an application's execution, it couples accurate automatic phase identification with statistical regeneration of event parameters to create compact, portable, and to some degree reconfigurable parallel application benchmarks. Experiments with four NAS Parallel Benchmarks (NPB) and three real scientific simulation codes confirm the fidelity of APPrime benchmarks. They retain the original applications' performance characteristics, in particular their relative performance across platforms. Also, the result benchmarks, already released online, are much more compact and easy-toport compared to the original applications.

Original languageEnglish (US)
Pages (from-to)309-320
Number of pages12
JournalPerformance Evaluation Review
Issue number1
StatePublished - Jun 24 2015
Externally publishedYes
EventACM SIGMETRICS International Conference on Measurement and Modeling of Computer Systems, SIGMETRICS 2015 - Portland, United States
Duration: Jun 15 2015Jun 19 2015

All Science Journal Classification (ASJC) codes

  • Software
  • Hardware and Architecture
  • Computer Networks and Communications


  • Asynchronous I/O
  • Benchmark generation
  • HPC applications
  • Markov chain model
  • Phase identification
  • Traces


Dive into the research topics of 'Combining phase identification and statistic modeling for automated parallel benchmark generation'. Together they form a unique fingerprint.

Cite this