TY - JOUR
T1 - Combining phase identification and statistic modeling for automated parallel benchmark generation
AU - Jin, Ye
AU - Ma, Xiaosong
AU - Liu, Mingliang
AU - Liu, Qing
AU - Logan, Jeremy
AU - Podhorszki, Norbert
AU - Choi, Jong Youl
AU - Klasky, Scott
N1 - Funding Information:
We thank the anonymous reviewers for their valuable com-ments and suggestions. The work is facilitated by QCRI''s in-ternship program. It is also partially supported by research grants at involved institutes: NSF awards CCF-0937908, CCF-0937690, and CNS-1318564 (NCSU), DOE O_ce of Science, O_ce of Advanced Scienti_c Computing Research, SciDAC program, and Oak Ridge Leadership Computing Facility (ORNL), and the National High-Tech Research and Development Plan (863 project) 2012AA01A302 and NSFC project 61472201 (Tsinghua University).
PY - 2015/6/24
Y1 - 2015/6/24
N2 - Parallel application benchmarks are indispensable for evaluating/optimizing HPC software and hardware. However, it is very challenging and costly to obtain high-fidelity benchmarks reecting the scale and complexity of state-of-the-art parallel applications. Hand-extracted synthetic benchmarks are time- and labor-intensive to create. Real applications themselves, while offering most accurate performance evaluation, are expensive to compile, port, reconfigure, and often plainly inaccessible due to security or ownership concerns. This work contributes APPrime, a novel tool for tracebased automatic parallel benchmark generation. Taking as input standard communication-I/O traces of an application's execution, it couples accurate automatic phase identification with statistical regeneration of event parameters to create compact, portable, and to some degree reconfigurable parallel application benchmarks. Experiments with four NAS Parallel Benchmarks (NPB) and three real scientific simulation codes confirm the fidelity of APPrime benchmarks. They retain the original applications' performance characteristics, in particular their relative performance across platforms. Also, the result benchmarks, already released online, are much more compact and easy-toport compared to the original applications.
AB - Parallel application benchmarks are indispensable for evaluating/optimizing HPC software and hardware. However, it is very challenging and costly to obtain high-fidelity benchmarks reecting the scale and complexity of state-of-the-art parallel applications. Hand-extracted synthetic benchmarks are time- and labor-intensive to create. Real applications themselves, while offering most accurate performance evaluation, are expensive to compile, port, reconfigure, and often plainly inaccessible due to security or ownership concerns. This work contributes APPrime, a novel tool for tracebased automatic parallel benchmark generation. Taking as input standard communication-I/O traces of an application's execution, it couples accurate automatic phase identification with statistical regeneration of event parameters to create compact, portable, and to some degree reconfigurable parallel application benchmarks. Experiments with four NAS Parallel Benchmarks (NPB) and three real scientific simulation codes confirm the fidelity of APPrime benchmarks. They retain the original applications' performance characteristics, in particular their relative performance across platforms. Also, the result benchmarks, already released online, are much more compact and easy-toport compared to the original applications.
KW - Asynchronous I/O
KW - Benchmark generation
KW - HPC applications
KW - Markov chain model
KW - Phase identification
KW - Traces
UR - http://www.scopus.com/inward/record.url?scp=84955562122&partnerID=8YFLogxK
UR - http://www.scopus.com/inward/citedby.url?scp=84955562122&partnerID=8YFLogxK
U2 - 10.1145/2796314.2745876
DO - 10.1145/2796314.2745876
M3 - Conference article
AN - SCOPUS:84955562122
SN - 0163-5999
VL - 43
SP - 309
EP - 320
JO - Performance Evaluation Review
JF - Performance Evaluation Review
IS - 1
T2 - ACM SIGMETRICS International Conference on Measurement and Modeling of Computer Systems, SIGMETRICS 2015
Y2 - 15 June 2015 through 19 June 2015
ER -