Bootstrapping in-situ workflow auto-Tuning via combining performance models of component applications

Tong Shu, Yanfei Guo, Justin Wozniak, Xiaoning Ding, Ian Foster, Tahsin Kurc

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Abstract

In an in-situ workflow, multiple components such as simulation and analysis applications are coupled with streaming data transfers the multiplicity of possible configurations necessitates an auto-Tuner for workflow optimization. Existing auto-Tuning approaches are computationally expensive because many configurations must be sampled by running the whole workflow repeatedly in order to train the autotuner surrogate model or otherwise explore the configuration space. To reduce these costs, we instead combine the performance models of component applications by exploiting the analytical workflow structure, selectively generating test configurations to measure and guide the training of a machine learning workflow surrogate model. Because the training can focus on well-performing configurations, the resulting surrogate model can achieve high prediction accuracy for good configurations despite training with fewer total configurations. Experiments with real applications demonstrate that our approach can identify significantly better configurations than other approaches for a fixed computer time budget.

Original languageEnglish (US)
Title of host publicationProceedings of SC 2021
Subtitle of host publicationThe International Conference for High Performance Computing, Networking, Storage and Analysis: Science and Beyond
PublisherIEEE Computer Society
ISBN (Electronic)9781450384421
DOIs
StatePublished - Nov 14 2021
Event33rd International Conference for High Performance Computing, Networking, Storage and Analysis: Science and Beyond, SC 2021 - Virtual, Online, United States
Duration: Nov 14 2021Nov 19 2021

Publication series

NameInternational Conference for High Performance Computing, Networking, Storage and Analysis, SC
ISSN (Print)2167-4329
ISSN (Electronic)2167-4337

Conference

Conference33rd International Conference for High Performance Computing, Networking, Storage and Analysis: Science and Beyond, SC 2021
Country/TerritoryUnited States
CityVirtual, Online
Period11/14/2111/19/21

All Science Journal Classification (ASJC) codes

  • Computer Networks and Communications
  • Computer Science Applications
  • Hardware and Architecture
  • Software

Keywords

  • Auto-Tuning
  • Bootstrapping
  • Component model combination
  • In-situ workflow

Fingerprint

Dive into the research topics of 'Bootstrapping in-situ workflow auto-Tuning via combining performance models of component applications'. Together they form a unique fingerprint.

Cite this