In-situ workflow auto-tuning through combining component models

Tong Shu; Yanfei Guo; Justin Wozniak; Xiaoning Ding; Ian Foster; Tahsin Kurc

doi:10.1145/3437801.3441615

In-situ workflow auto-tuning through combining component models

Tong Shu, Yanfei Guo, Justin Wozniak, Xiaoning Ding, Ian Foster, Tahsin Kurc

Computer Science

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution

8 Scopus citations

Abstract

In-situ parallel workflows couple multiple component applications via streaming data transfer to avoid data exchange via shared file systems. Such workflows are challenging to configure for optimal performance due to the huge space of possible configurations. Here, we propose an in-situ workflow auto-tuning method, ALIC, which integrates machine learning techniques with knowledge of in-situ workflow structures to enable automated workflow configuration with a limited number of performance measurements. Experiments with real applications show that ALIC identify better configurations than existing methods given a computer time budget.

Original language	English (US)
Title of host publication	PPoPP 2021 - Proceedings of the 2021 26th ACM SIGPLAN Symposium on Principles and Practice of Parallel Programming
Publisher	Association for Computing Machinery
Pages	467-468
Number of pages	2
ISBN (Electronic)	9781450382946
DOIs	https://doi.org/10.1145/3437801.3441615
State	Published - Feb 17 2021
Event	26th ACM SIGPLAN Symposium on Principles and Practice of Parallel Programming, PPoPP 2021 - Virtual, Online, Korea, Republic of Duration: Feb 27 2021 → Mar 3 2021

Publication series

Name	Proceedings of the ACM SIGPLAN Symposium on Principles and Practice of Parallel Programming, PPOPP

Conference

Conference	26th ACM SIGPLAN Symposium on Principles and Practice of Parallel Programming, PPoPP 2021
Country/Territory	Korea, Republic of
City	Virtual, Online
Period	2/27/21 → 3/3/21

All Science Journal Classification (ASJC) codes

Software

Keywords

auto-tuning
component model
in-situ workflow

Access to Document

10.1145/3437801.3441615

Cite this

Shu, T., Guo, Y., Wozniak, J., Ding, X., Foster, I., & Kurc, T. (2021). In-situ workflow auto-tuning through combining component models. In PPoPP 2021 - Proceedings of the 2021 26th ACM SIGPLAN Symposium on Principles and Practice of Parallel Programming (pp. 467-468). (Proceedings of the ACM SIGPLAN Symposium on Principles and Practice of Parallel Programming, PPOPP). Association for Computing Machinery. https://doi.org/10.1145/3437801.3441615

Shu, Tong ; Guo, Yanfei ; Wozniak, Justin et al. / In-situ workflow auto-tuning through combining component models. PPoPP 2021 - Proceedings of the 2021 26th ACM SIGPLAN Symposium on Principles and Practice of Parallel Programming. Association for Computing Machinery, 2021. pp. 467-468 (Proceedings of the ACM SIGPLAN Symposium on Principles and Practice of Parallel Programming, PPOPP).

@inproceedings{c583ec4c3693435394ce37fa30d1ba98,

title = "In-situ workflow auto-tuning through combining component models",

abstract = "In-situ parallel workflows couple multiple component applications via streaming data transfer to avoid data exchange via shared file systems. Such workflows are challenging to configure for optimal performance due to the huge space of possible configurations. Here, we propose an in-situ workflow auto-tuning method, ALIC, which integrates machine learning techniques with knowledge of in-situ workflow structures to enable automated workflow configuration with a limited number of performance measurements. Experiments with real applications show that ALIC identify better configurations than existing methods given a computer time budget.",

keywords = "auto-tuning, component model, in-situ workflow",

author = "Tong Shu and Yanfei Guo and Justin Wozniak and Xiaoning Ding and Ian Foster and Tahsin Kurc",

note = "Publisher Copyright: {\textcopyright} 2021 Owner/Author.; 26th ACM SIGPLAN Symposium on Principles and Practice of Parallel Programming, PPoPP 2021 ; Conference date: 27-02-2021 Through 03-03-2021",

year = "2021",

month = feb,

day = "17",

doi = "10.1145/3437801.3441615",

language = "English (US)",

series = "Proceedings of the ACM SIGPLAN Symposium on Principles and Practice of Parallel Programming, PPOPP",

publisher = "Association for Computing Machinery",

pages = "467--468",

booktitle = "PPoPP 2021 - Proceedings of the 2021 26th ACM SIGPLAN Symposium on Principles and Practice of Parallel Programming",

}

Shu, T, Guo, Y, Wozniak, J, Ding, X, Foster, I & Kurc, T 2021, In-situ workflow auto-tuning through combining component models. in PPoPP 2021 - Proceedings of the 2021 26th ACM SIGPLAN Symposium on Principles and Practice of Parallel Programming. Proceedings of the ACM SIGPLAN Symposium on Principles and Practice of Parallel Programming, PPOPP, Association for Computing Machinery, pp. 467-468, 26th ACM SIGPLAN Symposium on Principles and Practice of Parallel Programming, PPoPP 2021, Virtual, Online, Korea, Republic of, 2/27/21. https://doi.org/10.1145/3437801.3441615

In-situ workflow auto-tuning through combining component models. / Shu, Tong; Guo, Yanfei; Wozniak, Justin et al.
PPoPP 2021 - Proceedings of the 2021 26th ACM SIGPLAN Symposium on Principles and Practice of Parallel Programming. Association for Computing Machinery, 2021. p. 467-468 (Proceedings of the ACM SIGPLAN Symposium on Principles and Practice of Parallel Programming, PPOPP).

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution

TY - GEN

T1 - In-situ workflow auto-tuning through combining component models

AU - Shu, Tong

AU - Guo, Yanfei

AU - Wozniak, Justin

AU - Ding, Xiaoning

AU - Foster, Ian

AU - Kurc, Tahsin

PY - 2021/2/17

Y1 - 2021/2/17

N2 - In-situ parallel workflows couple multiple component applications via streaming data transfer to avoid data exchange via shared file systems. Such workflows are challenging to configure for optimal performance due to the huge space of possible configurations. Here, we propose an in-situ workflow auto-tuning method, ALIC, which integrates machine learning techniques with knowledge of in-situ workflow structures to enable automated workflow configuration with a limited number of performance measurements. Experiments with real applications show that ALIC identify better configurations than existing methods given a computer time budget.

AB - In-situ parallel workflows couple multiple component applications via streaming data transfer to avoid data exchange via shared file systems. Such workflows are challenging to configure for optimal performance due to the huge space of possible configurations. Here, we propose an in-situ workflow auto-tuning method, ALIC, which integrates machine learning techniques with knowledge of in-situ workflow structures to enable automated workflow configuration with a limited number of performance measurements. Experiments with real applications show that ALIC identify better configurations than existing methods given a computer time budget.

KW - auto-tuning

KW - component model

KW - in-situ workflow

UR - http://www.scopus.com/inward/record.url?scp=85101706149&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=85101706149&partnerID=8YFLogxK

U2 - 10.1145/3437801.3441615

DO - 10.1145/3437801.3441615

M3 - Conference contribution

AN - SCOPUS:85101706149

T3 - Proceedings of the ACM SIGPLAN Symposium on Principles and Practice of Parallel Programming, PPOPP

SP - 467

EP - 468

BT - PPoPP 2021 - Proceedings of the 2021 26th ACM SIGPLAN Symposium on Principles and Practice of Parallel Programming

PB - Association for Computing Machinery

T2 - 26th ACM SIGPLAN Symposium on Principles and Practice of Parallel Programming, PPoPP 2021

Y2 - 27 February 2021 through 3 March 2021

ER -

Shu T, Guo Y, Wozniak J, Ding X, Foster I, Kurc T. In-situ workflow auto-tuning through combining component models. In PPoPP 2021 - Proceedings of the 2021 26th ACM SIGPLAN Symposium on Principles and Practice of Parallel Programming. Association for Computing Machinery. 2021. p. 467-468. (Proceedings of the ACM SIGPLAN Symposium on Principles and Practice of Parallel Programming, PPOPP). doi: 10.1145/3437801.3441615

In-situ workflow auto-tuning through combining component models

Abstract

Publication series

Conference

All Science Journal Classification (ASJC) codes

Keywords

Access to Document

Other files and links

Fingerprint

Cite this