On performance modeling and prediction in support of scientific workflow optimization

Qishi Wu, Vivek V. Datla

Research output: Chapter in Book/Report/Conference proceedingConference contribution

20 Scopus citations

Abstract

The computing modules in distributed scientific workflows must be mapped to computer nodes in shared network environments for optimal workflow performance. Finding a good workflow mapping scheme critically depends on an accurate prediction of the execution time of each individual computational module in the workflow. The time prediction of a scientific computation does not have a silver bullet as it is determined collectively by several dynamic system factors including concurrent loads, memory size, CPU speed, and also by the complexity of the computational program itself. This paper investigates the problem of modeling scientific computations and predicting their execution time based on a combination of both hardware and software properties. We employ statistical learning techniques to estimate the effective computational power of a given computer node at any point of time and estimate the total number of CPU cycles needed for executing a given computational program on any input data size. We analytically derive an upper bound of the estimation error for execution time prediction given the hardware and software properties. The proposed statistical analysis-based solution to performance modeling and prediction is validated and justified by experimental results measured on the computing nodes that vary significantly in terms of the hardware specifications.

Original languageEnglish (US)
Title of host publicationProceedings - 2011 IEEE World Congress on Services, SERVICES 2011
Pages161-168
Number of pages8
DOIs
StatePublished - 2011
Externally publishedYes
Event2011 IEEE World Congress on Services, SERVICES 2011 - Washington, DC, United States
Duration: Jul 4 2011Jul 9 2011

Publication series

NameProceedings - 2011 IEEE World Congress on Services, SERVICES 2011

Other

Other2011 IEEE World Congress on Services, SERVICES 2011
Country/TerritoryUnited States
CityWashington, DC
Period7/4/117/9/11

All Science Journal Classification (ASJC) codes

  • Electrical and Electronic Engineering

Keywords

  • Performance modeling
  • Regression techniques
  • Scientific computation

Fingerprint

Dive into the research topics of 'On performance modeling and prediction in support of scientific workflow optimization'. Together they form a unique fingerprint.

Cite this