Latency modeling and minimization for large-scale scientific workflows in distributed network environments

Qishi Wu, Yi Gu, Yuchen Liao, Xukang Lu, Yunyue Lin, Nageswara S.V. Rao

Research output: Chapter in Book/Report/Conference proceedingConference contribution

8 Scopus citations

Abstract

Large-scale e-science applications feature complex workflows consisting of many computing modules. Mapping such workflows in distributed network environments and minimizing their latency are crucial to those applications that require fast system response and prompt user interaction. We model the time cost of each workflow component and design an efficient algorithm to compute the exact end-to-end delay of the entire workflow by explicitly accounting for the resource sharing dynamics. We further propose a workflow mapping approach to minimize the workflow latency using a recursive optimization procedure. The validity of the cost models and the accuracy of the latency computing algorithm are verified in comparison with an approximate solution, a dynamic system simulation program, and a workflow engine deployed in a real network. The performance superiority of the proposed mapping approach is illustrated by extensive simulation-based comparisons with existing algorithms.

Original languageEnglish (US)
Title of host publication44th Annual Simulation Symposium, ANSS 2011 - Book 2 of SCS's 2011 Spring Simulation Multiconference, Proceedings
EditorsSaad Biaz, Shaoen Wu
PublisherThe Society for Modeling and Simulation International (SCS)
Pages205-212
Number of pages8
ISBN (Electronic)1930638566, 9781930638563
StatePublished - 2011
Externally publishedYes
Event44th Annual Simulation Symposium, ANSS 2011, Part of the 2011 Spring Simulation Multiconference, SpringSim 2011 - Boston, United States
Duration: Apr 3 2011Apr 7 2011

Publication series

Name44th Annual Simulation Symposium, ANSS 2011 - Book 2 of SCS's 2011 Spring Simulation Multiconference, Proceedings

Conference

Conference44th Annual Simulation Symposium, ANSS 2011, Part of the 2011 Spring Simulation Multiconference, SpringSim 2011
Country/TerritoryUnited States
CityBoston
Period4/3/114/7/11

All Science Journal Classification (ASJC) codes

  • Modeling and Simulation

Keywords

  • Distributed computing
  • Latency
  • Modeling
  • Workflow

Fingerprint

Dive into the research topics of 'Latency modeling and minimization for large-scale scientific workflows in distributed network environments'. Together they form a unique fingerprint.

Cite this