On an integrated mapping and scheduling solution to large-scale scientific workflows in resource sharing environments

Daqing Yun, Qishi Wu, Yi Gu, Xiyang Liu

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Abstract

Next-generation e-science applications feature large-scale data-intensive workflows comprised of many interrelated computing modules. The end-to-end performance of such scientific workflows depends on both the mapping scheme, which determines module assignment, and the scheduling policy, which determines resource allocation if multiple modules are mapped to the same node. These two aspects of workflow optimization are traditionally treated as two separated topics, and the interactions between them have not been fully explored by any existing efforts. As the scale of scientific workflows and the complexity of network environments rapidly increase, each individual aspect of performance optimization alone can only meet with limited success. We conduct an in-depth investigation into workflow execution dynamics of both mapping and scheduling, and propose an integrated solution, referred to as Mapping and Scheduling Interaction (MSI), to achieve a higher level of resource utilization and workflow performance. The efficacy of MSI is illustrated by extensive simulation-based workflow experiments.

Original languageEnglish (US)
Title of host publicationProceedings of the 2013 Spring Simulation Multiconference, SpringSim 2013 - 46th Annual Simulation Symposium, ANSS 2013
Pages49-56
Number of pages8
Edition2
StatePublished - May 3 2013
Externally publishedYes
Event46th Annual Simulation Symposium, ANSS 2013, Part of the 2013 Spring Simulation Multiconference, SpringSim 2013 - San Diego, CA, United States
Duration: Apr 7 2013Apr 10 2013

Publication series

NameSimulation Series
Number2
Volume45
ISSN (Print)0735-9276

Other

Other46th Annual Simulation Symposium, ANSS 2013, Part of the 2013 Spring Simulation Multiconference, SpringSim 2013
Country/TerritoryUnited States
CitySan Diego, CA
Period4/7/134/10/13

All Science Journal Classification (ASJC) codes

  • Computer Networks and Communications

Keywords

  • End-to-end delay
  • On-node job scheduling
  • Workflow mapping

Fingerprint

Dive into the research topics of 'On an integrated mapping and scheduling solution to large-scale scientific workflows in resource sharing environments'. Together they form a unique fingerprint.

Cite this