Modeling and simulation of distributed computing workflows in heterogeneous network environments

Qishi Wu, Yi Gu

Research output: Contribution to journalArticlepeer-review

3 Scopus citations

Abstract

Next-generation computation- and network-intensive collaborative applications in various science, engineering, and e-commerce fields feature large-scale computing workflows of complex structures. Efficient algorithms are needed for task scheduling, module deployment, and service provisioning to support the execution of such distributed workflows in heterogeneous network environments and optimize their end-to-end performance for fast system response or smooth data flow. However, deploying large-scale distributed applications in real network environments is extremely challenging due to the inherent dynamics in the reliability, availability, accessibility, and capacity of massively distributed system resources, which are typically shared among a broad community of users over the Internet or dedicated connections. We propose a simulation system to study the execution dynamics of distributed computing workflows and evaluate the network performance of workflow scheduling or mapping algorithms before actual deployment and experimentation. The proposed simulation system visually illustrates the dynamic execution process of workflows in network environments by simulating module execution on computer nodes and data transfer over network links in a completely distributed and parallel manner. Furthermore, the simulation system takes background traffic and workload into consideration to achieve a high level of simulation accuracy for distributed applications deployed in shared production network environments. We implement the simulation system using multi-threaded programming and conduct extensive testings on various mapping schemes using a large number of simulated workflows and networks. The simulation-based performance measurements are quantitatively confirmed by both the experimental observations collected in real networks and the theoretical results obtained by rigorous performance analysis based on well-defined mathematical models.

Original languageEnglish (US)
Pages (from-to)1049-1065
Number of pages17
JournalSIMULATION
Volume87
Issue number12
DOIs
StatePublished - Dec 2011
Externally publishedYes

All Science Journal Classification (ASJC) codes

  • Software
  • Modeling and Simulation
  • Computer Graphics and Computer-Aided Design

Keywords

  • dynamic simulation
  • end-to-end delay
  • frame rate
  • workflow mapping

Fingerprint Dive into the research topics of 'Modeling and simulation of distributed computing workflows in heterogeneous network environments'. Together they form a unique fingerprint.

Cite this