A distributed workflow management system with case study of real-life scientific applications

Qishi Wu, Mengxia Zhu, Yi Gu, Xukang Lu, Patrick Brown, Michael A. Reuter, Stephen D. Miller

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Abstract

Supporting large-scale scientific workflows in distributed network environments and optimizing their performances are crucial to the success of collaborative scientific discovery. We develop a generic scientific workflow platform, referred to as SciFlow, which constitutes a flexible framework to facilitate the distributed execution and management of scientific workflows and incorporates a class of workflow mapping schemes to achieve optimal end-to-end performances. The functionalities of SciFlow are provided and its interactions with other tools or systems are enabled through web services for easy access over standard Internet protocols while being independent of different platforms and programming languages. The performance superiority of SciFlow over existing workflow mapping schemes and management systems is illustrated by extensive simulations and is further verified by large-scale experiments on real-life scientific workflows through effective system implementation and deployment in distributed network environments.

Original languageEnglish (US)
Title of host publication30th IEEE International Performance Computing and Communications Conference, IPCCC 2011
DOIs
StatePublished - 2011
Externally publishedYes
Event30th IEEE International Performance, Computing and Communications Conference, IPCCC 2011 - Orlando, FL, United States
Duration: Nov 17 2011Nov 19 2011

Publication series

NameConference Proceedings of the IEEE International Performance, Computing, and Communications Conference

Other

Other30th IEEE International Performance, Computing and Communications Conference, IPCCC 2011
Country/TerritoryUnited States
CityOrlando, FL
Period11/17/1111/19/11

All Science Journal Classification (ASJC) codes

  • General Engineering

Keywords

  • distributed computing
  • performance modeling
  • scientific workflow
  • workflow system

Fingerprint

Dive into the research topics of 'A distributed workflow management system with case study of real-life scientific applications'. Together they form a unique fingerprint.

Cite this