A Distributed Workflow Management System with Case Study of Real-life Scientific Applications on Grids

Qishi Wu, Mengxia Zhu, Yi Gu, Patrick Brown, Xukang Lu, Wuyin Lin, Yangang Liu

Research output: Contribution to journalArticlepeer-review

16 Scopus citations

Abstract

Next-generation scientific applications feature complex workflows comprised of many computing modules with intricate inter-module dependencies. Supporting such scientific workflows in wide-area networks especially Grids and optimizing their performance are crucial to the success of collaborative scientific discovery. We develop a Scientific Workflow Automation and Management Platform (SWAMP), which enables scientists to conveniently assemble, execute, monitor, control, and steer computing workflows in distributed environments via a unified web-based user interface. The SWAMP architecture is built entirely on a seamless composition of web services: the functionalities of its own are provided and its interactions with other tools or systems are enabled through web services for easy access over standard Internet protocols while being independent of different platforms and programming languages. SWAMP also incorporates a class of efficient workflow mapping schemes to achieve optimal end-to-end performance based on rigorous performance modeling and algorithm design. The performance superiority of SWAMP over existing workflow mapping schemes is justified by extensive simulations, and the system efficacy is illustrated by large-scale experiments on real-life scientific workflows for climate modeling through effective system implementation, deployment, and testing on the Open Science Grid.

Original languageEnglish (US)
Pages (from-to)367-393
Number of pages27
JournalJournal of Grid Computing
Volume10
Issue number3
DOIs
StatePublished - Sep 2012
Externally publishedYes

All Science Journal Classification (ASJC) codes

  • Software
  • Information Systems
  • Hardware and Architecture
  • Computer Networks and Communications

Keywords

  • Climate modeling
  • Distributed computing
  • Open Science Grid
  • Scientific workflow

Fingerprint

Dive into the research topics of 'A Distributed Workflow Management System with Case Study of Real-life Scientific Applications on Grids'. Together they form a unique fingerprint.

Cite this