On scheduling of high-throughput scientific workflows under budget constraints in multi-cloud environments

Ruxia Li, Chase Q. Wu, Aiqin Hou, Yongqiang Wang, Tianyu Gao, Mingrui Xu

Research output: Chapter in Book/Report/Conference proceedingConference contribution

1 Scopus citations

Abstract

With the rapid development and deployment of cloud computing infrastructures, many applications in various scientific domains are increasingly utilizing cloud resources for big data storage and analysis. Particularly, it has become a significant challenge to manage and execute big data scientific workflows in multi-cloud environments to process streaming datasets. In this paper, within a three-layer workflow architecture with inter-and intra-cloud data transfer, we formulate a scientific workflow mapping problem under budget constraints to achieve the maximum throughput of streaming workflow applications. We propose a scheduling algorithm to identify the global bottleneck and maximize the throughput under budget constraints. Extensive simulation results show that the proposed algorithm exhibits superior performance over existing heuristic algorithms in scheduling streaming workflow applications.

Original languageEnglish (US)
Title of host publicationProceedings - 16th IEEE International Symposium on Parallel and Distributed Processing with Applications, 17th IEEE International Conference on Ubiquitous Computing and Communications, 8th IEEE International Conference on Big Data and Cloud Computing, 11th IEEE International Conference on Social Computing and Networking and 8th IEEE International Conference on Sustainable Computing and Communications, ISPA/IUCC/BDCloud/SocialCom/SustainCom 2018
EditorsJinjun Chen, Laurence T. Yang
PublisherInstitute of Electrical and Electronics Engineers Inc.
Pages1087-1094
Number of pages8
ISBN (Electronic)9781728111414
DOIs
StatePublished - Mar 20 2019
Event16th IEEE International Symposium on Parallel and Distributed Processing with Applications, 17th IEEE International Conference on Ubiquitous Computing and Communications, 8th IEEE International Conference on Big Data and Cloud Computing, 11th IEEE International Conference on Social Computing and Networking and 8th IEEE International Conference on Sustainable Computing and Communications, ISPA/IUCC/BDCloud/SocialCom/SustainCom 2018 - Melbourne, Australia
Duration: Dec 11 2018Dec 13 2018

Publication series

NameProceedings - 16th IEEE International Symposium on Parallel and Distributed Processing with Applications, 17th IEEE International Conference on Ubiquitous Computing and Communications, 8th IEEE International Conference on Big Data and Cloud Computing, 11th IEEE International Conference on Social Computing and Networking and 8th IEEE International Conference on Sustainable Computing and Communications, ISPA/IUCC/BDCloud/SocialCom/SustainCom 2018

Conference

Conference16th IEEE International Symposium on Parallel and Distributed Processing with Applications, 17th IEEE International Conference on Ubiquitous Computing and Communications, 8th IEEE International Conference on Big Data and Cloud Computing, 11th IEEE International Conference on Social Computing and Networking and 8th IEEE International Conference on Sustainable Computing and Communications, ISPA/IUCC/BDCloud/SocialCom/SustainCom 2018
CountryAustralia
CityMelbourne
Period12/11/1812/13/18

All Science Journal Classification (ASJC) codes

  • Computational Theory and Mathematics
  • Computer Science Applications

Keywords

  • Cloud computing
  • Scientific workflow
  • Throughput
  • Workflow mapping

Fingerprint Dive into the research topics of 'On scheduling of high-throughput scientific workflows under budget constraints in multi-cloud environments'. Together they form a unique fingerprint.

Cite this