Resource planning for parallel processing in the cloud

Justin Y. Shi, Moussa Taifi, Abdallah Khreishah

Research output: Chapter in Book/Report/Conference proceedingConference contribution

21 Scopus citations

Abstract

Before the emergence of commercial cloud computing, interests in parallel algorithm analysis have been mostly academic. When computing and communication resources are charged by hours, cost effective parallel processing would become a required skill. This paper reports a resource planning study using a method derived from classical program time complexity analysis, we call Timing Models. Unlike existing qualitative performance analysis methods, a Timing Model uses application instrumented capacity measures to capture the quantitative dependencies between a computer program (sequential or parallel) and its processing environments. For applications planning to use commercial clouds, this tool is ideally suited for choosing the most cost-effective configuration. The contribution of the proposed tool is its ability to explore multiple dimensions of a program quantitatively to gain non-trivial insights. This paper uses a simple matrix multiplication application to illustrate the modeling, program instrumentation and performance prediction processes. Since cloud vender do offer HPC hardware resources, we use Amazon EC2 as the target processing environments. The computing and communication models are not only useful in choosing the processing platform but also for understanding the resource usage bills. Comparisons between predicted and actual resource usages show that poor processing granularity wastes resources. Prediction errors are minimized near the optimal number of processors.

Original languageEnglish (US)
Title of host publicationProc.- 2011 IEEE International Conference on HPCC 2011 - 2011 IEEE International Workshop on FTDCS 2011 - Workshops of the 2011 Int. Conf. on UIC 2011- Workshops of the 2011 Int. Conf. ATC 2011
Pages828-833
Number of pages6
DOIs
StatePublished - Nov 24 2011
Externally publishedYes
Event13th IEEE International Workshop on FTDCS 2011, the 8th International Conference on ATC 2011, the 8th International Conference on UIC 2011 and the 13th IEEE International Conference on HPCC 2011 - Banff, AB, Canada
Duration: Sep 2 2011Sep 4 2011

Publication series

NameProc.- 2011 IEEE International Conference on HPCC 2011 - 2011 IEEE International Workshop on FTDCS 2011 -Workshops of the 2011 Int. Conf. on UIC 2011- Workshops of the 2011 Int. Conf. ATC 2011

Other

Other13th IEEE International Workshop on FTDCS 2011, the 8th International Conference on ATC 2011, the 8th International Conference on UIC 2011 and the 13th IEEE International Conference on HPCC 2011
Country/TerritoryCanada
CityBanff, AB
Period9/2/119/4/11

All Science Journal Classification (ASJC) codes

  • Artificial Intelligence
  • Computer Networks and Communications

Keywords

  • Parallel program scalability analysis
  • resource analysis for HPC cloud computing

Fingerprint

Dive into the research topics of 'Resource planning for parallel processing in the cloud'. Together they form a unique fingerprint.

Cite this