Autonomous learning of load and traffic patterns to improve cluster utilization

Andrew Sohn, Hukeun Kwak, Kyusik Chung

Research output: Chapter in Book/Report/Conference proceedingConference contribution

2 Scopus citations

Abstract

Adaptive clustering aims at improving cluster utilization for varying load and traffic patterns. Locality-based least-connection with replication (LBLCR) scheduling that comes with Linux is designed to help improve cluster utilization through adaptive clustering. A key issue with LBLCR, however, is that cluster performance depends much on a single threshold value that is used to determine adaptation. Once set, me threshold remains fixed regardless of the load and traffic patterns. If a cluster of PCs is to adapt to different traffic patterns for high utilization, a good threshold has to be selected and used dynamically. We present in this report an adaptive clustering framework that autonomously learns and adapts to different load and traffic patterns at runtime with no administrator intervention. Cluster is configured once and for all. As the patterns change the cluster automatically expands/contracts to meet the changing demands. At the same time, the patterns are proactively learned that when similar patterns emerge in me future, the cluster knows what to do to improve utilization. We have implemented this autonomous learning method and compared with LBLCR using published Web traces. Experimental results indicate that our autonomous learning method shows high performance scalability and adaptability for different patterns. On the other hand LBLCR-based clustering suffers from performance scalability and adaptability for different traffic patterns since it is not designed to obtain good threshold values and use mem at runtime.

Original languageEnglish (US)
Title of host publicationArchitecture of Computing Systems - ARCS 2007
Subtitle of host publication20th International Conference, Proceedings
PublisherSpringer Verlag
Pages224-239
Number of pages16
ISBN (Print)9783540712671
DOIs
StatePublished - Jan 1 2007
Event20th International Conference on Architecture of Computing Systems, ARCS 2007 - Zurich, Switzerland
Duration: Mar 12 2007Mar 15 2007

Publication series

NameLecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)
Volume4415 LNCS
ISSN (Print)0302-9743
ISSN (Electronic)1611-3349

Other

Other20th International Conference on Architecture of Computing Systems, ARCS 2007
CountrySwitzerland
CityZurich
Period3/12/073/15/07

All Science Journal Classification (ASJC) codes

  • Theoretical Computer Science
  • Computer Science(all)

Fingerprint Dive into the research topics of 'Autonomous learning of load and traffic patterns to improve cluster utilization'. Together they form a unique fingerprint.

Cite this