Stochastic approximation- based transport profiling for big data movement over dedicated connections

Daqing Yun, Chase Q. Wu

Research output: Chapter in Book/Report/Conference proceedingChapter

Abstract

High-performance networks featuring advance bandwidth reservation have been developed and deployed to support big data transfer in extreme-scale scientific applications. The performance of such big data transfer largely depends on the transport protocols being used. For a given protocol in a given network environment, different parameter settings may lead to different performance, and oftentimes the default settings do not yield the best performance. It is, however, impractical to conduct an exhaustive search in the large parameter space of transport protocols for a set of suitable parameter values. This chapter proposes a stochastic approximation-based transport profiler, namely FastProf, to quickly determine the optimal operational zone of a protocol over dedicated connections. The proposed method is evaluated using both emulations based on real-life measurements and experiments over physical connections. The results show that FastProf significantly reduces the profiling overhead while achieving a comparable level of transport performance with the exhaustive search-based approach.

Original languageEnglish (US)
Title of host publicationStochastic Methods for Estimation and Problem Solving in Engineering
PublisherIGI Global
Pages113-138
Number of pages26
ISBN (Electronic)9781522550464
ISBN (Print)1522550453, 9781522550457
DOIs
StatePublished - Mar 2 2018

All Science Journal Classification (ASJC) codes

  • Engineering(all)

Fingerprint Dive into the research topics of 'Stochastic approximation- based transport profiling for big data movement over dedicated connections'. Together they form a unique fingerprint.

Cite this