Profiling optimization for big data transfer over dedicated channels

Daqing Yun, Chase Q. Wu, Nageswara S.V. Rao, Qiang Liu, Rajkumar Kettimuthu, Eun Sung Jung

Research output: Chapter in Book/Report/Conference proceedingConference contribution

6 Scopus citations

Abstract

The transfer of big data is increasingly supported by dedicated channels in high-performance networks, where transport protocols play an important role in maximizing application-level throughput and link utilization. The performance of transport protocols largely depend on their control parameter settings, but it is prohibitively time consuming to conduct an exhaustive search in a large parameter space to find the best set of parameter values. We propose FastProf, a stochastic approximation-based transport profiler, to quickly determine the optimal operational zone of a given data transfer protocol/method over dedicated channels. We implement and test the proposed method using both emulations based on real-life performance measurements and experiments over physical connections with short (2ms) and long (380ms) delays. Both the emulation and experimental results show that FastProf significantly reduces the profiling overhead while achieving a comparable level of end-to-end throughput performance with the exhaustive search-based approach.

Original languageEnglish (US)
Title of host publication2016 25th International Conference on Computer Communications and Networks, ICCCN 2016
PublisherInstitute of Electrical and Electronics Engineers Inc.
ISBN (Electronic)9781509022793
DOIs
StatePublished - Sep 14 2016
Event25th International Conference on Computer Communications and Networks, ICCCN 2016 - Waikoloa, United States
Duration: Aug 1 2016Aug 4 2016

Publication series

Name2016 25th International Conference on Computer Communications and Networks, ICCCN 2016

Other

Other25th International Conference on Computer Communications and Networks, ICCCN 2016
CountryUnited States
CityWaikoloa
Period8/1/168/4/16

All Science Journal Classification (ASJC) codes

  • Computer Networks and Communications
  • Computer Science Applications
  • Hardware and Architecture

Keywords

  • Big data transfer
  • Dedicated channels
  • High-performance networks
  • Profiling
  • Stochastic approximation

Fingerprint Dive into the research topics of 'Profiling optimization for big data transfer over dedicated channels'. Together they form a unique fingerprint.

Cite this