Data Transfer Advisor with Transport Profiling Optimization

Daqing Yun, Chase Q. Wu, Nageswara S.V. Rao, Qiang Liu, Rajkumar Kettimuthu, Eun Sung Jung

Research output: Chapter in Book/Report/Conference proceedingConference contribution

8 Scopus citations

Abstract

The network infrastructures have been rapidly upgraded in many high-performance networks (HPNs). However, such infrastructure investment has not led to corresponding performance improvement in big data transfer, especially at the application layer, largely due to the complexity of optimizing transport control on end hosts. We design and implement ProbData, a PRofiling Optimization Based DAta Transfer Advisor, to help users determine the most effective data transfer method with the most appropriate control parameter values to achieve the best data transfer performance. ProbData employs a profiling optimization-based approach to exploit the optimal operational zone of various data transfer methods in support of big data transfer in extreme-scale scientific applications. We present a theoretical framework of the optimized profiling approach employed in ProbData as well as its detailed design and implementation. The advising procedure and performance benefits of ProbData are illustrated and evaluated by proof-of-concept experiments in real-life networks.

Original languageEnglish (US)
Title of host publicationProceedings - 2017 IEEE 42nd Conference on Local Computer Networks, LCN 2017
PublisherIEEE Computer Society
Pages269-277
Number of pages9
ISBN (Electronic)9781509065226
DOIs
StatePublished - Nov 14 2017
Event42nd IEEE Conference on Local Computer Networks, LCN 2017 - Singapore, Singapore
Duration: Oct 9 2017Oct 12 2017

Publication series

NameProceedings - Conference on Local Computer Networks, LCN
Volume2017-October

Other

Other42nd IEEE Conference on Local Computer Networks, LCN 2017
Country/TerritorySingapore
CitySingapore
Period10/9/1710/12/17

All Science Journal Classification (ASJC) codes

  • Computer Networks and Communications
  • Hardware and Architecture

Keywords

  • Big data transfer
  • data transfer advising
  • high-performance networks
  • profiling optimization

Fingerprint

Dive into the research topics of 'Data Transfer Advisor with Transport Profiling Optimization'. Together they form a unique fingerprint.

Cite this