Middleware support for RDMA-based data transfer in cloud computing

Yufei Ren, Tan Li, Dantong Yu, Shudong Jin, Thomas Robertazzi

Research output: Chapter in Book/Report/Conference proceedingConference contribution

7 Scopus citations

Abstract

Providing high-speed data transfer is vital to various data-intensive applications in cloud computing systems. We design a middleware layer of high-speed communication based on Remote Direct Memory Access (RDMA) that serves as the common substrate to accelerate various data transfer applications in cloud computing, such as FTP, HTTP, file copy, sync and remote file I/O. This middleware offers higher end-to-end bandwidth than the traditional TCP-based alternatives, while it hides the heterogeneity of the underlying high-speed architecture. In this paper, we describe the design of this middleware, including resource abstraction, and task synchronization and scheduling. We provide a reference implementation of the file-transfer protocol over this RDMA based middleware. Our experimental results show that it outperforms several TCP-based FTP tools, such as GridFTP, while maintaining very low CPU consumption on a variety of platforms. Furthermore, these results confirm that our middleware achieves near line-speed performance in both LAN and MAN, and scales consistently from 10Gbps Ethernet to 40Gbps Ethernet and InfiniBand environments.

Original languageEnglish (US)
Title of host publicationProceedings of the 2012 IEEE 26th International Parallel and Distributed Processing Symposium Workshops, IPDPSW 2012
Pages1095-1103
Number of pages9
DOIs
StatePublished - 2012
Externally publishedYes
Event2012 IEEE 26th International Parallel and Distributed Processing Symposium Workshops, IPDPSW 2012 - Shanghai, China
Duration: May 21 2012May 25 2012

Publication series

NameProceedings of the 2012 IEEE 26th International Parallel and Distributed Processing Symposium Workshops, IPDPSW 2012

Other

Other2012 IEEE 26th International Parallel and Distributed Processing Symposium Workshops, IPDPSW 2012
CountryChina
CityShanghai
Period5/21/125/25/12

All Science Journal Classification (ASJC) codes

  • Software

Keywords

  • Data Center Networks
  • Middleware
  • Protocol offloading
  • Remote Direct Memory Access

Fingerprint Dive into the research topics of 'Middleware support for RDMA-based data transfer in cloud computing'. Together they form a unique fingerprint.

Cite this