Saturday, December 13, 2014

Performance testing results for tools used in big data transfers

Many organizations face every day task of moving big data from one deployment to another. There are many options for doing so. Recently I came across a paper that presents results of network performance testing with common commercial tools used for data transfer. The 10 Gb/s network topology used -
Network topology for performance testing
They also conducted the test with background noise thrown in to the mix for a more realistic simulation. Following is the chart how adding more background noise affects the performance of the tools

Following is the abstract from the paper that summarizes the results of the study.
We present performance and fairness analysis of two TCP-
based (GridFTP and FDT) and one UDP-based (UDT)
big data transfer protocols. We perform long-haul performance
experiments using a 10 Gb/s national network, and conduct
fairness tests in our 10 Gb/s local network. Our results
show that GridFTP with jumbo frames provides fast data
transfers. GridFTP is also fair in sharing bandwidth with
competing background TCP flows.

Performance and Fairness Issues in Big Data Transfers by Se-young Yu, Nevil Brownlee, and Aniket Mahanti, University of Auckland, New Zealand