Tuesday, October 15, 2013

Performance test results for transport protocols for moving big data

As data grows the need for transporting the data grows as well. This data may need to transferred over long distances for safe keeping, warehousing or other purposes. It is important to use appropriate transfer protocols to optimize data transfer speed. A recent paper by Se-young Yu, Nevil Brownlee, and Aniket Mahanti; all from Department of Computer Science, University of Auckland, New Zealand, presented a preliminary comparative analysis of high-speed transfer protocols- GridFTP, FDT, UDT and Tsunami. Their results surprised me a bit but first the results.

Performance graph with no noise
Results with no traffic noise

Performance graph with noise
Throughput of GridFTP with TCP (left), GridFTP with UDP (middle), and FDT (right) using
jumbo frames with increasing RTT and UDP background noise


Here is the test network topology used for the experiments.
test network setup
SUT topology


The results clearly show that using GridFTP with TCP or FDT gives higher throughput over a 10 Gb/s network link. Even with signi cant level of congestion induced by background noise, FDT was stable enough to perform data transfer. On the other hand, UDP-based protocols did not perform well in most tests. Tsunami is not recommended due to its instability when using jumbo frames or when faced with background noise. Having some background in VoIP (Voice over IP) field where UDP is the king I find it interesting that UDP is not much of a bet in the big data transfer domain.

Quick diclaimer - Your mileage might vary.

Paper referenced was published at - 
IFIP WG 7.3 Performance 2013
31st International Symposium on Computer Performance, Modeling, Measurements and Evaluation 2013