Adaptive TCP Flow Control for Improving Performance of Mobile Cloud Clusters

Abstract

Significant innovations in mobile technologies are enabling mobile users to make real-time actionable decisions based on balancing opportunities and risks in order to take coordinated actions with others in the workplace. This requires a new distributed analytic framework that collects relevant information from internal and external sources, performs real-time distributed analytics, and delivers a critical analysis to any user at any place in a given time frame through the use of mobile devices such as smartphones and tablets.

This work discusses the benefits and challenges of building mobile cloud clusters using recent mobile devices for distributed analytic applications by showing its feasibility with MapReduce framework and also investigates performance issues of Hadoop mobile clusters by conducting extensive experiments using typical Hadoop benchmarks; the newest release of Hadoop software framework with its enhancements is ported to the latest Android-based mobile devices through mobile virtualization technique. In addition, it develops the MapReduce simulator based on the ns-2 network simulator to comprehensively examine the performance and efficiency of mobile cloud clusters in extensive operating environments, which enables it to identify critical performance issues under different cluster (or workload) scales, dynamic node mobility, and various wireless channel conditions.

From the performance analysis this work identifies TCP (Transmission Control Protocol) communication problems resulting from distinct traffic patterns of MapReduce-based Hadoop distributed framework and proposes adaptive TCP flow control algorithms for improving the performance of mobile cloud clusters. The overall computing power of the mobile cluster is no longer significantly bounded by typical processing capabilities of each individual mobile node as mobile devices have been constantly enhanced, but the mobile cluster has limitations on interchanging large amounts of analytical data among mobile devices and monitoring real-time status of cluster nodes through timely state updates, which result in significant delays in the processing time with corresponding performance degradation.

This work proposes an algorithm for cross-layer TCP flow control and dynamic network resource scheduling to avoid frequent overflows of the MAC (Media Access Control) layer transmit queue on the mobile nodes, which interrupt long-lived analytical data streams required for the partition and aggregation workflow of distributed analytic frameworks. It controls TCP's packet transmission based on the queueing level and implements dynamic resource scheduling for incoming and outgoing frames to minimize the queueing delay and stabilize the queueing level. In the evaluation test, the aggregate throughput of peer-to-peer TCP connections was significantly improved without incurring any throughput collapse.

In order to prevent TCP receive buffer overflows on the controller node due to the use of TCP push packets, which involve many interruptions in the reception of the latest status updates and progress reports from its worker nodes, this work also introduces another algorithm for mitigating the adverse effects of the TCP push flows that cause excessive transmissions from fast congestion window growth and frequent RTO (retransmission timeout) underestimation. It moderates the congestion window outburst and skips the RTO estimation using RTT (round-trip time) updates from the TCP push streams when the buffer overflow is detected. In the Hadoop TeraSort benchmark test, there were marked decreases in frequency of receive buffer overflows and TCP packet retransmissions and the overall processing time could be shortened accordingly.