Optimizing Timely Bulk Data Transfers With Hybrid Elastic Cloud Resources

Abstract

The persistent disparity between the slow growth of wide-area network bandwidth and the escalating demand for high-speed bulk data transfer poses a significant challenge for inter-datacenter data transmission. Existing approaches typically rely on optimized algorithms for dedicated networks and often struggle to meet the stringent deadline requirements of modern applications cost-effectively. To overcome these limitations, we propose a novel cloud-accelerated transfer approach that is intrinsically deadline-aware. We present a Profit-driven RelAxation-based TransfEr Scheduling (PRATER) system that leverages cloud proxies and multipath transmission to accelerate bulk data transfer. Specifically, we tackle the problem of profit maximization by jointly optimizing cloud proxy deployment and traffic allocation under transmission deadlines, a problem formally modeled and identified as NP-hard. To efficiently derive near-optimal solutions, we decompose this complex problem into two interconnected subproblems: cost minimization through proxy deployment optimization and revenue maximization via traffic allocation optimization. Our efficient iterative algorithm resolves these subproblems by minimizing cloud proxy and bandwidth costs while maximizing timely data transfer completions, thereby enhancing overall system profitability. Extensive experimental results demonstrate that our approach achieves a significant profit improvement of approximately 2.4×–6.1× compared to state-of-the-art existing algorithms.