Liudong Zuo

Concurrent Bandwidth Reservation Strategies for Big Data Transfers in High-Performance Networks

Because of the deployment of large-scale experimental and computational scientific applications, big data is being generated on a daily basis. Such large volumes of data usually need to be transferred from the data generating center to remotely located scientific sites for collaborative data analysis in a timely manner. Bandwidth reservation along paths provisioned by dedicated high-performance networks (HPNs) has proved to be a fast, reliable, and predictable way to satisfy the transfer requirements of massive time-sensitive data. In this paper, we study the problem of scheduling multiple bandwidth reservation requests (BRRs) concurrently within an HPN while achieving their best average transfer performance. Two common data transfer performance parameters are considered: the Earliest Completion Time (ECT) and the Shortest Duration (SD). Since not all BRRs in one batch can oftentimes be successfully scheduled, the problem of scheduling all BRRs in one batch while achieving their best average ECT and SD are converted into the problem of scheduling as many BRRs as possible while achieving the average ECT and SD of scheduled BRRs, respectively. The aforementioned two problems are proved to be NP-complete problems. Two fast and efficient heuristic algorithms with polynomial-time complexity are proposed. Extensive simulation experiments are conducted to compare their performance with two proposed naive algorithms in various performance metrics. Performance superiority of these two fast and efficient algorithms is verified.

Improved Scheduling Algorithms for Single-Path Multiple Bandwidth Reservation Requests

Colossal amounts of data are being generated in extreme-scale e-Sciences with the advent of new computation tools and experimental infrastructures. Such extremely large and complex data sets normally need to be transferred remotely for data storage and analysis. Reserving bandwidth as needed along selected paths in high-performance networks (HPNs) has proved to be an effective way to satisfy the high-demanding requirements of such data transfer. The most common data transfer requirement from users is the data transfer deadline. However, users oftentimes want to achieve other data transfer performance parameters, such as the earliest completion time (ECT) and the shortest duration (SD). For the bandwidth reservation service provider, all bandwidth reservation requests (BRRs) in one batch should be scheduled for high scheduling efficiency and system throughput. In this paper, we study the problem of scheduling all BRRs in one batch while achieving their best average transfer performance on one reservation path in an HPN. Two data transfer performance parameters, ECT and SD, are specifically considered. Because of the limited bandwidth resources of the reservation path, the problems of scheduling all BRRs in one batch on one reservation path while achieving their best average ECT and SD are converted into the problems of scheduling as many BRRs as possible while achieving the average ECT and SD of scheduled BRRs, respectively. We prove these two converted problems as NP-complete problems, and improve two existing heuristic algorithms proposed previously for similar problems. Extensive simulation experiments show the superior scheduling performance of these improved algorithms in terms of several performance metrics.

Fault-tolerant bandwidth reservation strategies for data transfers in high-performance networks

Many next-generation e-science applications require fast and reliable transfer of large volumes of data with guaranteed performance, which is typically enabled by the bandwidth reservation service in high-performance networks. One prominent issue in such network environments with large footprints is that node and link failures are inevitable, hence potentially degrading the quality of data transfer. We consider two generic types of bandwidth reservation requests (BRRs) concerning data transfer reliability: (i) to achieve the highest data transfer reliability under a given data transfer deadline, and (ii) to achieve the earliest data transfer completion time while satisfying a given data transfer reliability requirement. We propose two periodic bandwidth reservation algorithms with rigorous optimality proofs to optimize the scheduling of individual BRRs within BRR batches. The efficacy of the proposed algorithms is illustrated through extensive simulations in comparison with scheduling algorithms widely adopted in production networks in terms of various performance metrics.

Toward Flexible and Fast Routing Strategies for Dynamic Network Provisioning

Reserving bandwidth as needed in high-performance networks makes the fast and reliable data transfer with guaranteed performance possible in large-scale collaborative e-science. Besides the notification of acceptance or rejection for a particular reservation request, users normally want to know the earliest possible finish time or the minimum total transfer duration for the data transfer. Several routing algorithms have been proposed to achieve such desired goals given the data size, the data available time, and the deadline to finish the data transfer. Instead of directly processing the bandwidth reservation request (BRR) from users, our approach analyses various parameters to strategically narrow down the solution search space for fast system response. Adapted from some previous works, two algorithms are proposed to compute the reservation options with the earliest completion time (ECT) and with the shortest duration (SD) for multiple BRRs accumulated during a certain period. Extensive simulation results demonstrate the superiority of the proposed algorithms in terms of reduced execution time and improved success ratio of BRRs in comparison with existing scheduling algorithms.

Bandwidth provision strategies for reliable data movements in dedicated networks

Sheer volumes of data are being generated in extreme-scale distributed scientific applications, and need to be transferred remotely in fast, predictable and reliable way for data storage and analysis purpose. Reserving bandwidth along selected paths in high-performance networks (HPNs) has proved to be an effective way to satisfy the high-demanding performance requirements of such data transfer. However, node and link failures within the HPNs potentially degrade the quality of data transfer. In this paper, we focus on the scheduling of two generic types of bandwidth reservation requests concerning data transfer reliability: (i) to achieve the highest data transfer reliability under a given data transfer deadline, and (ii) to achieve the earliest data transfer completion time while satisfying a given data transfer reliability requirement. Poisson distribution is used to model the node and failures within the HPNs, and two periodic bandwidth reservation algorithms with rigorous optimality proofs are proposed.

Optimizing Trade-Off Between Cost and Performance of Data Transfers Using Bandwidth Reservation in Dedicated Networks

Because of the solid performance of providing quality of service for various applications for decades, bandwidth reservation has been increasingly used in recent years for large amounts of data transfer to achieve guaranteed performance. However, effective scheduling strategy to achieve the trade-off between data transfer cost and data transfer performance still remains to be investigated. In this paper, we focus on the trade-off between cost and the most common performance parameter, i.e., completion time, of data transfers using bandwidth reservation in dedicated networks. We consider the scheduling of two types of bandwidth reservation requests regarding such trade-off: (1) to achieve the minimum data transfer cost given the data transfer deadline, and (2) to achieve the earliest data transfer completion time given the maximum data transfer cost. We propose two bandwidth reservation algorithms with rigorous optimality proofs to optimize the scheduling of these two types of bandwidth reservation requests. We then compare the proposed algorithms with two scheduling algorithms originating from one widely used scheduling algorithm in production networks, and the efficacy of the proposed optimal algorithms is verified through extensive simulations.

Flexible Bandwidth Scheduling for Streaming Data Movement Over Dedicated Networks

A wide range of scientific disciplines are generating large amounts of data at a high speed, which must be transferred to remote sites for real-time processing. Reserving bandwidths over dedicated channels in high-performance networks (HPNs) within a specified time interval has proved to be an effective solution to such high-demanding data transfer. Given a bandwidth reservation request, if the desired bandwidth within the specified time interval cannot be satisfied, most of the existing scheduling algorithms simply reject the request, which would immediately terminate the application. One reasonable approach to mitigate this issue is to provide an alternative bandwidth reservation option to schedule the desired bandwidth within the time interval closest to the specified one. We propose a flexible bandwidth reservation algorithm that considers both the best and alternative bandwidth reservation options for a given request. Extensive simulations are conducted to show the superior performance of the proposed scheduling algorithm compared with a heuristic approach adapted from existing scheduling algorithms.