Xiaoqi Ren

Hopper: Decentralized Speculation-aware Cluster Scheduling at Scale

As clusters continue to grow in size and complexity, providing scalable and predictable performance is an increasingly important challenge. A crucial roadblock to achieving predictable performance is stragglers, i.e., tasks that take significantly longer than expected to run. At this point, speculative execution has been widely adopted to mitigate the impact of stragglers. However, speculation mechanisms are designed and operated independently of job scheduling when, in fact, scheduling a speculative copy of a task has a direct impact on the resources available for other jobs. In this work, we present Hopper, a job scheduler that is speculation-aware, i.e., that integrates the tradeoffs associated with speculation into job scheduling decisions. We implement both centralized and decentralized prototypes of the Hopper scheduler and show that 50% (66%) improvements over state-of-the-art centralized (decentralized) schedulers and speculation strategies can be achieved through the coordination of scheduling and speculation.

Joint Data Purchasing and Data Placement in a Geo-Distributed Data Market

This paper studies design challenges faced by a geo-distributed cloud data market: which data to purchase (data purchasing) and where to place/replicate the data (data placement). We show that the joint problem of data purchasing and data placement within a cloud data market is NP-hard in general. However, we give a provably optimal algorithm for the case of a data market made up of a single data center, and then generalize the structure from the single data center setting and propose Datum, a near-optimal, polynomial-time algorithm for a geo-distributed data market.

Speculation-aware Cluster Scheduling

Stragglers are a crucial roadblock to achieving predictable performance in todays clusters. Speculation has been widelyadopted in order to mitigate the impact of stragglers; however speculation mechanisms are designed and operated independently of job scheduling when, in fact, scheduling a speculative copy of a task has a direct impact on the resources available for other jobs. In this work, based on a simple model and its analysis, we design Hopper, a job scheduler that is speculation-aware, i.e., that integrates the tradeoffs associated with speculation into job scheduling decisions.

Sampling-based Smoothed Analysis for network algorithm evaluation

Accurate performance evaluation for network algorithms is vital to meet various requirements of different applications, such as QoS, network security, traffic engineering. Although worst-case and average-case analysis are widely used in algorithm evaluation, they are often insufficient due to the lack of practicality. Smoothed Analysis (SA) introduces a new concept of smoothed complexity, remedying the shortcomings in worst-case and average-case analysis. However, recent research towards SA focuses on theoretical evaluation, and thus those methods tend to be too complicated for the analysis of network algorithms. To address the problem, Sampling-based Smoothed Analysis (SSA) for network algorithm evaluation is proposed. SSA extends feasibility for practical performance evaluation and achieves promising experimental results. As examples, two algorithms for typical network problem are evaluated using the proposed SSA framework, and the results explicitly illustrate their significant performance difference in spite of the same theoretical worst-case complexity. Besides evaluation accuracy, SSA also provide more insight for algorithms to facilitate current algorithms improvement and new algorithms design.