Janardhan Kulkarni

ProjecToR: Agile Reconfigurable Data Center Interconnect

We explore a novel, free-space optics based approach for building data center interconnects. It uses a digital micromirror device (DMD) and mirror assembly combination as a transmitter and a photodetector on top of the rack as a receiver (Figure 1). Our approach enables all pairs of racks to establish direct links, and we can reconfigure such links (i.e., connect different rack pairs) within 12 us. To carry traffic from a source to a destination rack, transmitters and receivers in our interconnect can be dynamically linked in millions of ways. We develop topology construction and routing methods to exploit this flexibility, including a flow scheduling algorithm that is a constant factor approximation to the offline optimal solution. Experiments with a small prototype point to the feasibility of our approach. Simulations using realistic data center workloads show that, compared to the conventional folded-Clos interconnect, our approach can improve mean flow completion time by 30-95% and reduce cost by 25-40%.

Competitive algorithms from competitive equilibria: non-clairvoyant scheduling under polyhedral constraints

We introduce and study a general scheduling problem that we term the Packing Scheduling problem (PSP). In this problem, jobs can have different arrival times and sizes; a scheduler can process job j at rate xj, subject to arbitrary packing constraints over the set of rates (x) of the outstanding jobs. The PSP framework captures a variety of scheduling problems, including the classical problems of unrelated machines scheduling, broadcast scheduling, and scheduling jobs of different parallelizability. It also captures scheduling constraints arising in diverse modern environments ranging from individual computer architectures to data centers. More concretely, PSP models multidimensional resource requirements and parallelizability, as well as network bandwidth requirements found in data center scheduling. In this paper, we design non-clairvoyant online algorithms for PSP and its special cases -- in this setting, the scheduler is unaware of the sizes of jobs. Our results are summarized as follows. • For minimizing total weighted completion time, we show a O(1)-competitive algorithm. Surprisingly, we achieve this result by applying the well-known Proportional Fairness algorithm (PF) to perform allocations each time instant. Though PF has been extensively studied in the context of maximizing fairness in resource allocation, we present the first analysis in adversarial and general settings for optimizing job latency. Our result is also the first O(1)-competitive algorithm for weighted completion time for several classical non-clairvoyant scheduling problems. •For minimizing total weighted flow time, for any constant ε > 0, any O(n1---ε)-competitive algorithm requires extra speed (resource augmentation) compared to the offline optimum. We show that PF is a O(log n)-speed O(log n)-competitive non-clairvoyant algorithm, where n is the total number of jobs. We further show that there is an instance of PSP for which no non-clairvoyant algorithm can be O(n1---ε)-competitive with o(√log n) speed. •For the classical problem of minimizing total flow time for unrelated machines in the non-clairvoyant setting, we present the first online algorithm which is scalable ((1 + ε)-speed O(1)-competitive for any constant ε > 0). No non-trivial results were known for this setting, and the previous scalable algorithm could handle only related machines. We develop new algorithmic techniques to handle the unrelated machines setting that build on a new single machine scheduling policy. Since unrelated machine scheduling is a special case of PSP, when contrasted with the lower bound for PSP, our result also shows that PSP is significantly harder than perhaps the most general classical scheduling settings. Our results for PSP show that instantaneous fair scheduling algorithms can also be effective tools for minimizing the overall job latency, even when the scheduling decisions are non-clairvoyant and constrained by general packing constraints.

SelfishMigrate: A Scalable Algorithm for Non-clairvoyantly Scheduling Heterogeneous Processors

We consider the classical problem of minimizing the total weighted flow-time for unrelated machines in the online non-clairvoyant setting. In this problem, a set of jobs J arrive over time to be scheduled on a set of M machines. Each job j has processing length pj, weight wj, and is processed at a rate of ℓij when scheduled on machine i. The online scheduler knows the values of wj and ℓij upon arrival of the job, but is not aware of the quantity pj. We present the first online algorithm that is scalable ((1 + ϵ)-speed O(1/2)-competitive for any constant ϵ > 0) for the total weighted flow-time objective. No non-trivial results were known for this setting, except for the most basic case of identical machines. Our result resolves a major open problem in online scheduling theory. Moreover, we also show that no job needs more than a logarithmic number of migrations. We further extend our result and give a scalable algorithm for the objective of minimizing total weighted flow-time plus energy cost for the case of unrelated machines. In this problem, each machine can be sped up by a factor of f4-1 i (P) when consuming power P, wherefi is an arbitrary strictly convex power function. In particular, we get an O(γ2)-competitive algorithm when all power functions are of form sγ. These are the first non-trivial non-clairvoyant results in any setting with heterogeneous machines. The key algorithmic idea is to let jobs migrate selfishly until they converge to an equilibrium. Towards this end, we define a game where each jobs utility which is closely tied to the instantaneous increase in the objective the job is responsible for, and each machine declares a policy that assigns priorities to jobs based on when they migrate to it, and the execution speeds. This has a spirit similar to coordination mechanisms that attempt to achieve near optimum welfare in the presence of selfish agents (jobs). To the best our knowledge, this is the first work that demonstrates the usefulness of ideas from coordination mechanisms and Nash equilibria for designing and analyzing online algorithms.

Coordination mechanisms from (almost) all scheduling policies

We study the price of anarchy of coordination mechanisms for a scheduling problem where each job j has a weight wj, processing time pij, assignment cost hij, and communication delay (or release date) rij, on machine i. Each machine is free to declare its own scheduling policy. Each job is a selfish agent and selects a machine that minimizes its own disutility, which is equal to its weighted completion time plus its assignment cost. The goal is to minimize the total disutility incurred by all the jobs. Our model is general enough to capture scheduling jobs in a distributed environment with heterogeneous machines (or data centers) that are situated across different locations. Our main result is a characterization of scheduling policies that give a small (robust) Price of Anarchy. More precisely, we show that whenever each machine independently declares any scheduling policy that satisfies a certain bounded stretch condition introduced in this paper, the game induced between the jobs has a small Price of Anarchy. Our characterization is powerful enough to test almost all popular scheduling policies. On the technical side, to derive our results, we use a potential function whose derivative leads to an instantaneous smoothness condition, and linear programming and dual fitting. To the best of our knowledge, this is a novel application of these techniques in the context of coordination mechanisms, and we believe these tools will find more applications in analyzing PoA of games. We also extend our results to the lk-norms and l∞ norm (makespan) objectives.

Tight Bounds for Online Vector Scheduling

Modern data centers face a key challenge of effectively serving user requests that arrive online. Such requests are inherently multi-dimensional and characterized by demand vectors over multiple resources such as processor cycles, storage space, and network bandwidth. Typically, different resources require different objectives to be optimized, and Lr norms of loads are among the most popular objectives considered. Furthermore, the server clusters are also often heterogeneous making the scheduling problem more challenging. To address these problems, we consider the online vector scheduling problem in this paper. Introduced by Chekuri and Khanna (SIAM J. of Comp. 2006), vector scheduling is a generalization of classical load balancing, where every job has a vector load instead of a scalar load. The scalar problem, introduced by Graham in 1966, and its many variants (identical and unrelated machines, makespan and Lr-norm optimization, offline and online jobs, etc.) have been extensively studied over the last 50 years. In this paper, we resolve the online complexity of the vector scheduling problem and its important generalizations - for all Lr norms and in both the identical and unrelated machines settings. Our main results are: · For identical machines, we show that the optimal competitive ratio is Θ(log d/ log log d) by giving an online lower bound and an algorithm with an asymptotically matching competitive ratio. The lower bound is technically challenging, and is obtained via an online lower bound for the minimum mono-chromatic clique problem using a novel online coloring game and randomized coding scheme. Our techniques also extend to asymptotically tight upper and lower bounds for general Lr norms. · For unrelated machines, we show that the optimal competitive ratio is Θ(log m + log d) by giving an online lower bound that matches a previously known upper bound. Unlike identical machines, however, extending these results, particularly the upper bound, to general Lr norms requires new ideas. In particular, we use a carefully constructed potential function that balances the individual Lr objectives with the overall (convexified) min-max objective to guide the online algorithm and track the changes in potential to bound the competitive ratio.

Algorithms for Cost-Aware Scheduling

In this paper, we generalize classical machine scheduling problems by introducing a cost involved in processing jobs, which varies as a function of time. Before defining the problems formally and discussing the technical novelty, we present a few technological motivations for introducing this model.

On the Decidability of Model-Checking Information Flow Properties

Current standard security practices do not provide substantial assurance about information flow security: the end-to-end behavior of a computing system. Noninterference is the basic semantical condition used to account for information flow security. In the literature, there are many definitions of noninterference: Non-inference, Separability and so on. Mantel presented a framework of Basic Security Predicates (BSPs) for characterizing the definitions of noninterference in the literature. Model-checking these BSPs for finite state systems was shown to be decidable in [8]. In this paper, we show that verifying these BSPs for the more expressive system model of pushdown systems is undecidable. We also give an example of a simple security property which is undecidable even for finite-state systems: the property is a weak form of non-inference called WNI, which is not expressible in Mantels BSP framework.

Coordination Mechanisms for Selfish Routing over Time on a Tree

While selfish routing has been studied extensively, the problem of designing better coordination mechanisms for routing over time in general graphs has remained an open problem. In this paper, we focus on tree networks (single source multiple destinations) with the goal of minimizing (weighted) average sojourn time of jobs, and provide the first coordination mechanisms with provable price of anarchy for this problem. Interestingly, we achieve our price of anarchy results using simple and strongly local policies such as Shortest Job First and the Smith’s Rule (also called HDF). In particular, for the case of unweighted jobs, we design a coordination mechanism with polylogarithmic price of anarchy. For weighted jobs, on the other hand, we show that price of anarchy is a function of the depth of the tree and accompany this result by a lower bound for the price of anarchy for the Smith Rule policy and other common strongly local scheduling policies.

On allocations with negative externalities

We consider the problem of a monopolist seller who wants to sell some items to a set of buyers. The buyers are strategic, unit-demand, and connected by a social network. Furthermore, the utility of a buyer is a decreasing function of the number of neighbors who do not own the item. In other words, they exhibit negative externalities, deriving utility from being unique in their purchases. In this model, any fixed setting of the price induces a sub-game on the buyers. We show that it is an exact potential game which admits multiple pure Nash Equilibria. A natural problem is to compute those pure Nash equilibria that raise the most and least revenue for the seller. These correspond respectively to the most optimistic and most pessimistic revenues that can be raised. We show that the revenues of both the best and worst equilibria are hard to approximate within sub-polynomial factors. Given this hardness, we consider a relaxed notion of pricing, where the price for the same item can vary within a constant factor for different buyers. We show a 4-approximation to the pessimistic revenue when the prices are relaxed by a factor of 4. The interesting aspect of this algorithm is that it uses a linear programming relaxation that only encodes part of the strategic behavior of the buyers in its constraints, and rounds this relaxation to obtain a starting configuration for performing relaxed Nash dynamics. Finally, for the maximum revenue Nash equilibrium, we show a 2-approximation for bipartite graphs (without price relaxation), and complement this result by showing that the problem is NP-Hard even on trees.

Temporal Fairness of Round Robin: Competitive Analysis for Lk-norms of Flow Time

Fairness is an important criterion considered in scheduling together with overall job latency. Round Robin is a popular scheduling policy that distributes resources to jobs equally at any point in time guaranteeing instantaneous fairness of jobs. In this paper we give the first analysis of Round Robin for the L_2-norm of flow time and show that it is O(1)-speed O(1)-competitive on multiple machines. The L_2-norm is a popular scheduling objective that makes a natural balance between temporal fairness and jobs latency. Prior to our work, Round Robin has not been analyzed for the L_2-norm even in the single machine setting. Our result establishes that Round Robin is fair not only instantaneously but also temporarily.