Edi Shmueli

Backfilling with lookahead to optimize the packing of parallel jobs

The utilization of parallel computers depends on how jobs are packed together: if the jobs are not packed tightly, resources are lost due to fragmentation. The problem is that the goal of high utilization may conflict with goals of fairness or even progress for all jobs. The common solution is to use backfilling, which combines a reservation for the first job in the interest of progress with packing of later jobs to fill in holes and increase utilization. However, backfilling considers the queued jobs one at a time, and thus might miss better packing opportunities. We propose the use of dynamic programming to find the best packing possible given the current composition of the queue, thus maximizing the utilization on every scheduling step. Simulations of this algorithm, called lookahead optimizing scheduler (LOS), using trace files from several IBM SP parallel systems, show that LOS indeed improves utilization, and thereby reduces the mean response time and mean slowdown of all jobs. Moreover, it is actually possible to limit the lookahead depth to about 50 jobs and still achieve essentially the same results. Finally, we experimented with selecting among alternative sets of jobs that achieve the same utilization. Surprising results indicate that choosing the set at the head of the queue does not necessarily guarantee best performance. Instead, repeatedly selecting the set with the maximal overall expected slowdown boosts performance when compared to all other alternatives checked.

Backfilling with Lookahead to Optimize the Performance of Parallel Job Scheduling

The utilization of parallel computers depends on how jobs are packed together: if the jobs are not packed tightly, resources are lost due to fragmentation. The problem is that the goal of high utilization may conflict with goals of fairness or even progress for all jobs. The common solution is to use backfilling, which combines a reservation for the first job in the interest of progress with packing of later jobs to fill in holes and increase utilization. However, backfilling considers the queued jobs one at a time, and thus might miss better packing opportunities. We propose the use of dynamic programming to find the best packing possible given the current composition of the queue. Simulation results show that this indeed improves utilization, and thereby reduces the average response time and average slowdown of all jobs.

On Simulation and Design of Parallel-Systems Schedulers: Are We Doing the Right Thing?

It is customary to use open-system trace-driven simulations to evaluate the performance of parallel-system schedulers. As a consequence, all schedulers have evolved to optimize the packing of jobs in the schedule, as a means to improve a number of performance metrics that are conjectured to be correlated with user satisfaction, with the premise that this will result in a higher productivity in reality. We argue that these simulations suffer from severe limitations that lead to suboptimal scheduler designs and to even dismissing potentially good design alternatives. We propose an alternative simulation methodology called site-level simulation, in which the workload for the evaluation is generated dynamically by user models that interact with the system. We present a novel scheduler called CREASY that exploits knowledge on user behavior to directly improve user satisfaction and compare its performance to the original packing-based EASY scheduler. We show that user productivity improves by up to 50 percent under the user-aware design, while according to the conventional metrics, performance may actually degrade.

Using Site-Level Modeling to Evaluate the Performance of Parallel System Schedulers

The conventional performance evaluation methodology for parallel system schedulers uses an open model to generate the workloads used in simulations. In many cases recorded workload traces are simply played back, assuming that they are reliable representatives of real workloads, and leading to the expectation that the simulation results actually predict the scheduler’s true performance. We show that the lack of feedback in these workloads results in performance prediction errors, which may reach hundreds of percents. We also show that load scaling, as currently performed, further ruins the representativeness of the workload, by generating conditions which cannot exist in a real environment. As an alternative, we suggest a novel sitelevel modeling evaluation methodology, in which we model not only the actions of the scheduler but also the activity of users who generate the workload dynamically. This advances the simulation in a manner that reliably mimics feedback effects found in real sites. In particular, saturation is avoided because the generation of additional work is throttled when the system is overloaded. While our experiments were conducted in the context of parallel scheduling, the idea of site-level simulation is applicable to many other types of systems.

Uncovering the Effect of System Performance on User Behavior from Traces of Parallel Systems

Intuitively, it seems that understanding how the performance of a system affects its users requires research in psychology and the conducting of live experiments. We demonstrate that it is possible to uncover the effect from traces of the system. In particular, we found that the behavior of users of parallel systems is correlated with the response time of their jobs, not the slowdown as was previously assumed. We show that response times affect the decision of users to continue or abort their interactive session with the system, and that this may relate to expectations the users develop. Although this research was conducted in the context of parallel systems, we believe our results are more general and may pertain to other types of systems as well.

A case for conservative workload modeling: Parallel job scheduling with daily cycles of activity

Computer workloads have many attributes. When modeling these workloads it is often difficult to decide which attributes are important, and which can be abstracted away. In many cases, the modeler only includes attributes that are believed to be important, and ignores the rest. We argue, however, that this can lead to impaired workloads and unreliable system evaluations. Using parallel job scheduling as a case study, and daily cycles of activity as the attribute in dispute, we present two schedulers whose simulated performance seems identical without cycles, but then becomes significantly different when daily cycles are included in the workload. We trace this to the ability of one scheduler to prioritize interactive jobs, which leads to implicitly delaying less critical work to nighttime, when it can utilize resources that otherwise would have been left idle. Notably, this was not a design feature of this scheduler, but rather an emergent property that was not anticipated in advance.

Heuristics for Resource Matching in Intel's Compute Farm

In this paper we investigate the issue of resource matching between jobs and machines in Intel’s compute farm. We show that common heuristics such as Best-Fit and Worse-Fit may fail to properly utilize the available resources when applied to either cores or memory in isolation. In an attempt to overcome the problem we propose Mix-Fit, a heuristic which attempts to balance usage between resources. While this indeed usually improves upon the single-resource heuristics, it too fails to be optimal in all cases. As a solution we default to Max-Jobs, a meta-heuristic that employs all the other heuristics as sub-routines, and selects the one which matches the highest number of jobs. Extensive simulations that are based on real workload traces from four different Intel sites demonstrate that Max-Jobs is indeed the most robust heuristic for diverse workloads and system configurations, and provides up to 22 % reduction in the average wait time of jobs.

Highly Accurate Prediction of Jobs Runtime Classes

Separating the short jobs from the long is a known technique to improve scheduling performance. In this paper we describe a method we developed for accurately predicting the runtimes classes of the jobs to enable this separation. Our method uses the fact that the runtimes can be represented as a mixture of overlapping Gaussian distributions, in order to train a CART classifier to provide the prediction. The threshold that separates the short jobs from the long jobs is determined during the evaluation of the classifier to maximize prediction accuracy. Our results indicate overall accuracy of 90% for the data set used in our study, with sensitivity and specificity both above 90%.

Framework for Scalable File System Metadata Crawling and Differencing

Identifying file systems metadata changes such as which files have been added, modified or removed from the file system has many usages. In this paper we present a framework we developed for identifying those changes in increasing speeds. Our framework which is composed of crawling, hashing, and scheduling components, allows to scale the crawl to multiple client workstations that operate in parallel on the same file system in a non-overlapping fashion. Experiments carried using real-world data indicate performance improvement (speedup) of up to 36X using our framework compared to legacy crawling utilities such as Linux’s ‘find’.

Helping Protect Software Distribution with PSWD

The success of new technologies depends on whether proper usage models can be found to support them. In this paper we present such a model for Intels Software Guard Extensions (SGX) -- the leveraging of the technology to provide copy protection to software. We describe the system that we architected, designed and implemented, which transforms, in a fully automated manner, off-the-shelve applications into secured versions that run on top of the enclaves. Our system can be delivered as stand-alone, but also as a layer in existing software copy protection stacks.