Arka Aloke Bhattacharya

Hierarchical scheduling for diverse datacenter workloads

There has been a recent industrial effort to develop multi-resource hierarchical schedulers. However, the existing implementations have some shortcomings in that they might leave resources unallocated or starve certain jobs. This is because the multi-resource setting introduces new challenges for hierarchical scheduling policies. We provide an algorithm, which we implement in Hadoop, that generalizes the most commonly used multi-resource scheduler, DRF [1], to support hierarchies. Our evaluation shows that our proposed algorithm, H-DRF, avoids the starvation and resource inefficiencies of the existing open-source schedulers and outperforms slot scheduling.

The need for speed and stability in data center power capping

Data centers can lower costs significantly by pro-visioning expensive electrical equipment (such as UPS, diesel generators, and cooling capacity) for the actual peak power consumption rather than server nameplate power ratings. However, it is possible that this under-provisioned power level is exceeded due to software behaviors on rare occasions and could cause the entire data center infrastructure to breach the safety limits. A mechanism to cap servers to stay within the provisioned budget is needed, and processor frequency scaling based power capping methods are readily available for this purpose. We show that existing methods, when applied across a large number of servers, are not fast enough to operate correctly under rapid power dynamics observed in data centers. We also show that existing methods when applied to an open system (where demand is independent of service rate) can cause cascading failures in the software service hosted, causing the service performance to fall uncontrollably even when power capping is applied for only a small reduction in power consumption. We discuss the causes for both these short-comings and point out techniques that can yield a safe, fast, and stable power capping solution. Our techniques use admission control to limit power consumption and ensure stability, resulting in orders of magnitude improvement in performance. We also discuss why admission control cannot replace existing power capping methods but must be combined with them.

Brick: Towards a Unified Metadata Schema For Buildings

Commercial buildings have long since been a primary target for applications from a number of areas: from cyber-physical systems to building energy use to improved human interactions in built environments. While technological advances have been made in these areas, such solutions rarely experience widespread adoption due to the lack of a common descriptive schema which would reduce the now-prohibitive cost of porting these applications and systems to different buildings. Recent attempts have sought to address this issue through data standards and metadata schemes, but fail to capture the set of relationships and entities required by real applications. Building upon these works, this paper describes Brick, a uniform schema for representing metadata in buildings. Our schema defines a concrete ontology for sensors, subsystems and relationships among them, which enables portable applications. We demonstrate the completeness and effectiveness of Brick by using it to represent the entire vendor-specific sensor metadata of six diverse buildings across different campuses, comprising 17,700 data points, and running eight complex unmodified applications on these buildings.

Non-Intrusive Techniques for Establishing Occupancy Related Energy Savings in Commercial Buildings

The design of energy-efficient commercial building Heating Ventilation and Air Conditioning (HVAC) systems has been in the forefront of energy conservation efforts over the past few decades. The HVAC systems traditionally run on a static schedule that does not take occupancy into account, wasting a lot of energy in conditioning empty or partially-occupied spaces. This paper investigates the application of non-intrusive techniques to obtain a rough estimate of occupancy from coarse-grained measurements of the sensors that are commonly available through the building management system. Various per-zone schedules can be developed based on this approximate knowledge of occupancy at the level of individual zones. Our experiments in three large commercial buildings confirm that the proposed techniques can uncover the occupancy pattern of the zones, and schedules that incorporate these occupancy patterns can achieve more than 38% reduction in reheat energy consumption while maintaining indoor thermal comfort.

Short Paper: A Method for Discovering Functional Relationships Between Air Handling Units and Variable-Air-Volume Boxes From Sensor Data

In Building Automation Systems contextual information about sensors is frequently missing or hard-coded in the control code. Retrieving this data is time consuming and error-prone, but necessary to write any type of control application. Automating metadata acquisition is a new and active area of research. Methods to infer metadata from sensor labels or from recorded data have been previously proposed. However, these methods are ineffective in uncovering the association between HVAC components. In fact, measured variables (pressures, temperatures, flows, valve positions) have slow and attenuated responses to changes in input variables, thus impairing the efficacy of correlation methods. In addition, sensor readings are frequently constrained between physical limits and kept around setpoints by nested control loops. For this reason, pure statistical methods fail to capture the differences between sensor streams and are unable to classify them. In this article, we propose a new method for discovering functional relationships between Air Handling Units and Variable-Air-Volume Boxes from sensor data. The method utilizes perturbations of subsystem variables, while guaranteeing that the building zones remain within comfort boundaries. When applied to an existing building, our proposed method reveals correct associations in ~80% of the cases, and outperforms other methods.

Human tracking with an infrared camera using a curve matching framework

The Kalman filter (KF) has been improved for a mobile robot to human tracking. The proposed algorithm combines a curve matching framework and KF to enhance prediction accuracy of target tracking. Compared to other methods using normal KF, the Curve Matched Kalman Filter (CMKF) method predicts the next movement of the human by taking into account not only his present motion characteristics, but also the previous history of target behavior patterns-the CMKF provides an algorithm that acquires the motion characteristics of a particular human and provides a computationally inexpensive framework of human-tracking system. The proposed method demonstrates an improved target tracking using a heuristic weighted mean of two methods, i.e., the curve matching framework and KF prediction. We have conducted the experimental test in an indoor environment using an infrared camera mounted on a mobile robot. Experimental results validate that the proposed CMKF increases prediction accuracy by more than 30% compared to normal KF when the characteristic patterns of target motion are repeated in the target trajectory.

Toward tool support for interactive synthesis

Syntax-guided synthesis searches for an implementation of a given specification by exploring large spaces of candidate programs. Sketches reduce these search spaces, making synthesis more tractable, by predefining the structure of the desired implementation. Typically, this structure is obtained through human insight---this paper introduces a method for interactive, tool-supported discovery of such structure. The key idea is to decompose the specification into subcomputations such that the decomposition dictates the sketch. We rely on a readily obtainable specification that is nothing more than a finite set of sample input-output pairs or execution traces of the desired program. We introduce two complementary decomposition operators and demonstrate them on case studies. We find that our interactive methodology to discover structure extends the reach of computer-aided programming to problems that cannot be solved with synthesis alone.

Automated metadata transformation for a-priori deployed sensor networks

Sensor network research has facilitated advancements in various domains, such as industrial monitoring, environmental sensing, etc., and research challenges have shifted from creating infrastructure to utilizing it. Extracting meaningful information from sensor data, or control applications using the data, depends on the metadata available to interpret it, whether provided by novel networks or legacy instrumentation. Commercial buildings provide a valuable setting for investigating automated metadata acquisition and augmentation, as they typically comprise large sensor networks, but have limited, obscure metadata that are often meaningful only to the facility managers. Moreover, this primitive metadata is imprecise and varies across vendors and deployments. This state-of-the-art is a fundamental barrier to scaling analytics or intelligent control across the building stock, as even the basic steps involve labor intensive manual efforts by highly trained consultants. Writing building applications on its sensor network remains largely intractable as it involves extensive help from an expert in each buildings design and operation to identify the sensors of interest and create the associated metadata. This process is repeated for each application development in a particular building, and across different buildings. This results in customized building-specific application queries which are not portable or scalable across buildings. We present a synthesis technique that learns how to transform a buildings primitive sensor metadata to a common namespace by using a small number of examples from an expert, such as the building manager. Once the transformation rules are learned for one building, it can be applied across buildings with a similar primitive metadata structure. This common and understandable namespace captures the semantic relationship between sensors, enabling analytics applications that do not require apriori building-specific knowledge. Initial results show that learning the rules to transform 70% of the primitive metadata of two buildings (with completely different metadata structure), comprising 1600 and 2600 sensors, into a common namespace ([1]) took only 21 and 27 examples respectively(Figure 1c). The learned rules were able to transform similar primitive metadata in other buildings as well(Figure 1d), enabling writing of portable applications across these buildings. The techniques developed here may be applicable to the other large legacy sensor networks, such as industrial processing, or urban monitoring.

Portable Queries Using the Brick Schema for Building Applications: Demo Abstract

Sensorized commercial buildings are a rich target for building a new class of applications that improve operational and energy efficiency of building operations that take into account human activities. Such applications, however, rarely experience widespread adoption due to the lack of a common descriptive schema that would enable porting these applications and systems to different buildings. Our demo presents Brick [4], a uniform schema for representing metadata in buildings. Our schema defines a concrete ontology for sensors, subsystems and relationships among them, which enables portable applications. Using a web application, we will demonstrate real buildings that have been mapped to the Brick schema, and show application queries that extracts relevant metadata from these buildings. The attendees would be able to create example buildings and write their own queries.

Unobtrusive power proportionality for HPC frameworks

Building power proportional High Performance Computing (HPC) clusters comprising of servers which are not power-proportional is a well-studied problem, and has the potential to provide large energy savings. However, a large emphasis on maintaining cluster uptime disincentivizes system administrators from deploying prior research techniques that introduce changes to existing software configurations, modify the existing cluster job management framework, change user job submission procedures, or fail in unpredictable ways due to frequent server power cycling[3]. We present Hypnos, a meta-system that tackles the challenge of implementing power proportionality unobtrusively in an HPC cluster with an existing job management framework. Hypnos makes no changes to the existing cluster software or network stack, and uses only the standard interfaces exposed by the existing cluster framework to (a) obtain server state and job information, (b) add/remove servers from the existing frameworks purview, (c) infer the clusters scheduling logic, and (d) handle reliability challenges when servers fail to run jobs, boot up, or race conditions develop between Hypnos and the existing cluster scheduler. We evaluated Hypnos by deploying it on a production HPC cluster running the framework - Torque[4]. Hypnos was able to achieve a 36% reduction in energy consumption (compared to an optimal of 37.5%) while circumventing over 1500 network and software faults over a 21-day deployment.