Baptiste Lepers

Traffic management: a holistic approach to memory placement on NUMA systems

NUMA systems are characterized by Non-Uniform Memory Access times, where accessing data in a remote node takes longer than a local access. NUMA hardware has been built since the late 80s, and the operating systems designed for it were optimized for access locality. They co-located memory pages with the threads that accessed them, so as to avoid the cost of remote accesses. Contrary to older systems, modern NUMA hardware has much smaller remote wire delays, and so remote access costs per se are not the main concern for performance, as we discovered in this work. Instead, congestion on memory controllers and interconnects, caused by memory traffic from data-intensive applications, hurts performance a lot more. Because of that, memory placement algorithms must be redesigned to target traffic congestion. This requires an arsenal of techniques that go beyond optimizing locality. In this paper we describe Carrefour, an algorithm that addresses this goal. We implemented Carrefour in Linux and obtained performance improvements of up to 3.6 relative to the default kernel, as well as significant improvements compared to NUMA-aware patchsets available for Linux. Carrefour never hurts performance by more than 4% when memory placement cannot be improved. We present the design of Carrefour, the challenges of implementing it on modern hardware, and draw insights about hardware support that would help optimize system software on future NUMA systems.

Challenges of memory management on modern NUMA systems

Modern server-class systems are typically built as several multicore chips put together in a single system. Each chip has a local DRAM (dynamic random-access memory) module; together they are referred to as a node. Nodes are connected via a high-speed interconnect, and the system is fully coherent. This means that, transparently to the programmer, a core can issue requests to its node’s local memory as well as to the memories of other nodes. The key distinction is that remote requests will take longer, because they are subject to longer wire delays and may have to jump several hops as they traverse the interconnect. The latency of memory-access times is hence non-uniform, because it depends on where the request originates and where it is destined to go. Such systems are referred to as NUMA (non-uniform memory access).

Towards Proving Optimistic Multicore Schedulers

Operating systems have been shown to waste machine resources by leaving cores idle while work is ready to be scheduled. This results in suboptimal performance for user applications, and wasted power. Recent progress in formal verification methods have led to operating systems being proven safe, but operating systems have yet to be proven free of performance bottlenecks. In this paper we instigate the first effort in proving performance properties of operating systems by designing a multicore scheduler that is proven to be work-conserving.

Drowsy-DC, datacenter power management inspired by smartphones: poster

A lot of research work is put into reducing the power consumption of datacenters. Some works seek to improve the power efficiency of one server, but as of now the ideal state of power proportionality is only approached with a higher workload. Meanwhile, modern datacenters make use of the virtualization: the actual workload is materialized by virtual machines (VM). Thus, the problem shifts to consolidating VMs in order to increase the workload of a server. Roughly described, VM consolidation systems try to place as many VMs as possible, on as few servers as possible. Then, a server either gets a higher workload, bringing it closer to a state of energy proportionality; or hosts no VM and can be brought to sleep. However, industrial solutions are not very efficient, and in practice the number of VMs per server is increased by only 10%, with an effective workload rarely over 50% [3].