Jimi Xenidis

Virtualization for high-performance computing

The specific demands of high-performance computing (HPC) often mismatch the assumptions and algorithms provided by legacy operating systems (OS) for common workload mixes. While feature- and application-rich OSes allow for flexible and low-cost hardware configurations, rapid development, and flexible testing and debugging, the mismatch comes at the cost of --- oftentimes significant --- performance degradation for HPC applications.The ubiquitous availability of virtualization support in all relevant hardware architectures enables new programming and execution models for HPC applications without loosing the comfort and support of existing OS and application environments. In this paper we discuss the trends, motivations, and issues in hardware virtualization with emphasis on their value in HPC environments.

K42: building a complete operating system

K42 is one of the few recent research projects that is examining operating system design structure issues in the context of new whole-system design. K42 is open source and was designed from the ground up to perform well and to be scalable, customizable, and maintainable. The project was begun in 1996 by a team at IBM Research. Over the last nine years there has been a development effort on K42 from between six to twenty researchers and developers across IBM, collaborating universities, and national laboratories. K42 supports the Linux API and ABI, and is able to run unmodified Linux applications and libraries. The approach we took in K42 to achieve scalability and customizability has been successful.The project has produced positive research results, has resulted in contributions to Linux and the Xen hypervisor on Power, and continues to be a rich platform for exploring system software technology. Today, K42, is one of the key exploratory platforms in the DOEs FAST-OS program, is being used as a prototyping vehicle in IBMs PERCS project, and is being used by universities and national labs for exploratory research. In this paper, we provide insight into building an entire system by discussing the motivation and history of K42, describing its fundamental technologies, and presenting an overview of the research directions we have been pursuing.

Experience distributing objects in an SMMP OS

Designing and implementing system software so that it scales well on shared-memory multiprocessors (SMMPs) has proven to be surprisingly challenging. To improve scalability, most designers to date have focused on concurrency by iteratively eliminating the need for locks and reducing lock contention. However, our experience indicates that locality is just as, if not more, important and that focusing on locality ultimately leads to a more scalable system. In this paper, we describe a methodology and a framework for constructing system software structured for locality, exploiting techniques similar to those used in distributed systems. Specifically, we found two techniques to be effective in improving scalability of SMMP operating systems: (i) an object-oriented structure that minimizes sharing by providing a natural mapping from independent requests to independent code paths and data structures, and (ii) the selective partitioning, distribution, and replication of object implementations in order to improve locality. We describe concrete examples of distributed objects and our experience implementing them. We demonstrate that the distributed implementations improve the scalability of operating-system-intensive parallel workloads.

Experience with K42, an open-source, Linux-compatible, scalable operating-system kernel

K42 is an open-source, Linux-compatible, scalable operating-system kernel that can be used for rapid prototyping of operating-system policies and mechanisms. This paper reviews the structure and design philosophy of K42 and discusses our experiences in developing and using K42 in the open-source environment.

A Case for High Availability in a Virtualized Environment (HAVEN)

The cost and operational complexity of traditional high availability solutions has limited their widespread adoption. Virtualization allows availability properties to be associated with the system architecture, rather than depending on the intrinsic reliability of components. This paper introduces an extensible grammar that classifies the states and transitions of virtual machine images. From this grammar, rules for recovery and High Availability can be created which define how virtualization allows for simplified fault tolerance, making HAVENs accessible to the mainstream user.

Performance evaluation of interthread communicationmechanisms on multicore/multithreaded architectures

The three major solutions for increasing the nominal performance of a CPU are: multiplying the number of cores per socket, expanding the embedded cache memories and use multi-threading to reduce the impact of the deep memory hierarchy. Systems with tens or hundreds of hardware threads, all sharing a cache coherent UMA or NUMA memory space, are today the de-facto standard. While these solutions can easily provide benefits in a multi-program environment, they require recoding of applications to leverage the available parallelism. Threads must synchronize and exchange data, and the overall performance is heavily in influenced by the overhead added by these mechanisms, especially as developers try to exploit finer grain parallelism to be able to use all available resources.

IBM's PowerEN Developer Cloud: Fertile ground for academic research

IBMs newest technology, the Power Edge of Network (PowerEN) processor, merges network and server attributes to create a new class of wire-speed processor. PowerEN is a hybrid computer that employs: massive multithreading capabilities, integrated I/O and unique special-purpose accelerators for compression, cryptography, pattern matching, XML and Network processing. As a novel architecture, the PowerEN processor offers fertile ground for research. It can facilitate the development of faster applications in many fields of computer science such as Networking, Cryptography, Virtualization, Bioinformatics, SOA, and Systems. IBM encourages academic research and has set up the “PowerEN Developer Cloud” infrastructure to allow it. Israeli universities are of the first to perform research on this global infrastructure.

Specialized execution environments

Virtualization has become popular (again) as a means of consolidating multiple operating systems (OSes) onto a smaller set of hardware resources. The roles of OSes in such environments have changed. Whereas normally an OS provides balance between the demands of application and hardware support, in the world of virtualization it can be beneficial to split these roles. One OS may support a particular application set and use other OSes to interact with physical hardware. The hypervisor, or virtualization layer, provides communication facilities for the inter-OS communication needed to support such a deployment model.

A study in rapid prototyping: Leveraging software and hardware simulation tools in the bringup of system-on-a-chip based platforms

Traditional use of software and hardware simulators and emulators has been in efforts for chip level analysis and verification. However, prototyping and bringup requirements often demands system or platform level integration and analysis requiring new uses of these traditional pre-silicon methods along with novel interpretations of existing hardware to prototype some functions matching behaviors of future systems. In order to demonstrate the versatility and breadth of the pre-silicon environments in our systems lab, ranging from functional instruction set software simulators to Field Programmable Gate Array (FPGA) chip logic implementations to integrated systems of existing hardware built to mimic key functional aspects of the future platforms, we present our experiences with platform level verification, analysis and early software development/enablement for an I/O attached network appliance system. More specifically, we show how simulation tools along with these early prototype systems were used to do chip level verification, early software development and even system level software testing for a System on a Chip processor attached as an I/O accelerator via Peripheral Component Interconnect Express (PCI Express) to a host system. Our experiences demonstrate that leveraging the full range of pre-silicon environment capabilities results in full system level integrated software test for a I/O attached platform prior to the availability of fully functional ASICs.