Jonathan Appavoo

K42: building a complete operating system

K42 is one of the few recent research projects that is examining operating system design structure issues in the context of new whole-system design. K42 is open source and was designed from the ground up to perform well and to be scalable, customizable, and maintainable. The project was begun in 1996 by a team at IBM Research. Over the last nine years there has been a development effort on K42 from between six to twenty researchers and developers across IBM, collaborating universities, and national laboratories. K42 supports the Linux API and ABI, and is able to run unmodified Linux applications and libraries. The approach we took in K42 to achieve scalability and customizability has been successful.The project has produced positive research results, has resulted in contributions to Linux and the Xen hypervisor on Power, and continues to be a rich platform for exploring system software technology. Today, K42, is one of the key exploratory platforms in the DOEs FAST-OS program, is being used as a prototyping vehicle in IBMs PERCS project, and is being used by universities and national labs for exploratory research. In this paper, we provide insight into building an entire system by discussing the motivation and history of K42, describing its fundamental technologies, and presenting an overview of the research directions we have been pursuing.

Libra: a library operating system for a jvm in a virtualized execution environment

If the operating system could be specialized for every application, many applications would run faster. For example, Java virtual machines (JVMs) provide their own threading model and memory protection, so general-purpose operating system implementations of these abstractions are redundant. However, traditional means of transforming existing systems into specialized systems are difficult to adopt because they require replacing the entire operating system. This paper describes Libra, an execution environment specialized for IBMs J9 JVM. Libra does not replace the entire operating system. Instead, Libra and J9 form a single statically-linked image that runs in a hypervisor partition. Libra provides the services necessary to achieve good performance for the Java workloads of interest but relies on an instance of Linux in another hypervisor partition to provide a networking stack, a filesystem, and other services. The expense of remote calls is offset by the fact that Libras services can be customized for a particular workload; for example, on the Nutch search engine, we show that two simple customizations improve application throughput by a factor of 2.7.

Experience distributing objects in an SMMP OS

Designing and implementing system software so that it scales well on shared-memory multiprocessors (SMMPs) has proven to be surprisingly challenging. To improve scalability, most designers to date have focused on concurrency by iteratively eliminating the need for locks and reducing lock contention. However, our experience indicates that locality is just as, if not more, important and that focusing on locality ultimately leads to a more scalable system. In this paper, we describe a methodology and a framework for constructing system software structured for locality, exploiting techniques similar to those used in distributed systems. Specifically, we found two techniques to be effective in improving scalability of SMMP operating systems: (i) an object-oriented structure that minimizes sharing by providing a natural mapping from independent requests to independent code paths and data structures, and (ii) the selective partitioning, distribution, and replication of object implementations in order to improve locality. We describe concrete examples of distributed objects and our experience implementing them. We demonstrate that the distributed implementations improve the scalability of operating-system-intensive parallel workloads.

An infrastructure for multiprocessor run-time adaptation

Runtimeadaptationanddynamicrecon gurationallowasystemtodynamicallyswapinthemostappropriateimple-mentationofitscomp onentsbasedoncurrentorexp ecteduse,andtoselectivelyupgradecomp onentswithbug,secu-rity,orp erformance xeswithoutdown-time.Hot-swappingistheactofreplacinganactivesystemcomp onentsim-plementationwithanewordi erentimplemenwhilemaintainingavailabilityofthecomp onentsfunctionalit.Thispap erdescrib esamechanism tohot-swapsoftwarecom-p onentswithintheK421op eratingsystem.Wehavusedthiscapabilitytoimprovep erformanceunderaarietofconditions.K42isaresearchop eratingsystemforcache-coherentsharedmemorymultipro cessorsdesignedtoachievego o dscalabilityonawiderangeofworkloads.InK42,eachvir-tualandphysicalresource,e.g.,op en le,memoryregion,pagetable,ismanagedbyseparate, ne-granularit,ob jectinstances.Eachob jectinstancemayb ecustomized(di er-entsub-class).Thismo delprovidesthestandardsoftwareengineeringb ene ts,butmoreimp ortantly:1)allowsp er-formancecustomizationonanob ject-by-ob jectbasisand2)allows,onamultipro cessor,indep endentaccessestob edirectedtoindep endentinstancesandpro ceedinparallelthuseliminatingsharedmemoryaccessesandsynchroniza-tion,whicharefundamentalbarrierstomultipro cessorp er-formance.Anop eratingsystemisaparticularlydemandingenvi-ronmentasitneedstosatisfymultipleconcurrenclientswhoseresourcedemandsmayb eato dds.Forexample,multipleclientsmaysimultaneouslyaccessa lewithdif-ferentusagepatterns.Supp ortingmultipro cessorspresentsadditionalchallengesforop eratingsystems,andoftenimple-mentationsthatarerequiredforscalablep erformancehaveworseunipro cessorb ehavior.Toprovideb othunipro cessorandmultipro cessorcomp o-nentswithinasingleob jectmo del,K42sob jectsareim-1http://www.research.ibm.com/K42

Project Kittyhawk: building a global-scale computer: Blue Gene/P as a generic computing platform

This paper describes Project Kittyhawk, an undertaking at IBM Research to explore the construction of a next-generation platform capable of hosting many simultaneous web-scale workloads. We hypothesize that for a large class of web-scale workloads the Blue Gene/P platform is an order of magnitude more efficient to purchase and operate than the commodity clusters in use today. Driven by scientific computing demands the Blue Gene designers pursued an aggressive system-on-a-chip methodology that led to a scalable platform composed of air-cooled racks. Each rack contains more than a thousand independent computers with highspeed interconnects inside and between racks. We postulate that the same demands of efficiency and density apply to web-scale platforms. This project aims to develop the system software to enable Blue Gene/P as a generic platform capable of being used by heterogeneous workloads. We describe our firmware and operating system work to provide Blue Gene/P with generic system software, one of the results of which is the ability to run thousands of heterogeneous Linux instances connected by TCP/IP networks over the high-speed internal interconnects.

Experience with K42, an open-source, Linux-compatible, scalable operating-system kernel

K42 is an open-source, Linux-compatible, scalable operating-system kernel that can be used for rapid prototyping of operating-system policies and mechanisms. This paper reviews the structure and design philosophy of K42 and discusses our experiences in developing and using K42 in the open-source environment.

Providing a cloud network infrastructure on a supercomputer

Supercomputers and clouds both strive to make a large number of computing cores available for computation. More recently, similar objectives such as low-power, manageability at scale, and low cost of ownership are driving a more converged hardware and software. Challenges remain, however, of which one is that current cloud infrastructure does not yield the performance sought by many scientific applications. A source of the performance loss comes from virtualization and virtualization of the network in particular. This paper provides an introduction and analysis of a hybrid supercomputer software infrastructure, which allows direct hardware access to the communication hardware for the necessary components while providing the standard elastic cloud infrastructure for other components.

Kittyhawk: enabling cooperation and competition in a global, shared computational system

Kittyhawk represents our vision for a Web-scale computational resource that can accommodate a significant fraction of the worlds computation needs and enable various parties to compete and cooperate in the provisioning of services on a consolidated platform. In this paper, we explain both the vision and the system architecture that supports it. We demonstrate these ideas by way of a prototype implementation that uses the IBM Blue Genet/P platform. In the Kittyhawk prototype, we define a set of basic services that enable the allocation and interconnection of computing resources. By using examples, we show how higher layers of services can be built by using our basic services and standard open-source software.

Supporting hot-swappable components for system software

Summary form only given. A hot-swappable component is one that can be replaced with a new or different implementation while the system is running and actively using the component. For example, a component of a TCP/IP protocol stack, when hot-swappable, can be replaced (perhaps to handle new denial-of-service attacks or improve performance), without disturbing existing network connections. The capability to swap components offers a number of potential advantages such as: online upgrades for high availability systems, improved performance due to dynamic adaptability and simplified software structures by allowing distinct policy and implementation options to be implemented in separate components (rather than as a single monolithic component) and dynamically swapped as needed. In order to hot-swap a component, it is necessary to (i) instantiate a replacement component; (ii) establish a quiescent state in which the component is temporarily idle; (iii) transfer state from the old component to the new component; (iv) swap the new component for the old; and (v) deallocate the old component.

Towards General-Purpose Neural Network Computing

Machine learning is becoming pervasive, decades of research in neural network computation is now being leveraged to learn patterns in data and perform computations that are difficult to express using standard programming approaches. Recent work has demonstrated that custom hardware accelerators for neural network processing can outperform software implementations in both performance and power consumption. However, there is neither an agreed-upon interface to neural network accelerators nor a consensus on neural network hardware implementations. We present a generic set of software/hardware extensions, X-FILES, that allow for the general-purpose integration of feedforward and feedback neural network computation in applications. The interface is independent of the network type, configuration, and implementation. Using these proposed extensions, we demonstrate and evaluate an example dynamically allocated, multi-context neural network accelerator architecture, DANA. We show that the combination of X-FILES and our hardware prototype, DANA, enables generic support and increased throughput for neural-network-based computation in multi-threaded scenarios.