John Marberg

Highly available cluster: a case study

The methodology and design of a system that provides highly available data in a cluster is presented. A highly available cluster consists of multiple machines interconnected by a common bus. Data is replicated at a primary and one or more backup machines. Data is accessed at the primary, using a location independent mechanism that ensures data integrity. If the primary copy of the data fails, access is recovered by switching to a backup copy. Switchover is transparent to the application, hence called seamless switchover. The fault model is fail-stop. The entire cluster is resilient to at least single failures. Designating data as highly available is selective in scope, and the overhead of replication and recovery is incurred only by applications that access highly available data. An experimental prototype was implemented using IBM AS/400 machines and a high-speed bus with fiber-optic links.<<ETX>>

PDS Cloud: Long Term Digital Preservation in the Cloud

The emergence of the cloud and advanced object-based storage services provides opportunities to support novel models for long term preservation of digital assets. Among the benefits of this approach is leveraging the clouds inherent scalability and redundancy to dynamically adapt to evolving needs of digital preservation. PDS Cloud is an OAIS-based preservation-aware storage service employing multiple heterogeneous cloud providers. It materializes the logical concept of a preservation information-object into physical cloud storage objects. Preserved information can be interpreted by deploying virtual appliances in the compute cloud provisioned with cloud storage data objects together with their designated rendering software. PDS Cloud has a hierarchical data model supporting independent tenants whose assets are organized in multiple aggregations based on content and value. Continuous changes to data objects, life-cycle activities, virtual appliances and cloud providers are applied in a manner transparent to the client. PDS Cloud is being developed as an infrastructure component of the European Union ENSURE project, where it is used for preservation of medical and financial data.

Towards SIRF: self-contained information retention format

Many organizations are now required to preserve and maintain access to large volumes of digital content for dozens of years. There is a need for preservation systems and processes to support such long-term retention requirements and enable the usability of those digital objects in the distant future, regardless of changes in technologies and designated communities. A key component in such preservation systems is the storage subsystem where the digital objects are located for most of their lifecycle. We describe SIRF (Self-contained Information Retention Format) -- a logical storage container format specialized for long term retention. SIRF includes a set of digital preservation objects and a catalog with metadata related to the entire contents of the container as well as to the individual objects and their interrelationship. SIRF is being developed by the Storage Networking Industry Association (SNIA) with the intention of creating a standardized vendor-neutral storage format that will be interpretable by future preservation systems and that will simplify and reduce the costs of digital preservation.

RM: A resource-sharing system for personal computers

With the recent advances in personal computer technology, time-sharing of a processor is no longer a necessity; each user can have his own machine. It is valuable, however, to share resources among the individual machines. This paper discusses a system structure for interactive computing in which personal computers are connected by a local-area network for the purpose of resource sharing, and describes an experimental prototype that is being implemented using the IBM Personal Computer.

Design and implementation of a resource sharing system as an extension to a personal computer operating system

The software and hardware available today for personal computers provides a broad range of support for personal productivity, business applications, research, programming, and other activities. If personal computers are connected in a local area network, they can form a system whose total resources are very great compared to those of each computer. With appropriate system mechanisms, users can share these resources. We describe the design and implementation of a resource sharing system for IBM Personal Computers. The system generalizes the traditional file and device server approach, allowing applications of any kind to be offered as services on the network. The system supports services by maintaining service definitions, queuing requests by priority, creating server processes, loading service programs, and combining services into larger distributed applications. A user may start several independent activities that proceed concurrently. Each activity can span several machines. The system is built upon an existing operating system, PC-DOS, extending the view it provides to users. Multitasking and enhanced memory management are provided. Interprocess communication is supported by a high-level service request protocol. The discussion emphasizes the problems encountered in building the system and the solutions devised.

Storlet Engine for Executing Biomedical Processes Within the Storage System

The increase in large biomedical data objects stored in long term archives that continuously need to be processed and analyzed requires new storage paradigms. We propose expanding the storage system from only storing biomedical data to directly producing value from the data by executing computational modules - storlets - close to where the data is stored. This paper describes the Storlet Engine, an engine to support computations in secure sandboxes within the storage system. We describe its architecture and security model as well as the programming model for storlets. We experimented with several data sets and storlets including de-identification storlet to de-identify sensitive medical records, image transformation storlet to transform images to sustainable formats, and various medical imaging analytics storlets to study pathology images. We also provide a performance study of the Storlet Engine prototype for OpenStack Swift object storage.

A special distributed system in a general programming environment

An approach aimed at running a special class of computationally intensive applications in an affordable heterogeneous computing environment built around existing off-the-shelf hardware is described. The software support combines a general-purpose parallel programming language with a remote procedure call facility. The increase in throughput is a result of the overlap in servers computations. The attractiveness of the system comes mainly from its low cost. A typical application involving the traveling salesman problem is described, and performance issues are discussed.<<ETX>>