Julian Satran

Object storage: the future building block for storage systems

The concept of object storage was introduced in the early 1990s by the research community. Since then it has greatly matured and is now in its early stages of adoption by the industry. Yet, object storage is still not widely accepted. Viewing object store technology as the future building block particularly for large storage systems, our team in IBM Haifa Research Lab has invested substantial efforts in this area. In this position paper we survey the latest developments in the area of object store technology, focusing on standardization, research prototypes, and technology adoption and deployment. A major step has been the approval of the TIO OSD protocol (version I) as an OSD standard in late 2004. We also report on prototyping efforts that are carried out in IBM Haifa Research Lab in building an object store. Our latest prototype is compliant with a large subset of the TIO standard. To facilitate deployment of the new technology and protocol in the community at large, our team also implemented a TIO-compliant OSD (iSCSI) initiator for Linux. The initiator is interoperable with object disks of other vendors. The initiator is available as an open source driver for Linux.

Design of the iSCSI protocol

The iSCSI protocol enables accessing SCSI I/O devices over an IP network. TCP is used as a transport for SCSI I/O commands. We describe the design considerations and decisions in defining the iSCSI protocol: why we use TCP how multiple TCP connections can be used to increase performance and reliability, why we require allegiance of a command to a particular TCP connection, the importance of Direct Data Placement, various levels and complexity of error recovery, security and naming issues.

Parallel file systems for the IBM SP computers

Parallel computer architectures require innovative software solutions to utilize their capabilities. This statement is true for system software no less than for application programs. File system development for the IBM SP product line of computers started with the Vesta research project, which introduced the ideas of parallel access to partitioned files. This technology was then integrated with a conventional Advanced Interactive Executive™ (AIX™) environment to create the IBM AIX Parallel I/O File System product. We describe the design and implementation of Vesta, including user interfaces and enhancements to the control environment needed to run the system. Changes to the basic design that were made as part of the AIX Parallel I/O File System are identified and justified.

Internet Protocol storage area networks

The sheer scale of the storage needs of most organizations makes block storage management an important system administration problem. Application servers, databases, and file systems rely on an efficient underlying block storage system. The storage area network paradigm is fast emerging as a desirable block storage solution, due to its performance, resource-sharing, and capacity-scaling benefits. This paper shows that the ubiquitous Internet Protocol (IP) network is technically well-suited to host a storage area network. The paper presents the storage protocol, management, and security building blocks that are necessary for making IP storage a reality. The paper then discusses performance issues that must be addressed in order to make IP storage area networks competitive with other storage area network technologies.

A two layered approach for securing an object store network

Storage Area Networks (SAN) are based on direct interaction between clients and storage servers. This unmediated access exposes the storage server to network attacks, necessitating a verification, by the server, that the client requests conform with the system protection policy. Solutions today can only enforce access control at the granularity of entire storage servers. This is an outcome of the way storage servers abstract storage: an array of fixed size blocks. Providing access control at the granularity of blocks is infeasible there are too many active blocks in the server of entire servers is used. Object, stores (e.g, the NASD system) on the other hand provide means to address these issues. An object store control unit presents an abstraction of a dynamic collection of objects, each can be seen as a different array of blocks, thus providing the basis for Protection at the object level. In this paper we present a security model for the object store which leverages on existing security infrastructure. We give a simple generic mechanism capable of enforcing an arbitrary access control policy at object granularity. This mechanism is specifically designed to achieve low overhead by minimizing the cost of validating an operation along the critical data path, and lends itself for optimizations such as caching The key idea of the model is to separate the mechanisms for transport security from the one used for access control and to maximize the use standard security protocols when possible We utilize a standard industry protocol for authentication, integrity and privacy on the communication channel (IPSec for IP networks) anti fine a proprietary protocol for authorization on top of the secure communication layer.

High throughput reliable message dissemination

We consider applications that require high rate, reliable message dissemination in a many-to-many environment. Examples of such applications include stock market centers and synchronized server clusters. As network capacity increases, the achievable throughput of messaging applications becomes bounded by processing times rather than communication speed. To reduce processing times we suggest the use of message aggregation. We consider performing message aggregation at either the sender, a message-server, or a network switch. The performance of each of these methods in terms of throughput and delay is analytically evaluated and compared against that of a naive implementation that does not perform message aggregation. We show that in typical real-world messaging applications, performing message aggregation can increase throughput by order of magnitude.We base our results on experiments that have been conducted using various operating systems running on different hardware platforms. Our results indicate that the achievable throughput of messaging applications is determined by the number of packets-per-second, rather than bytes-per-second, a receiver or a transmitter should handle.

Loosely Coupled TCP Acceleration Architecture

We present a novel approach for scalable network acceleration. The architecture uses limited hardware support and preserves protocol processing flexibility, combining the benefits of TCP offload and onload. The architecture is based on decoupling the data movement functions, accelerated by a hardware engine, from complex protocol processing, controlled by an isolated software entity running on a central CPU. These operate in parallel and interact asynchronously. We describe a prototype implementation which achieves multi-gigabit throughput with extremely low CPU utilization

Highly available cluster: a case study

The methodology and design of a system that provides highly available data in a cluster is presented. A highly available cluster consists of multiple machines interconnected by a common bus. Data is replicated at a primary and one or more backup machines. Data is accessed at the primary, using a location independent mechanism that ensures data integrity. If the primary copy of the data fails, access is recovered by switching to a backup copy. Switchover is transparent to the application, hence called seamless switchover. The fault model is fail-stop. The entire cluster is resilient to at least single failures. Designating data as highly available is selective in scope, and the overhead of replication and recovery is incurred only by applications that access highly available data. An experimental prototype was implemented using IBM AS/400 machines and a high-speed bus with fiber-optic links.<<ETX>>

Capability based Secure Access Control to Networked Storage Devices

Today, access control security for storage area networks (zoning and masking) is implemented by mechanisms that are inherently insecure, and are tied to the physical network components. However, what we want to secure is at a higher logical level independent of the transport network; raising security to a logical level simplifies management, provides a more natural fit to a virtualized infrastructure, and enables a finer grained access control. In this paper, we describe the problems with existing access control security solutions, and present our approach which leverages the OSD (Object-based Storage Device) security model to provide a logical, cryptographically secured, in-band access control for todays existing devices. We then show how this model can easily be integrated into existing systems and demonstrate that this in-band security mechanism has negligible performance impact while simplifying management, providing a clean match to compute virtualization and enabling fine grained access control.

Block storage listener for detecting file-level intrusions

An intrusion detection system (IDS) is usually located and operated at the host, where it captures local suspicious events, or at an appliance that listens to the network activity. Providing an online IDS to the storage controller is essential for dealing with compromised hosts or coordinated attacks by multiple hosts. SAN block storage controllers are connected to the world via block-level protocols, such as iSCSI and Fibre Channel. Usually, block-level storage systems do not maintain information specific to the file-system using them. The range of threats that can be handled at the block level is limited. A file system view at the controller, together with the knowledge of which arriving block belongs to which file or inode, will enable the detection of file-level threats. In this paper, we present IDStor, an IDS for block-based storage. IDStor acts as a listener to storage traffic, out of the controllers I/O path, and is therefore attractive for integration into existing SAN-based storage solutions. IDStor maintains a block-to-file mapping that is updated online. Using this mapping, IDStor infers the semantics of file-level commands from the intercepted block-level operations, thereby detecting file-level intrusions by merely observing the block read and write commands passing between the hosts and the controller.