Deepak R. Kenchammana-Hosekote

IBM intelligent Bricks project: petabytes and beyond

This paper provides an overview of the Intelligent Bricks project in progress at IBM Research. It describes common problems faced by data center operators and proposes a comprehensive solution based on brick architectures. Bricks are hardware building blocks. Because of certain properties, defined here, scalable and reliable systems can be built with collections of identical bricks. An important feature is that brick-based systems must survive the failure of any brick without requiring human intervention, as long as most bricks are operational. This simplifies system management and allows very dense and very scalable systems to be built. A prototype storage server in the form of a 3 × 3 × 3 array of bricks, capable of storing 26 TB, is operational at the IBM Almaden Research Center. It successfully demonstrates the concepts of the Intelligent Bricks architecture. The paper describes this implementation of brick architectures based on newly developed communication and cooling technologies, the software developed, and techniques for building very reliable systems from low-cost bricks, and it discusses the performance and the future of intellegent brick systems.

I/O scheduling for digital continuous media

Abstract. A growing set of applications require access to digital video and audio. In order to provide playback of such continuous media (CM), scheduling strategies for CM data servers (CMS) are necessary. In some domains, particularly defense and industrial process control, the timing requirements of these applications are strict and essential to their correct operation. In this paper we develop a scheduling strategy for multiple access to a CMS such that the timing guarantees are maintained at all times. First, we develop a scheduling strategy for the steady state, i.e., when there are no changes in playback rate or operation. We derive an optimal Batched SCAN (BSCAN) algorithm that requires minimum buffer space to schedule concurrent accesses. The scheduling strategy incorporates two key constraints: (1) data fetches from the storage system are assumed to be in integral multiples of the block size, and (2) playback guarantees are ensured for frame-oriented streams when each frame can span multiple blocks. We discuss modifications to the scheduling strategy to handle compressed data like motion-JPEG and MPEG. Second, we develop techniques to handle dynamic changes brought about by VCR-like operations executed by applications. We define a suite of primitive VCR-like operations that can be executed. We show that an unregulated change in the BSCAN schedule, in response to VCR-like operations, will affect playback guarantees. We develop two general techniques to ensure playback guarantees while responding to VCR-like operations: passive and active accumulation. Using user response time as a metric we show that active accumulation algorithms outperform passive accumulation algorithms. An optimal response-time algorithm in a class of active accumulation strategies is derived. The results presented here are validated by extensive simulation studies.

Reliability of modular mesh-connected intelligent storage brick systems

A key objective of the IBM Intelligent Bricks project is to create a highly reliable system from commodity components. We envision such systems to be architected for a service model called fail-in-place or deferred maintenance. By delaying service actions, possibly for the entire lifetime of the system, management of the system is simplified. This paper examines the hardware reliability and deferred maintenance of intelligent storage brick (ISB) systems assuming a mesh-connected collection of bricks in which each brick includes processing power, memory, networking, and storage. On the basis of Monte Carlo simulations, we quantify the fraction of bricks that become unusable by a distributed data redundancy scheme due to degrading internal bandwidth and loss of external host connectivity. We derive a system hardware reliability expression and predict the length of time ISB systems can operate without replacement of failed bricks. We also show via a Markov analysis the level of fault tolerance that is required by the data redundancy scheme to achieve a goal of less than two data loss events per exabyte-year due to multiple failures.

Retrieval techniques for compressed video streams

A number of applications require audio and video data. Access to such data requires continuous real-time stream flow during playback (recording). This requires scheduling access to storage devices for such data. The high volume of such streams, especially video, makes it necessary to store them using compression schemes like JPEG and MPEG. However, doing so introduces playback data rate variability which in turn leads to poor storage system utilization if playback guarantees are to be maintained. Not all applications require perfect playback, and some tardiness and discontinuity during playback, within limits of acceptable Quality of Service (QoS), is tolerated. In this paper we briefly describe solutions to two problems that arise in scheduling the retrieval of compressed streams from secondary storage. Firstly, we propose QBSCAN, a scheme which reserves I/O bandwidth based on statistical estimates of playback rate in order to improve utilization. This is achieved by incorporating the QoS which comprises the playback rate (in frames per second), the maximum (consecutive) skipped frames, and the mean time between frame skips. Secondly, we develop storage techniques specifically designed for MPEG video, to be used in conjunction with QBSCAN, that allows robust playback. This is a problem because MPEG encoding introduces inter frame dependencies which make it hard to drop frames arbitrarily. Thus, the objectives of QBSCAN are to maximize the storage system utilization, and minimize buffer requirement, while providing the needed QoS despite data rate variability and dependency. Simulation studies are used to validate the efficacy of the proposed techniques.

A multimedia programming toolkit/environment

This paper provides details and implementation experiences of a multimedia programming language and associated toolkits. The language, a data-flow paradigm for multimedia streams, consists of blocks of code that can be connected through their data ports. Continuous media flows through these ports into and out of blocks. The blocks are responsible for the processing of continuous media data. Examples of such processing include capturing, displaying, storing, retrieving and analyzing their contents. The blocks also have parameter ports that specify other pertinent parameters, such as location, and display characteristics such as geometry, etc. The connection topology of blocks is specified using a graphical editor called the Program Development Tool (PDT) and the geometric parameters are specified by using another graphical editor called the User Interface Development Tool (UIDT). Experience with modeling multimedia presentations in our environment and the enhancements provided by the two graphical editors are discussed in detail.