William H. Tetzlaff

Disk striping and block replication algorithms for video file servers

In multimedia systems, compressed video files are stored digitally on disks within a video server. Object placement and object replication on disks influence the cost, the availability, and the throughput of a video server. The choice of algorithm, along with the workload, determines the number of disks needed to support a given number of users. There is a need to duplex video objects in order to provide reliable access in the event of disk failures. Duplexing also provides an opportunity to optimize I/O and improve throughput by providing two potential sources for each block. The Individual Permutation Algorithm uses a different permutation of the disks for each movie. This paper studies the performance impact of different means of selecting permutations. The value of dynamic load balancing afforded by duplex is also studied. Part of the motivation of this study was to better understand the wide striping of the IBM Tiger Shark video server. Simulations show that the choice of algorithm leads to maximum practical device utilizations from 63 to 95 percent, resulting in significant differences in system cost.

Position: short object lifetimes require a delete-optimized storage system

Early file systems were designed with the expectation that data would typically be read from disk many times before being deleted; on-disk structures were therefore optimized for reading. As main memory sizes increased, more read requests could be satisfied from data cached in memory, motivating file system designs that optimize write performance. Here, we describe how one might build a storage system that optimizes not only reading and writing, but creation and deletion as well. Efficiency is achieved, in part, by automating deletion based on relative retention values rather than requiring data be deleted explicitly by an application. This approach is well suited to an emerging class of applications that process data at consistently high rates of ingest. This paper explores trade-offs in clustering data by retention value and age and examines the effects of allowing the retention values to change under application control.

Long term resource allocation in video delivery systems

In typical video delivery systems offering programs on-demand, service should be be nearly immediate and continuous. A video server can provide this type of service by reserving sufficient network and server resources for the duration of playout. Scalability and reduced cost can be achieved using a single channel to serve multiple customers waiting for the same program (referred to as batching). Batching is especially useful during high load periods typically occuring during evening prime time hours. Typical channel allocation algorithms use a greedy, allocate-as-needed policy. Variations in system load can cause these algorithms to suffer poor and unpredictable short-term performance, and non-optimal long term performance. In this paper, we develop a set of realistic workloads, identify the limitations of greedy allocation algorithms, and propose a set of rate-based allocation schemes to solve these limitations. The performance of various video delivery systems are simulated and compared. The rate-based policies are shown to be robust for the workloads examined, and are easy to implement.

Using tertiary storage in video-on-demand servers

Video-on-demand is a new entertainment service that will soon be widely available. A small amount of material is very popular, while large amounts of material are viewed less frequently. This skew can be exploited by using a storage hierarchy, storing the less frequently viewed videos in lower-cost tertiary storage. This paper studies the use of tertiary storage for videos. Tertiary storage devices such as optical disks and magnetic tapes can be used to a) deliver data directly to viewers, or b) to stage data to disk for viewing. Analysis of these modes yields guidelines for server design. Examining device characteristics, workload characteristics, and cost, the two modes are compared to each other and to playing from disk. The data placement decision depends on the fraction of time a stream of a video is active. At current costs, videos having an active stream less than a third of the time should reside on tertiary storage. When a tertiary library has a much higher data rate than the video rate, videos should be staged disk for playing. Otherwise, they should be played directly from tertiary store.

Multimedia: an introduction

Multimedia--the combination of text, animated graphics, video, and sound--presents information in a way that is more interesting and easier to grasp than text alone. It has been used for education at all levels, job training, and games and by the entertainment industry. It is becoming more readily available as the price of personal computers and their accessories declines. Multimedia as a human-computer interface was made possible some half-dozen years ago by the rise of affordable digital technology. Previously, multimedia effects were produced by computer-controlled analog devices, like videocassette recorders, projectors, and tape recorders. Digital technologys exponential decline in price and increase in capacity has enabled it to overtake analog technology. The Internet is the breeding ground for multimedia ideas and the delivery vehicle of multimedia objects to a huge audience. This paper reviews the uses of multimedia, the technologies that support it, and the larger architectural and design issues.

Elements of scalable video servers

Large scale multimedia systems combine technologies from audio and video systems, communications systems, and computer systems. In multimedia systems compressed video files are stored digitally on a video server that is typically shared by a large number of users. The two key metrics for a video server are the amount of material that it can store and the number of video streams it can play simultaneously. A video server must be composed of multiple storage units, processing units and network connections in order to store the large amount of digital data and concurrently play the large number of streams. In this paper, we propose a taxonomy of video server organizations, compare the alternative systems and show how some of the systems which have been commercially proposed or built fit into the taxonomy.

Analysis of free-storage algorithms: revisited

Most research in free-storage management has centered around strategies that search a linked list and strategies that partition storage into predetermined sizes. Such algorithms are analyzed in terms of CPU efficiency and storage efficiency. The subject of this study is the free-storage management in the Virtual Machine/System Product (VM/SP) system control program. As a part of this study, simulations were done of established, and proposed, dynamic storage algorithms for the VM/SP operating system. Empirical evidence is given that simplifying statistical assumptions about the distribution of interarrival times and holding times has high predictive ability. Algorithms such as first-fit, modified first-fit, and best-fit are found to be CPU-inefficient. Buddy systems are found to be very fast but suffer from a high degree of internal fragmentation. A form of extended subpooling is shown to be as fast as buddy systems with improved storage efficiency. This algorithm was implemented for VM/SP, and then measured. Results for this algorithm are given for several production VM/SP systems.

A methodology for evaluating storage systems in distributed and hierarchical video servers

Large scale multimedia systems combine technologies from audio and video systems, communications systems and computer systems. Multimedia systems store compressed video files digitally on a server that is shared by a large number of users. A video server is characterized by the number of simultaneous video streams it can support and by the size of the video library. Storage costs are a significant aspect of a video server because of the very large size of the objects. In setting up a video service, there is a choice between having one very large central server, multiple decentralized servers, or a hierarchical server structure. This paper discusses the scaling and cost issues in the server in order to provide information on how to balance server costs against communications costs.<<ETX>>

Policy-Based Autonomic Storage Allocation

The goal of autonomic storage allocation is to achieve management of storage resources, including allocation, performance monitoring, and hotspot elimination, by specifying comparatively high-level goals, rather than by means of low-level manual steps. The process of automation should allow specification of policies as administrator specified constraints under which the resources are managed. This paper describes the system design and implementation experiences from a prototype autonomic storage manager being developed in IBM Research. The prototype is being developed for a storage network that includes a SAN switch, an IBM Enterprise Storage Subsystem, and AIX servers. Our early experience from this prototype implementation is that there are a large number of mundane manual steps in storage management and it is feasible to automate them such that the automation is driven by higher-level goals under policy control. However, to manage heterogeneous storage a standard ontology is needed for specification of goals and how to achieve them.

Block allocation in video servers for availability and throughput

Video servers aimed at the home market must deliver very large files at a low cost. The video files must be shared and reused to contain costs. The nature of videos, however, demand a low jitter (late block delivery) rate. Normal systems tolerate disk queues and deliver, typically, smaller objects in a less predictable manner. This paper explores in a multi disk, stripped, environment whether block placement, interdisk permuation, replication and compression impacts the rate of jitter in a multiuser setting with different assumptions as to the pattern of use. Correspondingly, the number of supportable users for a given level of quality (jitters per hour per user) is addressed. Block allocation is the term used to describe the placement of video blocks on selected disk(s).