Ahmed Amer

Design issues for a shingled write disk system

If the data density of magnetic disks is to continue its current 30–50% annual growth, new recording techniques are required. Among the actively considered options, shingled writing is currently the most attractive one because it is the easiest to implement at the device level. Shingled write recording trades the inconvenience of the inability to update in-place for a much higher data density by a using a different write technique that overlaps the currently written track with the previous track. Random reads are still possible on such devices, but writes must be done largely sequentially. In this paper, we discuss possible changes to disk-based data structures that the adoption of shingled writing will require. We first explore disk structures that are optimized for large sequential writes with little or no sequential writing, even of metadata structures, while providing acceptable read performance. We also examine the usefulness of non-volatile RAM and the benefits of object-based interfaces in the context of shingled disks. Finally, through the analysis of recent device traces, we demonstrate the surprising stability of written device blocks, with general purpose workloads showing that more than 93% of device blocks remain unchanged over a day, and that for more specialized workloads less than 0.5% of a shingled-write disks capacity would be needed to hold randomly updated blocks.

File access prediction with adjustable accuracy

We describe a novel on-line file access predictor, Recent Popularity, capable of rapid adaptation to workload changes while simultaneously predicting more events with greater accuracy than prior efforts. We distinguish the goal of predicting the most events accurately from the goal of offering the most accurate predictions (when declining to Offer a prediction is acceptable). For this purpose we present two distinct measures of accuracy, general and specific accuracy, corresponding to these goals. We describe how our new predictor and an earlier effort, Noah, can trade the number of events predicted for prediction accuracy by modifying simple parameters. When prediction accuracy is strictly more important than the number of predictions offered, trace-based evaluation demonstrates error rates as low as 2%, while offering predictions for more than 60% of all file access events.

Group-based management of distributed file caches

We describe a way to manage distributed file system caches based upon groups of files that are accessed together. We use file access patterns to automatically construct dynamic groupings of files and then manage our cache by fetching groups, rather than single files. We present experimental results, based on trace-driven workloads, demonstrating that grouping improves cache performance. At the file system client, grouping can reduce LRU demand fetches by 50 to 60%. At the server cache hit rate improvements are much more pronounced, but vary widely (20 to over 1200%) depending upon the capacity of intervening caches. Our treatment includes information theoretic results that justify our approach to file grouping.

Data Management and Layout for Shingled Magnetic Recording

Ultimately the performance and success of a shingled write disk (SWD) will be determined by more than the physical hardware realized, but will depend on the data layouts employed, the workloads experienced, and the architecture of the overall system, including the level of interfaces provided by the devices to higher levels of system software. While we discuss several alternative layouts for use with SWD, we also discuss the dramatic implications of observed workloads. Example data access traces demonstrate the surprising stability of written device blocks, with a small fraction requiring multiple updates (the problematic operation for a shingled-write device). Specifically, we discuss how general purpose workloads can show that more than 93% of device blocks can remain unchanged over a day, and that for more specialized workloads less than 0.5% of a shingled-write disks capacity would be needed to hold randomly updated blocks. We further demonstrate how different approaches to data layout can alternatively improve or reduce the performance of a shingled-write device in comparison to the performance of a traditional non-shingled device.

Predictive data grouping: Defining the bounds of energy and latency reduction through predictive data grouping and replication

We demonstrate that predictive grouping is an effective mechanism for reducing disk arm movement, thereby simultaneously reducing energy consumption and data access latency. We further demonstrate that predictive grouping has untapped dramatic potential to further improve access performance and limit energy consumption. Data retrieval latencies are considered a major bottleneck, and with growing volumes of data and increased storage needs it is only growing in significance. Data storage infrastructure is therefore a growing consumer of energy at data-center scales, while the individual disk is already a significant concern for mobile computing (accounting for almost a third of a mobile systems energy demands). While improving responsiveness of storage subsystems and hence reducing latencies in data retrieval is often considered contradictory with efforts to reduce disk energy consumption, we demonstrate that predictive data grouping has the potential to simultaneously work towards both these goals. Predictive data grouping has advantages in its applicability compared to both prior approaches to reducing latencies and to reducing energy usage. For latencies, grouping can be performed opportunistically, thereby avoiding the serious performance penalties that can be incurred with prior applications of access prediction (such as predictive prefetching of data). For energy, we show how predictive grouping can even save energy use for an individual disk that is never idle. Predictive data grouping with effective replication results in a reduction of the overall mechanical movement required to retrieve data. We have built upon our detailed measurements of disk power consumption, and have estimated both the energy expended by a hard disk for its mechanical components, and that needed to move the disk arm. We have further compared, via simulation, three models of predictive grouping of on-disk data, including an optimal arrangement of data that is guaranteed to minimize disk arm movement. These experiments have allowed us to measure the limits of performance improvement achievable with optimal data grouping and replication strategies on a single device, and have further allowed us to demonstrate the potential of such schemes to reduce energy consumption of mechanical components by up to 70%.

Noah: low-cost file access prediction through pairs

Prediction is a powerful tool for performance and usability. It can reduce access latency for I/O systems, and can improve usability for mobile computing systems by automating the file hoarding process. We present recent research that has resulted in a file successor predictor that matches the performance of state-of-the-art context-modeling predictors, while requiring a small fraction of their space requirements. Noah is an online algorithm for predicting successor file access events, effectively identifying strong pairings (successor relationships) among files. Noah can accurately predict approximately 80% of all file access events while tracking only two candidate successors, of which only one requires regular dynamic updates.

Evaluating the Impact of Irrecoverable Read Errors on Disk Array Reliability

We investigate the impact of irrecoverable read errors--also known as bad blocks--on the MTTDL of mirrored disks, RAID level 5 arrays and RAID level 6 arrays. Our study is based on the data collected by Bairavasundaram et al. from a population of 1.53 million disks over a period of 32 months. Our study indicates that irrecoverable read errors can reduce the mean time to data loss (MTTDL) of the three arrays by up to 99 percent, effectively canceling most of the benefits of fast disk repairs. It also shows the benefits of frequent scrubbing scans that map out bad blocks thus preventing future irrecoverable read errors. As an example, once-a-month scrubbing scans were found to improve the MTTDL of the three arrays by at least 300 percent compared to once-a-year scrubbing scans.

Using multiple predictors to improve the accuracy of file access predictions

Existing file access predictors keep track of previous file access patterns and rely on a single heuristic to predict which of the previous successors to the file being currently accessed is the most likely to be accessed next. We present here a novel composite predictor that applies multiple heuristics to this selection problem. As a result, it can make use of specialized heuristics that can make very accurate predictions when access patterns are observed to meet their particular criteria. Simulation results involving a total of seven file access traces indicate that our predictor delivers more correct predictions and less inaccurate guesses than predictors relying on a single heuristic for selecting a successor.

Emulating a Shingled Write Disk

Shingled Magnetic Recording technology is expected to play a major role in the next generation of hard disk drives. But it introduces some unique challenges to system software researchers and prototype hardware is not readily available for the broader research community. It is crucial to work on system software in parallel to hardware manufacturing, to ensure successful and effective adoption of this technology. In this work, we present a novel Shingled Write Disk (SWD) emulator that uses a hard disk utilizing traditional Perpendicular Magnetic Recording (PMR) and emulates a Shingled Write Disk on top of it. We implemented the emulator as a pseudo block device driver and evaluated the performance overhead incurred by employing the emulator. The emulator has a slight overhead which is only measurable during pure sequential reads and writes. The moment disk head movement comes into picture, due to any random access, the emulator overhead becomes so insignificant as to become immeasurable.

Multi‐criteria routing in wireless sensor‐based pervasive environments

Wireless sensor networks are expected to be an integral part of any pervasive computing environment. This implies an ever‐increasing need for efficient energy and resource management of both the sensor nodes, as well as the overall sensor network, in order to meet the expected quality of data and service requirements. There have been numerous studies that have looked at the routing of data in sensor networks with the sole intention of reducing communication power consumption. However, there has been comparatively little prior art in the area of multi‐criteria based routing that exploit both the semantics of queries and the state of sensor nodes to improve network service longevity. In this paper, we look at routing in sensor networks from this perspective and propose an adaptive multi‐criteria routing protocol. Our algorithm offers automated reconfiguration of the routing tree as demanded by variations in the network state to meet application service requirements. Our experimental results show that our ap...