Fred Douglis

The Sprite network operating system

A description is given of Sprite, an experimental network operating system under development at the University of California at Berkeley. It is part of a larger research project, SPUR, for the design and construction of a high-performance multiprocessor workstation with special hardware support of Lisp applications. Sprite implements a set of kernel calls that provide sharing, flexibility, and high performance to networked workstations. The discussion covers: the application interface: the basic kernel structure; management of the file name space and file data, virtual memory; and process migration.<<ETX>>

Beating the I/O bottleneck: a case for log-structured file systems

CPU speeds are improving at a dramatic rate, while disk speeds are not. This technology shift suggests that many engineering and office applications may become so I/O-limited that they cannot benefit from further CPU improvements. This paper discusses several techniques for improving I/O performance, including caches, battery-backed-up caches, and cache logging. We then examine in particular detail an approach called log-structured file systems, where the file systems only representation on disk is in the form of an append-only log. Log-structured file systems potentially provide order-of-magnitude improvements in write performance. When log-structured file systems are combined with arrays of small disks (which provide high bandwidth) and large main-memory file caches (which satisfy most read accesses), we believe it will be possible to achieve 1000-fold improvements in I/O performance over todays systems.

Storage alternatives for mobile computers

Mobile computers such as notebooks, subnotebooks, and palmtops require low weight, low power consumption, and good interactive performance. These requirements impose many challenges on architectures and operating systems. This paper investigates three alternative storage devices for mobile computers: magnetic hard disks, flash memory disk emulators, and flash memory cards. We have used hardware measurements and trace-driven simulation to evaluate each of the alternative storage devices and their related design strategies. Hardware measurements on an HP OmniBook 300 highlight differences in the performance of the three devices as used on the Omnibook, especially the poor performance of version 2.00 of the Microsoft Flash File System [11] when accessing large files. The traces used in our study came from different environments, including mobile computers (Macintosh Power-Books) and desktop computers (running Windows or HPUX), as well as synthetic workloads. Our simulation study shows that flash memory can reduce energy consumption by an order of magnitude, compared to magnetic disk, while providing good read performance and acceptable write performance. These energy savings can translate into a 22% extension of battery life. We also find that the amount of unused memory in a flash memory card has a substantial impact on energy consumption, performance, and endurance: compared to low storage utilizations (40% full), running flash memory near its capacity (95% full) can increase energy consumption by 70-190%, degrade write response time by 30%, and decrease the lifetime of the memory card by up to a third. For flash disks, asynchronous erasure can improve write response time by a factor of 2.5.

The AT&T Internet Difference Engine: Tracking and viewing changes on the web

The AT&T Internet Difference Engine (AIDE) is a system that finds and displays changes to pages on the World Wide Web. The system consists of several components, including a web‐crawler that detects changes, an archive of past versions of pages, a tool called HtmlDiff to highlight changes between versions of a page, and a graphical interface to view the relationship between pages over time. This paper describes AIDE, with an emphasis on the evolution of the system and experiences with it. It also raises some sociological and legal issues.

Flash memory file caching for mobile computers

We examine the impact of using flash memory as a second-level file system buffer cache to reduce power consumption and file access latency on a mobile computer. We use trace-driven simulation to evaluate the impact of what we call a FLASHCACHE. We relate the power consumption and access latency of the storage sub-system to the characteristics of the FLASHCACHE: its size, the unit of erasure, and access costs. We find that a FLASHCACHE can reduce the power consumption of the storage subsystem by 20-40% and can improve overall response time by 30-70% when combined with an aggressive disk management policy. When combined with a more conservative policy, power is reduced from 40-70% while overall response time is improved 20-60%. We also find that durability is not a problem; a 4 MB FLASHCACHE will last 33 years.<<ETX>>

Automatic detection of fragments in dynamically generated web pages

Dividing web pages into fragments has been shown to provide significant benefits for both content generation and caching. In order for a web site to use fragment-based content generation, however, good methods are needed for dividing web pages into fragments. Manual fragmentation of web pages is expensive, error prone, and unscalable. This paper proposes a novel scheme to automatically detect and flag fragments that are cost-effective cache units in web sites serving dynamic content. We consider the fragments to be interesting if they are shared among multiple documents or they have different lifetime or personalization characteristics. Our approach has three unique features. First, we propose a hierarchical and fragment-aware model of the dynamic web pages and a data structure that is compact and effective for fragment detection. Second, we present an efficient algorithm to detect maximal fragments that are shared among multiple documents. Third, we develop a practical algorithm that effectively detects fragments based on their lifetime and personalization characteristics. We evaluate the proposed scheme through a series of experiments, showing the benefits and costs of the algorithms. We also study the impact of adopting the fragments detected by our system on disk space utilization and network bandwidth consumption.

Automatic fragment detection in dynamic Web pages and its impact on caching

Constructing Web pages from fragments has been shown to provide significant benefits for both content generation and caching. In order for a Web site to use fragment-based content generation, however, good methods are needed for fragmenting the Web pages. Manual fragmentation of Web pages is expensive, error prone, and unscalable. This paper proposes a novel scheme to automatically detect and flag fragments that are cost-effective cache units in Web sites serving dynamic content. Our approach analyzes Web pages with respect to their information sharing behavior, personalization characteristics, and change patterns. We identify fragments which are shared among multiple documents or have different lifetime or personalization characteristics. Our approach has three unique features. First, we propose a framework for fragment detection, which includes a hierarchical and fragment-aware model for dynamic Web pages and a compact and effective data structure for fragment detection. Second, we present an efficient algorithm to detect maximal fragments that are shared among multiple documents. Third, we develop a practical algorithm that effectively detects fragments based on their lifetime and personalization characteristics. This paper shows the results when the algorithms are applied to real Web sites. We evaluate the proposed scheme through a series of experiments, showing the benefits and costs of the algorithms. We also study the impact of using the fragments detected by our system on key parameters such as disk space utilization, network bandwidth consumption, and load on the origin servers.

Log-structured file systems

A discussion is presented of several techniques for improving I/O performance, including caches, battery-backed-up caches, and cache logging. The authors then examine in particular detail an approach called log-structured file systems, where the file systems only representation on disk is in the form of an append-only log. Log-structured file systems potentially provide order-of-magnitude improvements in write performance. When log-structured file systems are combined with arrays of small disks (which provide high bandwidth) and large main-memory file caches (which satisfy most read accesses), the authors believe it will be possible to achieve 1000-fold improvements in I/O performance over current systems.<<ETX>>

WebGUIDE: querying and navigating changes in Web repositories

Abstract WebGUIDE is a system for exploring changes to World Wide Web pages and Web structure that supports recursive document comparison: users may explore the differences between pages with respect to two dates. Differences between pages are computed automatically and summarized in a new HTML page, and differences in link structure are shown via graphical representations. WebGUIDE is the combination of two tools that complement one another: the ATT Ciao [6] is a graphical navigator that allows users to query and browse structural connections embedded in a document repository. The union of these tools let users get information on the evolution of pages of interest (both textually and graphically), browse the differences interactively, and dynamically modify the set of pages with which they interact.

Operating system implications of solid-state mobile computers

Trends in storage technology indicate that future notebook, palmtop, and smaller mobile computers will contain battery-backed DRAM as primary storage and direct-mapped hash memory as secondary storage, but no disk. All storage will offer uniform, random access read times through a single-level 64-bit address space. The paper explores the operating system implications of this storage organization. The system should exploit the benefits of having all data reside in fast memory. It can do away with much of the data duplication and related data movement that take place in conventional organizations. The system also needs to hide the limitations of flash memory: write access times higher than read access times, the need to erase memory before rewriting it, and a limited number of write cycles in the lifetime of the device. It needs to limit write traffic to flash memory and avoid writing repeatedly to the same area of flash memory. These steps will increase performance, improve space utilization, and prolong the life of flash memory.<<ETX>>