Sandra Johnson Baylor

Parallel access to files in the Vesta file system

The Vesta parallel file system is intended to solve the I/O problems of massively parallel multicomputers executing numerically intensive scientific applications. It provides parallel access from the applications to files distributed across multiple storage nodes in the multicomputer, thereby exposing an opportunity for high-bandwidth data transfer across the multicomputers low-latency network. The Vesta interface provides a user-defined parallel view of file data, which gives users some control over the layout of data. This is useful for tailoring data layout to much common access patterns. The interface also allows user-defined partitioning and repartitioning of files without moving data among storage nodes. Libraries with higher-level interfaces that hide the layout details, while exploiting the power of parallel access, may be implemented above the basic interface. It is shown how collective I/O operations can be implemented, and six parallel access modes to Vesta files are defined. Each mode has unique characteristics in terms of how the processes share the file and how their accesses are interleaved. The combination of user-defined file partitioning and the six access modes gives users very versatile parallel file access.

Parallel file systems for the IBM SP computers

Parallel computer architectures require innovative software solutions to utilize their capabilities. This statement is true for system software no less than for application programs. File system development for the IBM SP product line of computers started with the Vesta research project, which introduced the ideas of parallel access to partitioned files. This technology was then integrated with a conventional Advanced Interactive Executive™ (AIX™) environment to create the IBM AIX Parallel I/O File System product. We describe the design and implementation of Vesta, including user interfaces and enhancements to the control environment needed to run the system. Changes to the basic design that were made as part of the AIX Parallel I/O File System are identified and justified.

Overview of the Vesta parallel file system

The Vesta parallel file system provides parallel access from compute nodes to files distributed across I/O nodes in a massively parallel computer. Vesta is intended to solve the I/O problems of massively parallel computers executing numerically intensive scientific applications. Vesta has three interesting characteristics: First, it provides a user defined parallel view of file data, and allows user defined partitioning and repartitioning of files without moving data among I/O nodes. The parallel file access semantics of Vesta directly support the operations required by parallel language I/O libraries. Second, Vesta is scalable to a very large number (many hundreds) of I/O and compute nodes and does not contain any sequential bottlenecks in the data-access path. Third, it provides user-directed checkpointing of files during continuing program execution with very little processing overhead.

Parallel I/O Workload Characteristics Using Vesta

To develop optimal parallel I/O subsystems, one must have a thorough understanding of the workload characteristics of parallel I/O and its exploitation of the associated parallel file system. Presented are the results of a study conducted to analyze the parallel I/O workloads of several applications on a parallel processor using the Vesta parallel file system. Traces of the applications are obtained to collect system events, communication events, and parallel I/O events. The traces are then analyzed to determine workload characteristics. The results show I/O request rates on the order of hundreds of requests per second, a large majority of requests are for small amounts of data (less than 1500 bytes), a few requests are for large amounts of data (on the order of megabytes), significant file sharing among processes within a job, and strong temporal, traditional spatial, and interprocess spatial locality.

Performance evaluation of a parallel I/O architecture

Presented are the results of a study conducted to evaluate the performance of parallel I/O on a massively parallel processor (MPP). The network traversal and total processing times are calculated for I/O reads and writes while varying the I/O and non-I/O request rates and the request size. Also studied is the performance impact of I/O and non-I/O traffic on each other. The results show that the system is scalable for I/O loads considered; however, the scalability is limited by I/O node saturation or considerable network contention.

Performance evaluation of a massively parallel I/O subsystem

Presented are the trace-driven simulation results of a study conducted to evaluate the performance of the internal parallel I/O subsystem of the Vulcan MPP architecture. The system sizes evaluated vary from 16 to 512 nodes. The results show that a compute node to I/O node ratio of four is the most cost effective for all system sizes, showing high scalability. Also, processor-to-processor communication effects are negligible for small message sizes and the greater the fraction of I/O reads, the better the I/O performance. Worse case I/O node placement is within 13% of more efficient placement strategies. Introducing parallelism into the internal I/O subsystem improves I/O performance significantly.

Lazy home migration for distributed shared memory systems

In a distributed shared memory system, each memory page is associated with a home node that maintains the directory state for cache lines within that page. Memory access patterns and home node locations have a strong influence on performance, especially if remote communication is costly. Since access patterns are difficult to predict and may change dynamically, it is useful to dynamically migrate home nodes to reduce the amount of remote communication. The paper presents a new and efficient algorithm for migrating home nodes in distributed shared memory systems. Unlike previous page migration algorithms, our algorithm avoids global coordination. Allowing the system to be more responsive to changing workloads. We verify the algorithms correctness with the Mur/spl sigma/ protocol verification tool. We explore several policies for deciding when and where to migrate home nodes. Trace driven simulations of several SPLASH-2 benchmarks show that our home migration algorithm and policies can reduce the amount of remote communication by 50%. The results also emphasize the importance of minimizing the cost of migration.

An evaluation of the memory reference behavior of engineering/scientific applications in parallel systems

This paper presents the results of a study conducted to evaluate the inherent memory reference behavior of several engineering/scientific applications, executing on shared memory, MIN-based, parallel systems. In this study, system sizes of two to 64 processors were evaluated. A trace-driven simulation model was used to obtain dynamic reference characteristics of the code. Included in this code were explicit declarations of shared variables. Our results indicate that a significant amount of explicitly declared shared data is accessed as either readonly by several processors, or read-write by a single processor. Furthermore, lines containing synchronization variables tend to see small ownership times at a processor and are accessed by several processors in the system. We also note that, as expected, relatively more references are to data with smaller ownership times, as the number of processors increase. Finally, the application data set size can have an impact on ownership time, as the number of processors increase.