Jianwei Liao

A New Concurrent Checkpoint Mechanism for Real-Time and Interactive Processes

This paper presents a new concurrent checkpoint mechanism that allows the checkpointed process to run without stopping while checkpoints are set. The checkpointed process can keep running until a memory access request is captured by tracing TLB misses while dumping memory pages (the most time-consuming step when setting a checkpoint). At that time, the checkpointer in the kernel will copy the memory access target page to the designated memory buffer for constructing a consistent state of the checkpointed process, and then resume the memory access. From the experimental results, in contrast to non-concurrent checkpoint techniques, this mechanism can reduce the downtime time of the checkpointed process by 47.4% - 89.8% to ensure concurrency between setting a checkpoint and execution of the checkpointed process. In addition, compared with a traditional concurrent checkpoint system, this mechanism saves more than 2.2% of the checkpoint time and decreases the downtime of the checkpointed process by more than 10%.

Adaptive Replica Synchronization for Distributed File Systems

This paper presents an adaptive replica synchronization mechanism among storage servers (SSs) without the interference from the metadata server (MDS) in a distributed file system. This mechanism employs a chunk list data structure, which holds the information about the relevant chunk replicas and is stored on the associated SSs corresponding to the replicas. Combined with version-based update replay technique, our proposed mechanism can ensure data consistency from the viewpoint of clients and help the SSs to conduct replica synchronization proactively with a per-chunk granularity. In contrast to conventional approaches of replica synchronization that are reliant on the MDS to trigger the synchronization, our proposed mechanism can side step the cumbersome centralized synchronization process to accomplish the same synchronization job. Moreover, a deferred update approach for replica synchronization has been adopted by the newly proposed mechanism for contributing to I/O data rate in write-intensive workloads. The evaluation experiments show that this newly presented mechanism can achieve attractive I/O performance enhancement with less synchronization overhead in specific application contexts.

Prefetching on Storage Servers through Mining Access Patterns on Blocks

Distributed file systems have been widely deployed as back-end storage systems to offer I/O services for parallel/distributed applications that process large amounts of data. Data prefetching in distributed file systems is a well-known optimization technique which can mask both network and disk latency and consequently boost I/O performance. Traditionally, data prefetching is initiated by the client file systems, however, conventional prefetching schemes are not well suited for client machines that have limited memory and computing capacity. To offer an efficient prefetching approach for resource-limited client machines, this paper proposes a novel server-side prefetching mechanism. Specifically, we propose to piggyback client identification to I/O requests so that server side block access history can be put into context. On the server side, we utilize the horizontal visibility graph technique to transform per-client time series of block access sequences into a connected graph for which we employ Tarjans algorithm to disclose cut points in the connected graph. We express these patterns with feature tuples and we propose the X-step pattern matching algorithm to find a matching access pattern (i.e., a feature tuple) for a given block access history. Experimental results indicate that our newly proposed prefetching mechanism can ease client machines and their applications from the process of data prefetching, boosting client performance accordingly, and that it yields an attractive increase in data throughput as well.

Performing Initiative Data Prefetching in Distributed File Systems for Cloud Computing

This paper presents an initiative data prefetching scheme on the storage servers in distributed file systems for cloud computing. In this prefetching technique, the client machines are not substantially involved in the process of data prefetching, but the storage servers can directly prefetch the data after analyzing the history of disk I/O access events, and then send the prefetched data to the relevant client machines proactively. To put this technique to work, the information about client nodes is piggybacked onto the real client I/O requests, and then forwarded to the relevant storage server. Next, two prediction algorithms have been proposed to forecast future block access operations for directing what data should be fetched on storage servers in advance. Finally, the prefetched data can be pushed to the relevant client machine from the storage server. Through a series of evaluation experiments with a collection of application benchmarks, we have demonstrated that our presented initiative prefetching technique can benefit distributed file systems for cloud environments to achieve better I/O performance. In particular, configuration-limited client machines in the cloud are not responsible for predicting I/O access operations, which can definitely contribute to preferable system performance on them.

Toward a General I/O Arbitration Framework for netCDF Based Big Data Processing

On the verge of the convergence between high performance computing HPC and Big Data processing, it has become increasingly prevalent to deploy large-scale data analytics workloads on high-end supercomputers. Such applications often come in the form of complex workflows with various different components, assimilating data from scientific simulations as well as from measurements streamed from sensor networks, such as radars and satellites. For example, as part of the next generation flagship post-K supercomputer project of Japan, RIKEN is investigating the feasibility of a highly accurate weather forecasting system that would provide a real-time outlook for severe guerrilla rainstorms. One of the main performance bottlenecks of this application is the lack of efficient communication among workflow components, which currently takes place over the parallel file system. In this paper, we present an initial study of a direct communication framework designed for complex workflows that eliminates unnecessary file I/O among components. Specifically, we propose an I/O arbitrator layer that provides direct parallel data transfer among job components that rely on the netCDF interface for performing I/O operations, with only minimal modifications to application code. We present the design and an early evaluation of the framework on the K Computer using upi?źto 4800 nodes running RIKENs experimental weather forecasting workflow as a case study.

Adaptive Wear-Leveling in Flash-Based Memory

The paper presents an adaptive wear-leveling scheme based on several wear-thresholds in different periods. The basic idea behind this scheme is that blocks can have different wear-out speeds and the wear-leveling mechanism does not conduct data migration until the erasure counts of some hot blocks hit a threshold. Through a series of emulation experiments based on several realistic disk traces, we show that the proposed wear-leveling mechanism can reduce total erasure counts and yield uniform erasure counts among all blocks at the late lifetime of the storage devices. As a result, not only can the performance of storage systems be advanced, the lifespan of the flash-based memory can also be extended to a certain degree.

Selecting Points of Interest in Traces Using Patterns of Events

Over the past few years, the architecture of supercomputing platforms has evolved towards more complexity: multicore processors attached to multiple memory banks are now combined with accelerators. Exploiting such architecture often requires to mix programming models (MPI + CUDA for instance). As a result, understanding the performance of an application has become tedious. The use of performance analysis tools, such as tracing tools, now becomes unavoidable to optimize a parallel application. However, analyzing a trace file composed of millions of events requires a tremendous amount of work in order to spot the cause of the poor performance of an application. In this paper, we propose mechanisms for assisting application developers in their exploration of trace files. We propose an algorithm for detecting repetitive patterns of events in trace files. Thanks to this algorithm, a trace can be viewed as loops and groups of events instead of the usual representation as a sequential list of events. We also propose a method to filter traces in order to eliminate duplicated information and to highlight points of interest. These mechanisms allow the performance analysis tool to pre-select the subsets of the trace that are more likely to contain useful information. We implemented the proposed mechanism in the EZTrace performance analysis framework and the experiments show that detecting patterns in various benchmarking applications is done in reasonable time, even when the trace contains millions of events. We also show that the filtering process can reduce the quantity of information in the trace that the user has to analyze by up to 99 %.

An integrated, ontology-based agricultural information system

This paper proposes an integrated, ontology-based agricultural information system (AIS) to provide all-round and precise information for efficiently guiding farmers and agri-professionals to conduct agricultural processing. Since the existing independent AIS platforms can only offer specific but incomplete agricultural information service, aiming at this issue, the newly proposed AIS system employs ontology techniques, including RDF-based representation and semantic reasoning, to integrate the index information provided by all involved independent agricultural information platforms. As a result, this newly proposed AIS system can provide users with integrated and accurate response information. Through a realistic case study and relevant experiments, it is clear that the accuracy ratio and integrity ratio of response information offered by this ontology-based integrated AIS can be enhanced to a great extent. In summary, not only agri-professionals, but also farmers, who might not understand information retrieval skills, can benefit from this newly proposed AIS to conduct activities in agricultural production life cycle.

Server-side prefetching in distributed file systems

This paper presents a proactive data prefetching mechanism on storage servers for distributed file systems to achieve better input/output (I/O) performance. This mechanism requires keeping tracks of what blocks were accessed and the information about client I/O requests to predict which blocks will be accessed in the future. Except for showing a linear access tendency, I/O access operations on blocks also reveal a random or similar‐random tendency but with certain access patterns. Thus, to address the prediction for future possible read operations, a linear regression algorithm and a chaotic prediction algorithm have been proposed to forecast the future access requests according to different access patterns. As a result, by employing the information, that is, offsets and request sizes, about the predicted future read requests on disk blocks, the storage servers in our newly presented mechanism can prefetch block data more precisely and then send it to associated client file system proactively. The experimental results illustrate that our proposed mechanism is able to shorten read latency to a great extent and then contribute to better read throughput significantly in certain application contexts. Copyright

Dynamical Re-striping Data on Storage Servers in Parallel File Systems

This paper proposes a mechanism for tracing I/O operations and their corresponding physical access on the storage servers in a parallel file system, then it guides data re-striping on the storage servers for handling complex physical access cases. To put this framework to work, we first figured out the way to map logical I/O operations on the client side to physical I/O access on the storage server side. We then developed a toolkit to generate I/O access patterns and their corresponding disk access. Finally, the information about both logical I/O access patterns and physical I/O access patterns can benefit to dynamical data re-striping and data pre-fetching on the storage servers for improving I/O performance. Experimental results show that the proposed data re-striping method by analyzing I/O access patterns and their associated disk access patterns can boost I/O data throughput significantly for applications with complicated access patterns. Especially, it work fairy well for the applications that deal with multi-dimension data sets across the fields of science, technology, education and business.