Ho Min Jung

Data deduplication using dynamic chunking algorithm

Data deduplication is widely used in storage systems to prevent duplicated data blocks. In this paper, we suggest a dynamic chunking approach using fixed-length chunking and file similarity technique. The fixed-length chunking struggles with boundary shift problem and shows poor performance when handling duplicated data files. The key idea of this work is to utilize duplicated data information in the file similarity information. We can easily find several duplicated point by comparing hash key value and file offset within file similarity information. We consider these duplicated points as a hint for starting position of chunking. With this approach, we can significantly improve the performance of data deduplication system using fixed-length chunking. In experiment result, the proposed dynamic chunking results in significant performance improvement for deduplication processing capability and shows fast processing time comparable to that of fixed length chunking.

Real-Time Data Stream Management System for Large Volume of RFID Events

A data stream management system (DSMS) for the real-time processing of a large volume of RFID events is proposed herein. It allows the real-time scheduling of groups of operators, the components of a query, to process user queries of various kinds within the time limit. It also creates user-level threads consisting of operators, and the DSMS scheduler schedules the threads of a query, depending on the priority. A prototype of the proposed system was built, and the scheduling of a large volume of RFID events was shown through experiments.

ICLS : Intelligent Cricket-based Location Tracking System Using Sensor Fusion

The key issues in ubiquitous sensor networks for context awareness is location-tracking. The main goal of location-tracking is to find the position of a moving object. In this paper, we describes the design and implementation of ICLS (Intelligent Cricket-based Location System). Especially, a more accurate location-tracking system using a sensor fusing mechanism is proposed herein. The benefits of sensor fusion are that it can obtain more accurate, complete, or dependable results. ICLS employs an ltrasonic sensor for ranging by time-of-flight measurements and uses a magnetic sensor to detect obstacles. To yield a more accurate navigation, a novel pass finding algorithm is also proposed in this paper. The results of the experiment that was conducted in this study show that a mobile robot moves towards a destination with a high degree of accuracy.

Design and Implementation of Deduplication Storage Server Using Stride Scheme

In this paper, we propose a deduplication storage system exploiting stride scheme to minimize storage space and network bandwidth. The key idea of this paper is to provide duplication checking mechanism based on stride scheme of fixed-length block algorithm. Our approach can minimize computation time for detecting duplicated region of blocks while decreasing computation time. We made several experiments to show performance enhancement of the proposed system. In experiments, stride scheme can diminish storage space and efficiently manages duplicated data blocks.

Low Overhead File Synchronization Mechanism

Traditional file synchronization programs have difficulties in handling lots of metadata information when the file is transferred to the server. In this paper, we propose a practical approach for minimizing metadata exchanging overhead on the file synchronization system. The key idea of this paper is to adapt hash key compression technique that computes new hash key with several hash keys.

Energy Efficient Deduplication System Exploiting Similarity Information

In this paper, we propose a file deduplication system considering file similarity information. The key idea of this paper is to reduce the energy consumption of a mobile device by minimizing network traffic. The proposed system can detect duplication ratio of a file using file similarity algorithm. In this work, for a file with high data redundancy, data deduplication is performed during file transferring, otherwise, conventional file transfer scheme works without data deduplication. We implemented Fixed-length Chunking and Variable-length Chunking deduplication algorithm using file similarity. Experiment result shows that the proposed scheme significantly reduces energy consumption of data deduplication.

Minimizing Metadata Size for File Synchronization Using Variable-Length Chunking

A file synchronization program has to exchange lots of metadata information between a client and a server. As data have been growing rapidly in file synchronization systems, the metadata exchanging overhead is more critical now than before. In this paper, we propose a practical approach for minimizing metadata exchange overhead on the file synchronization system using variable-length chunking. The key idea is to minimize metadata by adapting file similarity information and generating only one hash key for several metadata. In the experiment, traditional file synchronization program sends 4.3 Mbyte over 500 MByte data file. However, the proposed algorithm only sends 37 Kbyte metadata.

Energy efficient file transfer mechanism using deduplication scheme

In a mobile device, it is important to minimize battery consumption when we operate an application. In this paper, we propose an energy efficient file transfer scheme for mobile device by exploiting deduplication scheme. Generally, a mobile phone needs more power for WiFi work rather than CPU work, therefore we have to minimize WiFi work. We can minimize the number of packets by adapting deduplication which consumes lots of CPU power. Deduplication work consumes additional CPU power during data hashing, however overall energy consumption can be diminished. Experiment result shows that the proposed scheme significantly reduces battery consumption if a file has over 5% SRP(Space Reduction Percentage).

DTAR: Deduplication TAR Scheme for Data Backup System

Tar archive format does not support file deduplication scheme, therefore it has drawbacks for utilizing disk storage system efficiently. In this paper, we propose an extended TAR file format called DTAR (Deduplication Tape Archive) which provides file-level deduplication. Key idea of our work is to provide block aligned compressed file that can speed up file insertion and deletion for archiving. Furthermore, DTAR shows performance enhancement using file similarity scheme for distributing files into several storage nodes. Experiment results show that the proposed system can reduce data storage space efficiently and diminish network data traffic compared to general file transfer system.