Junjie Xie

Totoro: A Scalable and Fault-Tolerant Data Center Network by Using Backup Port

Scalability and fault tolerance become a fundamental challenge of data center network structure due to the explosive growth of data. Both structures proposed in the area of parallel computing and structures based on tree hierarchy are not able to satisfy these two demands. In this paper, we propose Totoro, a scalable and fault-tolerant network to handle the challenges by using backup built-in Ethernet ports. We connect a bunch of servers to an intra-switch to form a basic partition. Then we utilize half of backup ports to connect those basic partitions with inter-switches to build a larger partition. Totoro is hierarchically and recursively defined and the high-level Totoro is constructed by many low-level Totoros. Totoro can scale to millions of nodes. We also design a fault-tolerant routing protocol. Its capability is very close to the performance bound. Our experiments show that Totoro is a viable interconnection structure for data centers.

Exploiting Fingerprint Prefetching to Improve the Performance of Data Deduplication

Data deduplication has become an important and economic way to remove the redundant data segments, thus alleviating the pressure incurred by large amounts of data need to store. Fingerprints are used to represent and identify identical data blocks when performing data deduplication. However, the amount of fingerprints grows with the increase of data. Due to the limited memory size, the fingerprints have to be stored in disk drives. When the fingerprints are not satisfied in memory, disk I/Os will be generated to obtain the on-disk fingerprints. This results in small and random I/Os, thus significantly degrading the performance of data deduplication. This paper introduces a fingerprint prefetching algorithm by leveraging file similarity and data locality. On the one hand, we present a similar file recognition algorithm to identify the similar files that are considered to have some modifications and share a large portion of identical data blocks. On the other hand, the on-disk fingerprints are organized according to the sequence of data streams, thus maintaining the data locality to improve the cache hit ratio. The proposed prefetching algorithm will request fingerprints from disk drives and place them in memory before they are actually needed. This will significantly improve the cache hit ratio when the fingerprints are actually needed, thus enhancing the performance of data deduplication. Two real data sets that represent typical cloud storage and cloud computing scenarios are collected to evaluate the effectiveness of the proposed approach.

EPAS: A Sampling Based Similarity Identification Algorithm for the Cloud

The explosive growth of data brings new challenges to the data storage and management in cloud environment. These data usually have to be processed in a timely fashion in the cloud. Thus, any increased latency may cause a massive loss to the enterprises. Similarity detection plays a very important role in data management. Many typical algorithms such as Shingle, Simhash, Traits and Traditional Sampling Algorithm (TSA) are extensively used. The Shingle, Simhash and Traits algorithms read entire source file to calculate the corresponding similarity characteristic value, thus requiring lots of CPU cycles and memory space and incurring tremendous disk accesses. In addition, the overhead increases with the growth of data set volume and results in a long delay. Instead of reading entire file, TSA samples some data blocks to calculate the fingerprints as similarity characteristics value. The overhead of TSA is fixed and negligible. However, a slight modification of source files will trigger the bit positions of file content shifting. Therefore, a failure of similarity identification is inevitable due to the slight modifications. This paper proposes an Enhanced Position-Aware Sampling algorithm (EPAS) to identify file similarity for the cloud by modulo file length. EPAS concurrently samples data blocks from the head and the tail of the modulated file to avoid the position shift incurred by the modifications. Meanwhile, an improved metric is proposed to measure the similarity between different files and make the possible detection probability close to the actual probability. Furthermore, this paper describes a query algorithm to reduce the time overhead of similarity detection. Our experimental results demonstrate that the EPAS significantly outperforms the existing well known algorithms in terms of time overhead, CPU and memory occupation. Moreover, EPAS makes a more preferable tradeoff between precision and recall than that of other similarity detection algorithms. Therefore, it is an effective approach of similarity identification for the cloud.

An Incrementally Scalable and Cost-Efficient Interconnection Structure for Data Centers

The explosive growth in the volume of data storing and complexity of data processing drive data center networks (DCNs) to become incrementally scalable and cost-efficient while to maintain high network capacity and fault tolerance. To address these challenges, this paper proposes a new structure, called Totoro, which is defined recursively and hierarchically: dual-port servers and commodity switches are used to make Totoro affordable; a bunch of servers are connected to an intra-switch to form a basic partition; to construct a high-level structure, a half of the backup ports of servers in the low-level structures are connected by inter-switches in order to incrementally build a larger partition. Totoro is incrementally scalable since expanding the structure does not require any rewiring or routing alteration. We further design a distributed and fault-tolerant routing protocol to handle multiple types of failures. Experimental results demonstrate that Totoro is able to satisfy the demands of fault tolerance and high throughput. Furthermore, architecture analysis indicates that Totoro balances between performance and costs in terms of robustness, structural properties, bandwidth, economic costs and power consumption.

Identifying File Similarity in Large Data Sets by Modulo File Length

Identifying file similarity is very important for data management. Sampling files is a simple and effective approach to identify the file similarity. However, the traditional sampling algorithm(TSA) is very sensitive to file modification. For example, a single bit shift would result in a failure of similarity detection. Many research efforts have been invested in solving/alleviating this problem. This paper proposes a Position-Aware Sampling(PAS) algorithm to identify file similarity in large data sets by modulo file length. This method is very effective in dealing with file modification when performing similarity detection. Comprehensive experimental results demonstrate that PAS significantly outperforms a well-known similarity detection algorithm called simhash in terms of precision and recall. Furthermore, the time overhead, CPU and memory occupation of PAS are much less than that of simhash.

mtCloudSim: A Flow-Level Network Simulator for Multi-Tenant Cloud

Currently, novel topologies and advanced resource allocation strategies for multi-tenant cloud datacenters are two research hotspots. Due to the high convenience and efficiency, researchers tend to use simulation to evaluate the proposed topologies or strategies. However, the current network simulators do not support the multi-tenant cloud environment inherently. Moreover, the low simulation speed and high memory consumption limit the traditional packet-level simulators to estimate the scenario of large-scale datacenters. In this paper, we propose a new flow-level network simulator, mtCloudSim, to overcome the above issues. The simulator estimates the data flows behavior in the real world, i.e., 1) increasing the sending rate when the network is not busy and 2) suspending when the congestion occurs. Bandwidth isolation is inherently provided and users are allowed to define bandwidth requirement for the experiments with our simulator. Object-oriented programming (OOP) makes it easy to evaluate novel network topologies. The tracing system is also able to generate abundant and detailed statistics for experiments. The experiments demonstrate that mtCloudSim is available for multi-tenant cloud evaluation.

Leverage similarity and locality to enhance fingerprint prefetching of data deduplication

Data deduplication has been widely used at data backup system due to the significantly reduced requirements of storage capacity and network bandwidth. However, the performance of data deduplication gradually decreases with the growth of deduplicated data. This is because the volume of fingerprints grows significantly with the increase of backup data, and a large portion of fingerprints have to be stored on disk drives. This incurs frequent disk accesses to locate fingerprints and blocks the process of data deduplication. Furthermore, the fingerprints belonging to the same file may be discretely stored on disk drives. This generates random and small disk accesses, and results in significant performance degradation when the fingerprints are referred. Additionally, a single fingerprint may appear only once during a backup process. This results in very low cache hit ratio due to lacking temporal locality. This paper proposes to employ file similarity to enhance the fingerprint prefetching, thus improving the cache hit ratio and the performance of data deduplication. Furthermore, the fingerprints are arranged sequently in terms of the backup data stream to maintain the locality and promote the performance. Experimental results demonstrate that the proposed idea can effectively reduce the number of fingerprint accesses going to disk drives, decrease the query overhead of fingerprints, thus significantly alleviating the disk bottleneck of data deduplication.

Athena: A Fault-Tolerant, Efficient and Applicable Routing Mechanism for Data Centers

The overall performance of data center depends on the physical topology and the corresponding routing mechanism. Many novel network structures have been proposed in recent years to remedy the shortcomings of traditional tree-based structure. Especially some hybrid recursively defined structures with acceptable costs can perform well. These structures mainly adopt the conventional routing mechanism which maintains large and complex link states. However, this routing mechanism still can not work out the cost-optimal path to meet the requirement of short latency and low extra traffic consumption. Hence, this paper presents Athena Routing Mechanism (ARM) based on Dynamic Programming with path probing scheme to further promote the performance of those structures. ARM is fault-tolerant since it makes full use of redundant links. It is also able to work out the shortest paths, which shortens the communication delay and releases intermediate servers from forwarding loads as well as extra CPU and bandwidth resources. Results from theoretical analysis, simulations and experiments firmly support the conclusion that ARM is a fault-tolerant and efficient routing mechanism which is able to be generalized to many other hybrid structures.

Reducing the read latency of in-line deduplication file system

In-line dedupliaction systems mainly focus on secondary storage for backup and archiving, and just offer several simple APIs. Applications can not directly invoke these APIs without modification. Although file systems offer abundant APIs and amity to applications, building a file system for in-line deduplication brings new challenges in the I/O path. Read operations involve multiple disk accessing, which includes getting fingerprints in file recipes, obtaining the addresses by checking fingerprint index and reading corresponding data blocks in disk drive. This extremely increases the latency in read path. We present a Low-Read-Latency File System (LRLFS) for the in-line deduplication. Experiments suggest that LRLFS obtains low read latency in read path with negligible storage overhead, acceptable CPU and memory utilization rate.

Improving Routing Performance via Dynamic Programming in Large-Scale Data Centers

The Internet of Things has become a spotlight for a long period of time and generates massive amounts of sensor data. Thus, data centers play more and more crucial roles in processing and analyzing the explosively increasing data. To remedy the shortcomings of traditional tree-based structure, many novel server-centric network structures have been proposed in recent years. Their original routing mechanisms based on divide and conquer (DC) are not able to work out the shortest paths. So, there is still promotion room for communication delay reduction. Since dynamic programming (DP) is a classical strategy to obtain optimal solution, this paper proposes a routing mechanism based on DP and applies it to data center for better solving the weakness occurred by DC. Experiments firmly support the conclusion that adopting DP in routing calculation achieves appealing performance of short latency, great fault-tolerance and reasonable resource consumption. Theoretical analysis also proves that it is applicable to most popular structures.