Islene Calciolari Garcia

From Backup to Hot Standby: High Availability for HDFS

Cluster-based distributed file systems generally have a single master to service clients and manage the namespace. Although simple and efficient, that design compromises availability, because the failure of the master takes the entire system down. Before version 2.0.0-alpha, the Hadoop Distributed File System (HDFS) -- an open-source storage, widely used by applications that operate over large datasets, such as MapReduce, and for which an uptime of 24x7 is becoming essential -- was an example of such systems. Given that scenario, this paper proposes a hot standby for the master of HDFS achieved by (i) extending the masters state replication performed by its check pointer helper, the Backup Node, and by (ii) introducing an automatic fail over mechanism. The step (i) took advantage of the message duplication technique developed by other high availability solution for HDFS named Avatar Nodes. The step (ii) employed another Hadoop software: ZooKeeper, a distributed coordination service. That approach resulted in small code changes, 1373 lines, not requiring external components to the Hadoop project. Thus, easing the maintenance and deployment of the file system. Compared to HDFS 0.21, tests showed that both in loads dominated by metadata operations or I/O operations, the reduction of data throughput is no more than 15% on average, and the time to switch the hot standby to active is less than 100 ms. Those results demonstrate the applicability of our solution to real systems. We also present related work on high availability for other file systems and HDFS, including the official solution, recently included in HDFS 2.0.0-alpha.

Progressive construction of consistent global checkpoints

A checkpoint pattern is an abstraction of the computation performed by a distributed application. A progressive view of this abstraction is formed by a sequence of consistent global checkpoints that may have occurred in this order during the execution of the application. Considering pairs of checkpoints, we have determined that a checkpoint must be observed before another in a progressive view if the former Z-precedes the latter. Based on the Z-precedence and characteristics of the checkpoint pattern, we propose original algorithms for the progressive construction of consistent global checkpoints. We demonstrate that the Z-precedence between a pair of checkpoints is a much simpler way to express the existence of a zigzag path connecting them, and we discuss other advantages of our relation.

An efficient checkpointing protocol for the minimal characterization of operational rollback-dependency trackability

A checkpointing protocol that enforces rollback-dependency trackability (RDT) during the progress of a distributed computation must induce processes to take forced checkpoints to avoid the formation of nontrackable rollback dependencies. A protocol based on the minimal characterization of RDT tests only the smallest set of nontrackable dependencies. The literature indicated that this approach would require the processes to maintain and propagate O(n/sup 2/) control information, where n is the number of processes in the computation. In this paper, we present a protocol that implements this approach using only O(n) control information.

OFSwitch13: Enhancing ns-3 with OpenFlow 1.3 Support

The world is witnessing the rapid evolution of communication technologies, and meeting current market requirements is virtually impossible with traditional network architectures. Many works point to the use of Software Defined Networking (SDN) paradigm and the OpenFlow protocol as enabling solutions to overcome current limitations. Despite the fact that the Network Simulator 3 (ns-3 ) already has a module that supports OpenFlow simulations, it is possible to note that the available implementation provides a very outdated protocol version (0.8.9). As many new major features were introduced up to the latest versions, it is interesting to have some of them available for use. In this context, this paper presents the OFSwitch13: a module to enhance the ns-3 with OpenFlow 1.3 technology support. This module provides both an OpenFlow 1.3 switch device and a controller application interface. Details about module design and implementation are discussed throughout this paper, and a case study scenario is used to illustrate some of the available OFSwitch13 module features.

Integrating OpenFlow to LTE: Some issues toward software-defined mobile networks

The world is witnessing a rapid growth in mobile communication, and operators have been forced to handle network traffic more resourcefully. Many works point to the use of Software-Defined Networking (SDN) as an enabling technology to overcome current limitations, making possible upcoming 5G networks. In this paper we discuss how the OpenFlow protocol can be integrated into existing Long-Term Evolution (LTE) networks, going through state-of-the-art solutions and highlighting open issues. We also introduce a novel OpenFlow module for ns-3 simulator, allowing reliable simulations of LTE-SDN integrated networks, as illustrate in a case study example.

A Cognitive Mechanism for Rate Adaptation in Wireless Networks

Sophisticated wireless interfaces support multiple transmission data rates and the selection of the optimal data rate has a critical impact on the overall network performance. Proper rate adaptation requires dynamically adjusting data rate based on current channel conditions. Despite several rate adaptation algorithms have been proposed in the literature, there are still challenging issues related to this problem. The main limitations of current solutions are concerned with how to estimate channel quality to appropriately adjust the rate. In this context, we propose a Cognitive Rate Adaptation mechanism for wireless networks. This mechanism includes a distributed self-configuration algorithm in which the selection of data rate is based on past experience. The proposed approach can react to changes in channel conditions and converge to the optimal data rate, while allowing a fair channel usage among network nodes. Simulation results obtained underline performance benefits with respect to existing rate adaptation algorithms.

Non-Blocking Synchronous Checkpointing Based on Rollback-Dependency Trackability

This article proposes an original approach that applies the rollback-dependency trackability (RDT) property to implement a new non-blocking synchronous checkpointing protocol, called RDT-NBS, that takes mutable checkpoints and efficiently supports concurrent initiators. Mutable checkpoints can be saved in non-stable storage and make it possible for non-blocking synchronous checkpointing protocols to save a minimal number of checkpoints in stable storage during the construction of a consistent global checkpoint. We prove that this minimality property does not hold in presence of concurrent checkpointing initiations. Even though, RDT-NBS uses mutable checkpoints to reduce the use of stable memory assuring the existence of a consistent global checkpoint in stable storage. We also present simulation results that compare RDT-NBS to quasi-synchronous RDT

Optimal Asynchronous Garbage Collection for RDT Checkpointing Protocols

Communication-induced checkpointing protocols that ensure rollback-dependency trackability (RDT) guarantee important properties to the recovery system without explicit coordination. However, there was no garbage collection algorithm for them which did not use some type of process synchronization, like time assumptions or reliable control message exchanges. This paper addresses the problem of garbage collection for RDT checkpointing protocols and presents an optimal solution for the case where coordination is done only by means of timestamps piggybacked in application messages. The algorithm uses the same timestamps as off-the-shelf RDT protocols and ensures the tight upper bound on the number of uncollected checkpoints for each process during all the system execution

Asynchronous construction of consistent global snapshots in the Object and Action Model

The Object and Action Model (OAM) is well-known as an adequate paradigm to build fault-tolerant configurable distributed applications. The reconfiguration of an application depends on the construction of a consistent global snapshot of its global state. An atomic action that reads the states of all objects of the application is a simple and straightforward way to obtain such a global snapshot, but reduces concurrency and interferes with the underlying computation. In the Process and Message Model (PMM), consistent snapshots can be constructed asynchronously by a component that passively receives process states. This paper presents OAM-based asynchronous global snapshot algorithms that are equivalent to PMM-based algorithms, built using a precedence relation defined for atomic actions. Arjuna, an object-oriented action-based distributed programming environment, has been used to implement these OAM-based global snapshot algorithms, allowing us to conclude that our approach is promising.

sendmmsg: An Efficient System Call to Send Multiple Messages on Linux

This article describes the limitations of sendmsg, the current system call responsible for sending network messages in Linux. It is shown that this system call performance is not good enough, mainly when it needs to send a batch of similar data to many destinations in the network, as commonly experienced in IPTV servers. In order to prove that, a new system call has been implemented, sendmmsg, which acts similarly to the original one, but presenting some performance advantages, specially when sending similar data to peers. This new call needs only an extra parameter, when compared to the original one.(Portuguese version) Este artigo descreve as limitações da chamada de sistema sendmsg responsável por enviar mensagens de rede no sistema operacional Linux. É mostrado que tal chamada não possui um bom desempenho quando se necessita enviar vários pacotes em lote, contendo dados similares, para vários destinatários remotos -- caso comum em servidores de IPTV. Para tanto, é implementada uma nova função, sendmmsg, parecida com a original, porém, mais eficiente, apenas com o acréscimo de um parâmetro.