Borja Peleato

Distributed Optimization and Statistical Learning via the Alternating Direction Method of Multipliers

Many problems of recent interest in statistics and machine learning can be posed in the framework of convex optimization. Due to the explosion in size and complexity of modern datasets, it is increasingly important to be able to solve problems with a very large number of features or training examples. As a result, both the decentralized collection or storage of these datasets as well as accompanying distributed solution methods are either necessary or at least highly desirable. In this review, we argue that the alternating direction method of multipliers is well suited to distributed convex optimization, and in particular to large-scale problems arising in statistics, machine learning, and related areas. The method was developed in the 1970s, with roots in the 1950s, and is equivalent or closely related to many other algorithms, such as dual decomposition, the method of multipliers, Douglas–Rachford splitting, Spingarns method of partial inverses, Dykstras alternating projections, Bregman iterative algorithms for l1 problems, proximal methods, and others. After briefly surveying the theory and history of the algorithm, we discuss applications to a wide variety of statistical and machine learning problems of recent interest, including the lasso, sparse logistic regression, basis pursuit, covariance selection, support vector machines, and many others. We also discuss general distributed optimization, extensions to the nonconvex setting, and efficient implementation, including some details on distributed MPI and Hadoop MapReduce implementations.

Distance aware collision avoidance protocol for ad-hoc underwater acoustic sensor networks

This paper proposes a channel access protocol for ad-hoc underwater acoustic networks which are characterized by long propagation delays and unequal transmit/receive power requirements. The protocol saves transmission energy by avoiding collisions while maximizing throughput. It is based on minimizing the duration of a hand-shake by taking advantage of the receivers tolerance to interference when the two nodes are closer than the maximal transmission range. Nodes do not need to be synchronized, can move, are half-duplex, and use the same transmission power. This protocol achieves a throughput several times higher than that of the slotted FAMA, while offering similar savings in energy. Although carrier sensing ALOHA offers a higher throughput, it wastes much more power on collisions.

A MAC protocol for ad-hoc underwater acoustic sensor networks

A medium access control (MAC) protocol is proposed that is suitable for non-synchronized ad-hoc networks, and in particular for the energy-constrained underwater acoustic networks which are characterized by long propagation delays. The protocol exploits the difference in the link lengths between the nodes instead of using waiting times proportional to the maximal link length. To do so, it relies on a receivers ability to tolerate a certain level of interference. By minimizing the length of the hand-shake procedure preceeding the data transmission, the throughput efficiency is increased as compared to the previously proposed protocols, while collision avoidance minimizes the energy consumption.

Maximizing MLC NAND lifetime and reliability in the presence of write noise

The aggressive scaling of the NAND flash technology has led to write noise becoming the dominant source of disturbance in the currently shipping sub-30 nm MLC NAND memories. Write noise can be mitigated by reducing the magnitude of the voltage levels programmed into the cells, which additionally translates to longer flash memory lifetime. However, if all the target levels are small and close together, the probability of error could become excessively high. It is therefore necessary to optimize the target level placement in order to achieve a trade-off between flash lifetime and error probability. This paper proposes a method to maximize flash lifetime subject to reliability constraints, and vice versa. Simulation results show that the proposed method doubles flash lifetime in comparison to a naive scheme, for a 2% reliability constraint. It also comes very close to the optimal solution obtained by brute force search, while maintaining negligible computational complexity in comparison.

Towards minimizing read time for NAND flash

On NAND flash, a primary source of increased read time comes from the fact that in the presence of noise, the flash medium must be read several times using different read threshold voltages to find the optimal read location, which minimizes bit-error-rate. This paper proposes an algorithm to estimate the optimal read threshold in a fast manner using a limited number of re-reads. Then it derives an expression for the resulting BER in terms of the minimum possible BER. It is also shown that minimizing BER and minimizing codeword-error-rate are competing objectives in the presence of a limited number of allowed re-reads, and a tradeoff between the two is proposed.

A Channel Sharing Scheme for Underwater Cellular Networks

We propose a channel-allocation and scheduling protocol for underwater acoustic cellular networks. The protocol exploits both the acoustic path loss and the long propagation delay of the underwater channel to enable efficient use of system resources. By scheduling co-channel transmissions to avoid strong interference, it allows grouping of the cells into small clusters, thus achieving higher efficiency than a conventional scheme based on spatial frequency (or code) reuse alone.

Adaptive Read Thresholds for NAND Flash

A primary source of increased read time on NAND flash comes from the fact that, in the presence of noise, the flash medium must be read several times using different read threshold voltages for the decoder to succeed. This paper proposes an algorithm that uses a limited number of rereads to characterize the noise distribution and recover the stored information. Both hard and soft decoding are considered. For hard decoding, this paper attempts to find a read threshold minimizing bit error rate (BER) and derives an expression for the resulting codeword error rate. For soft decoding, it shows that minimizing BER and minimizing codeword error rate are competing objectives in the presence of a limited number of allowed rereads, and proposes a tradeoff between the two. The proposed method does not require any prior knowledge about the noise distribution but can take advantage of such information when it is available. Each read threshold is chosen based on the results of previous reads, following an optimal policy derived through a dynamic programming backward recursion. The method and results are studied from the perspective of an SLC Flash memory with Gaussian noise, but this paper explains how the method could be extended to other scenarios.

Probabilistic graphical model for flash memory programming

Flash memory presents significant advantages over hard drives in terms of read speed and power efficiency; however its lifetime can be several orders of magnitude smaller. Thus increasing lifetime of flash memory via signal processing techniques is an important research area. The first half of the paper presents a statistical method for estimating the health of the cells in a Flash memory, based on which a variable error correction coding scheme can be used to increase lifetime. The second half of the paper proposes a statistical approach to increase lifetime when the flash controller can dynamically vary the program and erase operation strategy. This approach uses Markov Decision Processes (MDP) to choose the optimal program or erase strategy at any given point in the life of a Flash memory based on its current state or health. From a bigger picture stand-point, the paper presents a novel way of flash management using a Markov model for health of the memory at any given point in its lifetime.

BER-based wear leveling and bad block management for NAND flash

One of the main challenges keeping flash memories from achieving widespread distribution is their limited endurance. The programming and erasing from re-writes damages the cells, progressively increasing the number of errors until information can no longer be stored reliably. Most manufacturers employ powerful ECC techniques, but there is a limit to the number of errors that these can correct. When the number of errors goes beyond the capability of the ECC, it is necessary to invoke RAID, which introduces significant latency and jeopardizes the speed of the memory if used too often. This paper introduces a method for estimating the BER that a flash page will exhibit after retention and uses this estimate for wear leveling. Instead of leveling out the number of PE cycles in all the blocks, the proposed scheme attempts to wear all the blocks evenly so that they all suffer the same BER. Additionally, the estimate will be used to detect bad blocks, those likely to exhibit a number of errors beyond the ECC correction capability, and retire them from further use.

Spatial Reuse in Dense Wireless Areas: A Cross-Layer Optimization Approach via ADMM

This paper introduces an efficient method for communication resource use in dense wireless areas where all nodes must communicate with a common destination node. The proposed method groups nodes based on their distance from the destination and creates a structured multi-hop configuration in which each group can relay its neighbors data. The large number of active radio nodes and the common direction of communication toward a single destination are exploited to reuse the limited spectrum resources in spatially separated groups. Spectrum allocation constraints among groups are then embedded in a joint routing and resource allocation framework to optimize the route and amount of resources allocated to each node. The solution to this problem uses coordination among the lower-layers of the wireless-network protocol stack to outperform conventional approaches where these layers are decoupled. Furthermore, the structure of this problem is exploited to obtain a semi-distributed optimization algorithm based on the alternating direction method of multipliers (ADMM) where each node can optimize its resources independently based on local channel information.