Jacob H. Gunther

Multiple Constraint Satisfaction by Belief Propagation: An Example Using Sudoku

The popular Sudoku puzzle bears structural resemblance to the problem of decoding linear error correction codes: solution is over a discrete set, and several constraints apply. We express the constraint satisfaction using a Tanner graph. The belief propagation algorithm is applied to this graph. Unlike conventional computer-based solvers, which rely on humanly specified tricks for solution, belief propagation is generally applicable, and requires no human insight to solve a problem. The presence of short cycles in the graph creates biases so that not every puzzle is solved by this method. However, all puzzles are at least partly solved by this method. The Sudoku application thus demonstrates the potential effectiveness of BP algorithms on a general class of constraint satisfaction problems

Sinkhorn Solves Sudoku

The Sudoku puzzle is a discrete constraint satisfaction problem, as is the error correction decoding problem. We propose here an algorithm for solution to the Sinkhorn puzzle based on Sinkhorn balancing. Sinkhorn balancing is an algorithm for projecting a matrix onto the space of doubly stochastic matrices. The Sinkhorn balancing solver is capable of solving all but the most difficult puzzles. A proof of convergence is presented, with some information theoretic connections. A random generalization of the Sudoku puzzle is presented, for which the Sinkhorn-based solver is also very effective.

Algorithms for blind equalization with multiple antennas based on frequency domain subspaces

This paper considers the problem of recovering an unknown signal transmitted over an unknown (but stationary) multipath channel, and received by a narrowband array with unknown calibration. Unlike previously proposed multichannel blind equalization techniques, the methods described herein employ a model based on physical channel parameters rather than unstructured multiple output FIR filters. The algorithms exploit the structure of the signal and noise subspaces of the array output data when transformed to the frequency domain. Two approaches are presented. The first is an ESPRIT-like solution that provides a closed-form, but suboptimal, blind signal estimate. The second is based on maximum likelihood and, though requiring a search, is easily initialized with the ESPRIT solution. A mathematical development of the two algorithms is given, and their advantages and disadvantages relative to other currently available techniques are discussed.

Methods for blind equalization and resolution of overlapping echoes of unknown shape

This paper considers the related problems of using an uncalibrated antenna array to (1) recover an unknown signal transmitted over an unknown (but stationary) multipath channel and (2) resolve overlapping pulse echoes with unknown shape. Unlike previously proposed multichannel blind equalization techniques, the methods described herein employ a model based on physical channel parameters rather than unstructured single-input, multi-output FIR filters. The algorithms exploit similarities between a model for the data in the frequency domain and the standard direction-of-arrival estimation problem. This connection between the two problems suggests several different approaches based on, for example, maximum likelihood, MODE, IQML, and ESPRIT. These approaches are developed in some detail, and the results of several simulation examples are included to compare their performance.

A new approach for symbol frame synchronization and carrier frequency estimation in OFDM communications

This work considers the problem of jointly estimating symbol frame boundaries and carrier frequency offsets for orthogonal frequency division multiplexed (OFDM) communications in frequency selective fading environments. Orthogonality between the modulated and virtual carriers over an interference free window of the received signal is used to develop an algorithm for estimating the carrier frequency offset and detecting the beginning of a symbol frame. By using a cyclic prefix to remove interference from neighboring frames, the the method is applicable in the presence of dispersive channels. The main contribution of this work is the joint estimation of the frequency offset and the frame boundary.

Fast nonlinear filtering via Galerkin's method

The conditional probability density function of the state of a stochastic dynamic system represents the complete solution to the nonlinear filtering problem because, with the conditional density in hand, optimal estimates of the state can be computed. It is well known that, for systems with continuous dynamics, the conditional density evolves, between measurements, according to Kolmogorovs forward equation. At a measurement, it is updated according to Bayes formula. Therefore, these two equations can be viewed as the dynamic equations of the conditional density and, hence, the optimal filter. In this paper, Galerkins method is used to approximate the nonlinear filter by solving for the entire conditional density. Using an FFT to approximate the projections required in Galerkins method leads to a computationally efficient nonlinear filter. The implementation details are given and performance is assessed through simulations.

A generalized LDPC decoder for blind turbo equalization

The equations for iteratively decoding low-density parity-check (LDPC) codes are generalized to compute joint probabilities of arbitrary sets of codeword bits and parity checks. The standard iterative LDPC decoder, which computes single variable probabilities, is realized as a special case. Another specialization allows pairwise joint posterior probabilities of pairs of codeword bits to be computed. These pairwise joint probabilities are used in an expectation-maximization (EM) based blind channel estimator that is ignorant of the code constraints. Channel estimates are input to a turbo equalizer that exploits the structure of the LDPC code. Feeding pairwise posterior probabilities back to the channel estimator eliminates the need to average across time for channel estimation. Therefore, this scheme can be used to equalize very long codewords, even when the channel is time varying. This blind turbo equalizer is evaluated through computer simulations and found to perform as well as a channel-informed turbo equalizer but with approximately twice the number of turbo iterations.

A simplified natural gradient learning algorithm

Adaptive natural gradient learning avoids singularities in the parameter space of multilayer perceptrons. However, it requires a larger number of additional parameters than ordinary backpropagation in the form of the Fisher information matrix. This article describes a new approach to natural gradient learning that uses a smaller Fisher information matrix. It also uses a prior distribution on the neural network parameters and an annealed learning rate. While this new approach is computationally simpler, its performance is comparable to that of Adaptive natural gradient learning.

An Iterative Least Square Approach to Elastic-Lidar Retrievals for Well-Characterized Aerosols

An iterative least square method is presented for estimating the solution to the lidar equation. The method requires knowledge of the backscatter values at a boundary point for all channels and a priori defined relationships between backscatter, extinction, and mass-fraction concentration for all scattering components. The lidar equation is formulated in vector form, and a solution is computed using an iterative least square technique. The solution is stable for signals with extremely low signal-to-noise ratios and for signals at ranges far beyond the boundary point. The solution can be applied to lidar signals with an arbitrary number of wavelengths and scattering components.

Direct semi-blind symbol estimation for multipath channels

A number of recent papers have treated the problem of channel estimation when known training data are present. The unknown part of the signal is typically estimated in an independent step, where the channel inverse or the Viterbi algorithm is applied to the received data. In this paper a technique is presented for directly estimating the unknown symbols in a block of data that also contains training information. An unstructured estimate of the channel can also be obtained that exploits the finite alphabet property of the transmitted signal. The proposed method is implemented in the frequency domain, and works best in situations where the transmitted symbols act to convert the linear convolution of the channel into circular convolution (as with a cyclic prefix in multicarrier systems). However reasonable results are still obtained asymptotically even without this constraint.