G. David Forney

On the Effective Weights of Pseudocodewords for Codes Defined on Graphs with Cycles

The behavior of an iterative decoding algorithm for a code defined on a graph with cycles and a given decoding schedule is characterized by a cycle-free computation tree. The pseudocodewords of such a tree are the words that satisfy all tree constraintsj pseudocodewords govern decoding performance. Wiberg [12] determined the effective weight of pseudocodewords for binary codewords on an AWGN channel. This paper extends Wiberg’s formula for AWGN channels to nonbinary codes, develops similar results for BSC and BEC channels, and gives upper and lower bounds on the effective weight. The 16-state tail-biting trellis of the Golay code [2] is used for exampIes. Although in this case no pseudocodeword is found with effective weight less than the minimum Hamming weight of the Golay code on an AWGN channel, it is shown by example that the minimum effective pseudocodeword weight can be less than the minimum codeword weight.

Codes on Graphs: Duality and MacWilliams Identities

A conceptual framework involving partition functions of normal factor graphs is introduced, paralleling a similar recent development by Al-Bashabsheh and Mao. The partition functions of dual normal factor graphs are shown to be a Fourier transform pair, whether or not the graphs have cycles. The original normal graph duality theorem follows as a corollary. Within this framework, MacWilliams identities are found for various local and global weight generating functions of general group or linear codes on graphs; this generalizes and provides a concise proof of the MacWilliams identity for linear time-invariant convolutional codes that was recently found by Gluesing-Luerssen and Schneider. Further MacWilliams identities are developed for terminated convolutional codes, particularly for tail-biting codes, similar to those studied recently by Bocharova, Hug, Johannesson, and Kudryashov.

Iterative Decoding of Tail-Biting Trellises and Connections with Symbolic Dynamics

The sum-product and min-sum algorithms are used to decode codes defined by trellises. In this paper, we discuss the behavior of these and related algorithms on tail-biting (TB) trellises.

Minimal Realizations of Linear Systems: The “Shortest Basis” Approach

Given a discrete-time linear system <i>C</i>, a shortest basis for <i>C</i> is a set of linearly independent generators for <i>C</i> with the least possible lengths. A basis <i>B</i> is a shortest basis if and only if it has the predictable span property (i.e., has the predictable delay and degree properties, and is non-catastrophic), or alternatively if and only if it has the subsystem basis property (for any interval <i>J</i>, the generators in <i>B</i> whose span is in <i>J</i> is a basis for the subsystem <i>CJ</i>). The dimensions of the minimal state spaces and minimal transition spaces of <i>C</i> are simply the numbers of generators in a shortest basis <i>B</i> that are active at any given state or symbol time, respectively. A minimal linear realization for <i>C</i> in controller canonical form follows directly from a shortest basis for <i>C</i>, and a minimal linear realization for <i>C</i> in observer canonical form follows directly from a shortest basis for the orthogonal system <i>C</i>^⊥. This approach seems conceptually simpler than that of classical minimal realization theory.

MacWilliams identities for codes on graphs

The MacWilliams identity for linear time-invariant convolutional codes that has recently been found by Gluesing-Luerssen and Schneider is proved concisely, and generalized to arbitrary group codes on graphs. A similar development yields a short, transparent proof of the dual sum-product update rule.

Local Irreducibility of Tail-Biting Trellises

This paper investigates tail-biting trellis realizations for linear block codes. Intrinsic trellis properties are used to characterize irreducibility on given intervals of the time axis. It proves beneficial to always consider the trellis and its dual simultaneously. A major role is played by trellis properties that amount to observability and controllability of trellis fragments of various lengths. For fragments of length less than the minimum span length of the code it is shown that fragment observability and fragment controllability are equivalent to irreducibility. For reducible trellises, a constructive reduction procedure is presented. The considerations also lead to a characterization for when the dual of a trellis allows a product factorization into elementary (“atomic”) trellises.

MacWilliams identities for terminated convolutional codes

Shearer and McEliece [8] showed that there is no MacWilliams identity for the free distance spectra of orthogonal linear convolutional codes. We show that on the other hand there does exist a MacWilliams identity between the generating functions of the weight distributions per unit time of a linear convolutional code C and its orthogonal code C⊥, and that this distribution is as useful as the free distance spectrum for estimating code performance. These observations are similar to those made recently by Bocharova et al. [1]; however, we focus on terminating by tail-biting rather than by truncation.

Group Codes and Behaviors

The close connections between convolutional coding theory and linear system theory are well known. A convolutional encoder is simply a discrete-time linear time-invariant (LTI) system over a finite field, always finite-dimensional, in general multivariable. These connections were exploited in the 1970’s to develop an algebraic structure theory of convolutional codes [5], the key elements of which turned out later to be useful in linear system theory [6, 14].

Codes on Graphs: Fundamentals

This paper develops a fundamental theory of realizations of linear and group codes on general graphs using elementary group theory, including basic group duality theory. Principal new and extended results include: normal realization duality; analysis of systems-theoretic properties of fragments of realizations and their connections; minimal Leftrightarrow trim and proper theorem for cycle-free codes; results showing that all constraint codes except interface nodes may be assumed to be trim and proper, and that the interesting part of a cyclic realization is its 2-core; notions of observability and controllability for fragments, and related tests; and relations between state-trimness and controllability, and dual state-trimness and observability.

Codes on graphs: recent progress

This paper surveys the field of “codes on graphs”, from its origins in the work of Gallager and Tanner, through its explosion in 1995–96 due to the rediscovery of LDPC codes and Wibergs thesis, and up to the present.