Donald S. Ornstein

Entropy and data compression schemes

Some new ways of defining the entropy of a process by observing a single typical output sequence as well as a new kind of Shannon-McMillan-Breiman theorem are presented. This provides a new and conceptually very simple ways of estimating the entropy of an ergodic stationary source as well as new insight into the workings of such well-known data compression schemes as the Lempel-Ziv algorithm. >

Billiards and Bernoulli schemes

Some two dimensional billiards are Bernoulli flows.

The Shannon-McMillan-Breiman theorem for a class of amenable groups

We prove the SMB theorem for amenable groups that possess Følner sets {An} with the property that for some constantM, and all,n, |An−1An| ≦M· |An|.

Geodesic flows are Bernoullian

A geometric method is developed for proving that transformations are isomorphic to Bernoulli shifts. The method is applied to the geodesic flows on surfaces of negative curvature and it is shown that they are isomorphic to Bernoulli flows.

Guessing the next output of a stationary process

Suppose we start watching a stationary process at time 0. Then the conditional probability of a particular output at time −1, given the outputs at times 0 throughk, will converge. In this paper we will show that we can make a guess, depending only on the outputs from 0 throughk (and not, of course, on the process) that will converge to the above limit with probability one.

On the Bernoulli nature of systems with some hyperbolic structure

It is shown that systems with hyperbolic structure have the Bernoulli property. Some new results on smooth cross-sections of hyperbolic Bernoulli flows are also derived. The proofs involve an abstract version of our original methods for showing that the geodesic flow on surfaces of negative curvature are Bernoulli.

Cutting and stacking, interval exchanges and geometric models

Every aperiodic measure-preserving transformation can be obtained by a cutting and stacking construction. It follows that all such transformations are infinite interval exchanges. This in turn is used to represent any ergodic measure-preserving flow as aC∞-flow on an open 2-manifold. Several additional applications of the basic theorems are also given.

Finitely determined implies very weak Bernoulli

It is shown that if a process is finitely determined then it is very weak Bernoulli (VWB). Combined with known results this says that a process is isomorphic to a Bernoulli shift if and only if it satisfies an asymptotic independence condition, namely that of being VWB.

Entropy and recurrence rates for stationary random fields

For a stationary random field {x(u): u /spl isin/ Z/sup d/}, the recurrence time R/sub n/(x) may be defined as the smallest positive k, such that the pattern {x(u): 0 /spl les/ u/sub i/ < n} is seen again, in a new position in the cube {0 /spl les/ |u/sub i/| < k}. In analogy with the case of d = 1, where the pioneering work was done by Wyner and Ziv (1989), we prove here that the asymptotic growth of R/sub n/(x) for ergodic fields is given by the entropy of the random field. The nonergodic case is also treated, as well as the recurrence times of central patterns in centered cubes. Both finite and countable state spaces are treated.

Sliding-block joint source/noisy-channel coding theorems

Sliding-block codes are nonblock coding structures consisting of discrete-time time-invariant possibly nonlinear filters. They are equivalent to time-invariant trellis codes. The coupling of Forneys rigorization of Shannons random-coding/typical-sequence approach to block coding theorems with the strong Rohlin-Kakutani Theorem of ergodic theory is used to obtain a sliding-block coding theorem for ergodic sources and discrete memoryless noisy channels. Combining this result with a theorem on sliding-block source coding with a fidelity criterion yields a sliding-block information transmission theorem. Thus, the basic existence theorems of information theory hold for stationary nonblock structures, as well as for block codes.