Matthew B. Kennel

LYAPUNOV EXPONENTS IN CHAOTIC SYSTEMS: THEIR IMPORTANCE AND THEIR EVALUATION USING OBSERVED DATA

We review the idea of Lyapunov exponents for chaotic systems and discuss their evaluation from observed data alone. These exponents govern the growth or decrease of small perturbations to orbits of a dynamical system. They are critical to the predictability of models made from observations as well as known analytic models. The Lyapunov exponents are invariants of the dynamical system and are connected with the dimension of the system attractor and to the idea of information generation by the system dynamics. Lyapunov exponents are among the many ways we can classify observed nonlinear systems, and their appeal to physicists remains their clear interpretation in terms of system stability and predictability. We discuss the familiar global Lyapunov exponents which govern the evolution of perturbations for long times and local Lyapunov exponents which determine the predictability over a finite number of time steps.

Variation of Lyapunov exponents on a strange attractor

SummaryWe introduce the idea of local Lyapunov exponents which govern the way small perturbations to the orbit of a dynamical system grow or contract after afinite number of steps,L, along the orbit. The distributions of these exponents over the attractor is an invariant of the dynamical system; namely, they are independent of the orbit or initial conditions. They tell us the variation of predictability over the attractor. They allow the estimation of extreme excursions of perturbations to an orbit once we know the mean and moments about the mean of these distributions. We show that the variations about the mean of the Lyapunov exponents approach zero asL → ∞ and argue from our numerical work on several chaotic systems that this approach is asL−v. In our examplesv ≈ 0.5–1.0. The exponents themselves approach the familiar Lyapunov spectrum in this same fashion.

Local Lyapunov exponents computed from observed data

SummaryWe develop methods for determining local Lyapunov exponents from observations of a scalar data set. Using average mutual information and the method of false neighbors, we reconstruct a multivariate time series, and then use local polynomial neighborhood-to-neighborhood maps to determine the phase space partial derivatives required to compute Lyapunov exponents. In several examples we demonstrate that the methods allow one to accurately reproduce results determined when the dynamics is known beforehand. We present a new recursive QR decomposition method for finding the eigenvalues of products of matrices when that product is severely ill conditioned, and we give an argument to show that local Lyapunov exponents are ambiguous up to order 1/L in the number of steps due to the choice of coordinate system. Local Lyapunov exponents are the critical element in determining the practical predictability of a chaotic system, so the results here will be of some general use.

Synchronization and communication using semiconductor lasers with optoelectronic feedback

Semiconductor lasers provide an excellent opportunity for communication using chaotic waveforms. We discuss the characteristics and the synchronization of two semiconductor lasers with optoelectronic feedback. The systems exhibit broadband chaotic intensity oscillations whose dynamical dimension generally increases with the time delay in the feedback loop. We explore the robustness of this synchronization with parameter mismatch in the lasers, with mismatch in the optoelectronic feedback delay, and with the strength of the coupling between the systems. Synchronization is robust to mismatches between the intrinsic parameters of the lasers, but it is sensitive to mismatches of the time delay in the transmitter and receiver feedback loops. An open-loop receiver configuration is suggested, eliminating feedback delay mismatch issues. Communication strategies for arbitrary amplitude of modulation onto the chaotic signals are discussed, and the bit-error rate for one such scheme is evaluated as a function of noise in the optical channel.

Estimating Entropy Rates with Bayesian Confidence Intervals

The entropy rate quantifies the amount of uncertainty or disorder produced by any dynamical system. In a spiking neuron, this uncertainty translates into the amount of information potentially encoded and thus the subject of intense theoretical and experimental investigation. Estimating this quantity in observed, experimental data is difficult and requires a judicious selection of probabilistic models, balancing between two opposing biases. We use a model weighting principle originally developed for lossless data compression, following the minimum description length principle. This weighting yields a direct estimator of the entropy rate, which, compared to existing methods, exhibits significantly less bias and converges faster in simulation. With Monte Carlo techinques, we estimate a Bayesian confidence interval for the entropy rate. In related work, weap-ply these ideas to estimate the information rates between sensory stimuli and neural responses in experimental data (Shlens, Kennel, Abarbanel, & Chichilnisky, in preparation).

The permutation entropy rate equals the metric entropy rate for ergodic information sources and ergodic dynamical systems

Abstract Permutation entropy quantifies the diversity of possible orderings of the values a random or deterministic system can take, as Shannon entropy quantifies the diversity of values. We show that the metric and permutation entropy rates—measures of new disorder per new observed value—are equal for ergodic finite-alphabet information sources (discrete-time stationary stochastic processes). With this result, we then prove that the same holds for deterministic dynamical systems defined by ergodic maps on n -dimensional intervals. This result generalizes a previous one for piecewise monotone interval maps on the real line [C. Bandt, G. Keller, B. Pompe, Entropy of interval maps via permutations, Nonlinearity 15 (2002) 1595–1602.] at the expense of requiring ergodicity and using a definition of permutation entropy rate differing modestly in the order of two limits. The case of non-ergodic finite-alphabet sources is also studied and an inequality developed. Finally, the equality of permutation and metric entropy rates is extended to ergodic non-discrete information sources when entropy is replaced by differential entropy in the usual way.

Estimating Good Discrete Partitions from Observed Data: Symbolic False Nearest Neighbors

A symbolic analysis of observed time series data typically requires making a discrete partition of a continuous state space containing observations of the dynamics. A particular kind of partition, called “generating”, preserves all dynamical information of a deterministic map in the symbolic representation, but such partitions are not obvious beyond one dimension, and existing methods to find them require significant knowledge of the dynamical evolution operator or the spectrum of unstable periodic orbits. We introduce a statistic and algorithm to refine empirical partitions for symbolic state reconstruction. This method optimizes an essential property of a generating partition: avoiding topological degeneracies. It requires only the observed time series and is sensible even in the presence of noise when no truly generating partition is possible. Because of its resemblance to a geometrical statistic frequently used for reconstructing valid time‐delay embeddings, we call the algorithm “symbolic false nearest neighbors”.

Forbidden patterns and shift systems

The scope of this paper is two-fold. First, to present to the researchers in combinatorics an interesting implementation of permutations avoiding generalized patterns in the framework of discrete-time dynamical systems. Indeed, the orbits generated by piecewise monotone maps on one-dimensional intervals have forbidden order patterns, i.e., order patterns that do not occur in any orbit. The allowed patterns are then those patterns avoiding the so-called forbidden root patterns and their shifted patterns. The second scope is to study forbidden patterns in shift systems, which are universal models in information theory, dynamical systems and stochastic processes. Due to its simple structure, shift systems are accessible to a more detailed analysis and, at the same time, exhibit all important properties of low-dimensional chaotic dynamical systems (e.g., sensitivity to initial conditions, strong mixing and a dense set of periodic points), allowing to export the results to other dynamical systems via order-isomorphisms.

Estimating Information Rates with Confidence Intervals in Neural Spike Trains

Information theory provides a natural set of statistics to quantify the amount of knowledge a neuron conveys about a stimulus. A related work (Kennel, Shlens, Abarbanel, & Chichilnisky, 2005) demonstrated how to reliably estimate, with a Bayesian confidence interval, the entropy rate from a discrete, observed time series. We extend this method to measure the rate of novel information that a neural spike train encodes about a stimulusthe average and specific mutual information rates. Our estimator makes few assumptions about the underlying neural dynamics, shows excellent performance in experimentally relevant regimes, and uniquely provides confidence intervals bounding the range of information rates compatible with the observed spike train. We validate this estimator with simulations of spike trains and highlight how stimulus parameters affect its convergence in bias and variance. Finally, we apply these ideas to a recording from a guinea pig retinal ganglion cell and compare results to a simple linear decoder.

Shaping current waveforms for direct modulation of semiconductor lasers

We demonstrate a technique for shaping current inputs for the direct modulation of a semiconductor laser for digital communication. The introduction of shaped current inputs allows for the suppression of relaxation oscillations and the avoidance of dynamical memory in the physical laser device, i.e., the output will not be influenced by previously communicated information. For the example of time-optimized bits, the possible performance enhancement for high data rate communications is shown numerically.