José M. Amigó

Estimating the Entropy Rate of Spike Trains via Lempel-Ziv Complexity

Normalized Lempel-Ziv complexity, which measures the generation rate of new patterns along a digital sequence, is closely related to such important source properties as entropy and compression ratio, but, in contrast to these, it is a property of individual sequences. In this article, we propose to exploit this concept to estimate (or, at least, to bound from below) the entropy of neural discharges (spike trains). The main advantages of this method include fast convergence of the estimator (as supported by numerical simulation) and the fact that there is no need to know the probability law of the process generating the signal. Furthermore, we present numerical and experimental comparisons of the new method against the standard method based on word frequencies, providing evidence that this new approach is an alternative entropy estimator for binned spike trains.

The permutation entropy rate equals the metric entropy rate for ergodic information sources and ergodic dynamical systems

Abstract Permutation entropy quantifies the diversity of possible orderings of the values a random or deterministic system can take, as Shannon entropy quantifies the diversity of values. We show that the metric and permutation entropy rates—measures of new disorder per new observed value—are equal for ergodic finite-alphabet information sources (discrete-time stationary stochastic processes). With this result, we then prove that the same holds for deterministic dynamical systems defined by ergodic maps on n -dimensional intervals. This result generalizes a previous one for piecewise monotone interval maps on the real line [C. Bandt, G. Keller, B. Pompe, Entropy of interval maps via permutations, Nonlinearity 15 (2002) 1595–1602.] at the expense of requiring ergodicity and using a definition of permutation entropy rate differing modestly in the order of two limits. The case of non-ergodic finite-alphabet sources is also studied and an inequality developed. Finally, the equality of permutation and metric entropy rates is extended to ergodic non-discrete information sources when entropy is replaced by differential entropy in the usual way.

Discrete Chaos-I: Theory

We propose a definition of the discrete Lyapunov exponent for an arbitrary permutation of a finite lattice. For discrete-time dynamical systems, it measures the local (between neighboring points) average spreading of the system. We justify our definition by proving that, for large classes of chaotic maps, the corresponding discrete Lyapunov exponent approaches the largest Lyapunov exponent of a chaotic map when Mrarrinfin, where M is the cardinality of the discrete phase space. In analogy with continuous systems, we say the system has discrete chaos if its discrete Lyapunov exponent tends to a positive number, when Mrarrinfin. We present several examples to illustrate the concepts being introduced

A Connection Between Chaotic and Conventional Cryptography

Many encryption methods involving chaotic dynamics have been proposed in the literature since the 1990s. Most of them consist of ldquomixingrdquo the confidential information being transmitted through an insecure channel, with a chaotic analog or digital sequence. The recovering of the original information usually calls for reproducing, at the receiver side, the same chaotic signal as at the transmitter side. The synchronization mechanism of the two chaotic signals is known as chaos synchronization. In this paper, a connection between chaotic and conventional encryption is established with special emphasis on two of the most attractive schemes, namely, message embedding and hybrid message embedding. The main point can be stated as follows: the (hybrid) message-embedded cryptosystem is equivalent to a conventional self-synchronizing stream cipher under flatness conditions.

Characterizing spike trains with Lempel–Ziv complexity

Abstract We review several applications of Lempel–Ziv complexity to the characterization of neural responses. In particular, Lempel–Ziv complexity allows to estimate the entropy of binned spike trains in an alternative way to the usual method based on the relative frequencies of words, with the definitive advantage of no requiring very long registers. We also use complexity to discriminate neural responses to different kinds of stimuli and to evaluate the number of states of neuronal sources.

Information directionality in coupled time series using transcripts.

In ordinal symbolic dynamics, transcripts describe the algebraic relationship between ordinal patterns. Using the concept of transcript, we exploit the mathematical structure of the group of permutations to derive properties and relations among information measures of the symbolic representations of time series. These theoretical results are then applied for the assessment of coupling directionality in dynamical systems, where suitable coupling directionality measures are introduced depending only on transcripts. These measures improve the reliability of the information flow estimates and reduce to well-established coupling directionality quantifiers when some general conditions are satisfied. Furthermore, by generalizing the definition of transcript to ordinal patterns of different lengths, several of the commonly used information directionality measures can be encompassed within the same framework.

Application of Lempel–Ziv complexity to the analysis of neural discharges

Pattern matching is a simple method for studying the properties of information sources based on individual sequences (Wyner et al 1998 IEEE Trans. Inf. Theory 44 2045–56). In particular, the normalized Lempel–Ziv complexity (Lempel and Ziv 1976 IEEE Trans. Inf. Theory 22 75–88), which measures the rate of generation of new patterns along a sequence, is closely related to such important source properties as entropy and information compression ratio. We make use of this concept to characterize the responses of neurons of the primary visual cortex to different kinds of stimulus, including visual stimulation (sinusoidal drifting gratings) and intracellular current injections (sinusoidal and random currents), under two conditions (in vivo and in vitro preparations). Specifically, we digitize the neuronal discharges with several encoding techniques and employ the complexity curves of the resulting discrete signals as fingerprints of the stimuli ensembles. Our results show, for example, that if the neural discharges are encoded with a particular one-parameter method (‘interspike time coding’), the normalized complexity remains constant within some classes of stimuli for a wide range of the parameter. Such constant values of the normalized complexity allow then the differentiation of the stimuli classes. With other encodings (e.g. ‘bin coding’), the whole complexity curve is needed to achieve this goal. In any case, it turns out that the normalized complexity of the neural discharges in vivo are higher (and hence carry more information in the sense of Shannon) than in vitro for the same kind of stimulus.

Cryptographically secure substitutions based on the approximation of mixing maps

In this paper, we explore, following Shannons suggestion that diffusion should be one of the ingredients of resistant block ciphers, the feasibility of designing cryptographically secure substitutions (think of S-boxes, say) via approximation of mixing maps by periodic transformations. The expectation behind this approach is, of course, that the nice diffusion properties of such maps will be inherited by their approximations, at least if the convergence rate is appropriate and the associated partitions are sufficiently fine. Our results show that this is indeed the case and that, in principle, block ciphers with close-to-optimal immunity to linear and differential cryptanalysis (as measured by the linear and differential approximation probabilities) can be designed along these guidelines. We provide also practical examples and numerical evidence for this approximation philosophy.

Transcripts: An algebraic approach to coupled time series

Ordinal symbolic dynamics is based on ordinal patterns. Its tools include permutation entropy (in metric and topological versions), forbidden patterns, and a number of mathematical results that make this sort of symbolic dynamics appealing both for theoreticians and practitioners. In particular, ordinal symbolic dynamics is robust against observational noise and can be implemented with low computational cost, which explains its increasing popularity in time series analysis. In this paper, we study the perhaps less exploited aspect so far of ordinal patterns: their algebraic structure. In a first part, we revisit the concept of transcript between two symbolic representations, generalize it to N representations, and derive some general properties. In a second part, we use transcripts to define two complexity indicators of coupled dynamics. Their performance is tested with numerical and real world data.

Inhibition in multiclass classification

The role of inhibition is investigated in a multiclass support vector machine formalism inspired by the brain structure of insects. The so-called mushroom bodies have a set of output neurons, or classification functions, that compete with each other to encode a particular input. Strongly active output neurons depress or inhibit the remaining outputs without knowing which is correct or incorrect. Accordingly, we propose to use a classification function that embodies unselective inhibition and train it in the large margin classifier framework. Inhibition leads to more robust classifiers in the sense that they perform better on larger areas of appropriate hyperparameters when assessed with leave-one-out strategies. We also show that the classifier with inhibition is a tight bound to probabilistic exponential models and is Bayes consistent for 3-class problems. These properties make this approach useful for data sets with a limited number of labeled examples. For larger data sets, there is no significant comparative advantage to other multiclass SVM approaches.