Alvaro Martin

Type Classes of Tree Models

It is well known that a tree model does not always admit a finite-state machine (FSM) representation with the same (minimal) number of parameters. Therefore, known characterizations of type classes for FSMs do not apply, in general, to tree models. In this paper, the type class of a string with respect to a tree model is studied, and an exact formula is derived for the size of the class. The formula, which applies to arbitrary trees, generalizes Whittles formula for FSMs. The derivation is more intricate than the FSM case, since some basic properties of FSM types do not hold in general for tree-model types. The derivation also yields an efficient enumeration of the tree-model type class, which has applications in universal data compression and universal simulation. A formula for the number of type classes with respect to a given tree is also derived. The formula is asymptotically tight up to multiplication by a constant and also generalizes a known result for FSMs.

Bounds on estimated Markov orders of individual sequences

We study the maximal values estimated by commonly used Markov model order estimators on individual sequences. We start with penalized maximum likelihood (PML) estimators with cost functions of the form - log Pk(xn) + f (n)αk, where Pk (xn) is the ML probability of the input sequence xn under a Markov model of order k, a is the size of the input alphabet, and f(n) is an increasing (penalization) function of n (the popular BIC estimator corresponds to f(n) = α - 1/2 log n). Comparison with a memoryless model yields a known upper bound k(n) on the maximum order that xn can estimate. We show that, under mild conditions on f that are satisfied by commonly used penalization functions, this simple bound is not far from tight, in the following sense: for sufficiently large n, and any k<;k̅(n), there are sequences xn that estimate order k; moreover, for all but a vanishing fraction of the values of n such that k = k̅(n), there are sequences xn that estimate order k. We also study KT-based MDL Markov order estimators, and show that in this case, there are sequences xn that estimate order n1/2-ϵ, which is much larger than the maximum log n/log α(l + o(1)) attainable by BIC, or the order o(log n) required for consistency of the KT estimator. In fact, for these sequences, limiting the allowed estimated order might incur in a significant asymptotic penalty in description length. All the results are constructive, and in each case we exhibit explicit sequences that attain the claimed estimated orders.

Type Classes of Context Trees

It is well known that a tree model does not always admit a finite-state machine (FSM) representation with the same (minimal) number of parameters. Therefore, known characterizations of type classes for FSMs do not apply, in general, to tree models. In this paper, the type class of a sequence with respect to a given context tree is studied. An exact formula is derived for the size of the class, extending Whittles formula for type classes with respect to FSMs. The derivation is more intricate than in the FSM case, since some basic properties of FSM types do not hold in general for tree types. The derivation also yields an efficient enumeration of the tree type class. A formula for the number of type classes with respect to is also derived. The formula is asymptotically tight up to a multiplicative constant and also extends the corresponding result for FSMs. The asymptotic behavior of the number of type classes, and of the size of a class, is expressed in terms of the so-called minimal canonical extension of T, a tree that is generally larger than but smaller than its FSM closure.

Twice-Universal Simulation of Markov Sources and Individual Sequences

The problem of universal simulation given a training sequence is studied both in a stochastic setting and for individual sequences. In the stochastic setting, the training sequence is assumed to be emitted by a Markov source of unknown order, extending previous work where the order is assumed known and leading to the notion of twice-universal simulation. A simulation scheme, which partitions the set of sequences of a given length into classes, is proposed for this setting and shown to be asymptotically optimal. This partition extends the notion of type classes to the twice-universal setting. In the individual sequence scenario, the same simulation scheme is shown to generate sequences which are statistically similar, in a strong sense, to the training sequence, for statistics of any order, while essentially maximizing the uncertainty on the output.

Space-efficient representation of truncated suffix trees, with applications to Markov order estimation

Suffix trees (ST) are useful in many text processing applications, for example, to determine the number of occurrences of patterns of arbitrary length in an input string x. If the length n, of x, is large, the memory required to represent the ST may become a practical performance bottleneck. This problem can be alleviated, in cases where a nontrivial upper bound is known on the lengths of the patterns of interest, by using a truncated ST (TST). However, conventional TST implementations still require ? ( n ) bits of memory, since they store x. We describe a new TST representation that avoids this limitation by storing all the information necessary to reconstruct the TST edge labels in a string y that is often much shorter than x. We apply TSTs to the implementation of Markov order estimators, where an upper bound k n on the estimated order can be derived or it is imposed (for consistency, for example). The new representation allows for estimator implementations with sublinear space complexity in some cases of interest. In other cases we show, experimentally, that even when the new representation does not have an asymptotic advantage, it still achieves very significant memory savings in practice.

Efficient Sequential Compression of Multichannel Biomedical Signals

This paper proposes lossless and near-lossless compression algorithms for multichannel biomedical signals. The algorithms are sequential and efficient, which makes them suitable for low-latency and low-power signal transmission applications. We make use of information theory and signal processing tools (such as universal coding, universal prediction, and fast online implementations of multivariate recursive least squares), combined with simple methods to exploit spatial as well as temporal redundancies typically present in biomedical signals. The algorithms are tested with publicly available electroencephalogram and electrocardiogram databases, surpassing in all cases the current state of the art in near-lossless and lossless compression ratios.

Universal Enumerative Coding for Tree Models

Efficient enumerative coding for tree sources is, in general, surprisingly intricate-a simple uniform encoding of type classes, which is asymptotically optimal in expectation for many classical models, such as FSMs, turns out not to be so in this case. We describe an efficiently computable enumerative code that is universal in the family of tree models in the sense that, for a string emitted by an unknown source whose model is supported on a known tree, the expected normalized code length of the encoding approaches the entropy rate of the source with a convergence rate (K/2)(log n)/n, where K is the number of free parameters of the model family. Based on recent results characterizing type classes of context trees, the code consists of the index of the sequence in the tree type class, and an efficient description of the class itself using a nonuniform encoding of selected string counts. The results are extended to a twice-universal setting, where the tree underlying the source model is unknown.

Asymptotically tight bounds on the depth of estimated context trees

We study the maximum depth of context tree estimates, i.e., the maximum Markov order attainable by an estimated tree model given an (individual) input sequence of length n. We consider two classes of estimators: 1) Penalized maximum likelihood (PML) estimators where a context tree T is obtained by minimizing a cost of the form - log P T (xn)+f(n)|S|, where P ;T (xn) is the ML probability of the input sequence x n under a tree model T, S T is the set of states defined by T, and f(n) is an increasing (penalization) function of n (the popular BIC estimator corresponds to f(n) = α−1/2 log n where α is the size of the input alphabet). 2) MDL estimators based on the KT probability assignment. In each case we derive an asymptotic upper bound, n1/2+o(1), on the estimated depth, and we exhibit explicit input sequences that asymptotically attain the bound up to the term o(1) in the exponent.

Wearable EEG via lossless compression

This work presents a wearable multi-channel EEG recording system featuring a lossless compression algorithm. The algorithm, based in a previously reported algorithm by the authors, exploits the existing temporal correlation between samples at different sampling times, and the spatial correlation between different electrodes across the scalp. The low-power platform is able to compress, by a factor between 2.3 and 3.6, up to 300sps from 64 channels with a power consumption of 176μW/ch. The performance of the algorithm compares favorably with the best compression rates reported up to date in the literature.This work presents a wearable multi-channel EEG recording system featuring a lossless compression algorithm. The algorithm, based in a previously reported algorithm by the authors, exploits the existing temporal correlation between samples at different sampling times, and the spatial correlation between different electrodes across the scalp. The low-power platform is able to compress, by a factor between 2.3 and 3.6, up to 300sps from 64 channels with a power consumption of 176μW/ch. The performance of the algorithm compares favorably with the best compression rates reported up to date in the literature.

EEG Signal Pre-Processing for the P300 Speller

One of the workhorses of Brain Computer Interfaces (BCI) is the P300 speller, which allows a person to spell text by looking at the corresponding letters that are laid out on a flashing grid. The device functions by detecting the Event Related Potentials (ERP), which can be measured in an electroencephalogram (EEG), that occur when the letter that the subject is looking at flashes (unexpectedly). In this work, after a careful analysis of the EEG signals involved, we propose a preprocessing method that allows us to improve on the state-of-the-art results for this kind of applications. Our results are comparable, and sometimes better, than the best results published, and do not require a feature (channel) selection step, which is extremely costly, and which must be adapted to each user of the P300 speller separately.