Rakesh K. Bansal

An algorithm for detecting a change in a stochastic process

The problem of detecting a change from one given stationary and ergodic stochastic process to another such process is considered. It is assumed that both stochastic processes are processes with memory and that they are mutually independent. A sequential test is proposed and analyzed. It is proved that the proposed test is asymptotically optimal in a mathematically precise sense.

Outlier-resistant algorithms for detecting a change in a stochastic process

Outlier-resistant algorithms that detect a change from a given nominal stationary process to another such process are given. The nominal processes are assumed to be mutually independent and to satisfy some general regularity conditions. The outlier sequences are assumed to be independently and identically distributed and independent of the nominal processes. The proposed algorithms are sequential and consist of uniformly bounded steps. The asymptotic performance of the algorithms is analyzed, both in the absence and the presence of outliers. Breakdown points and influence functions are defined and analyzed. The algorithms are studied in more detail for Gaussian autoregressive nominal processes. >

Sequential change detection based on universal compression algorithms

The application of universal source coding algorithms to the problem of classification was initiated by Ziv and has been extended to other problems in statistics such as order estimation. In spite of the large literature, these studies have been limited to problems with fixed number of samples. In this paper we study the application of universal source coding to the problem of sequential hypothesis testing and sequential change detection. Algorithms are proposed which are inspired by Waldpsilas Sequential Probability Ratio Test (SPRT) and Pagepsilas Cumulative Sum Test (CUSUM) for these problems respectively. Performance of the proposed algorithms are studied in the asymptotic regime to demonstrate their effectiveness.

On large deviation property of recurrence times

We extend the study by Ornstein and Weiss on the asymptotic behavior of the normalized version of recurrence times and establish the large deviation property for a certain class of mixing processes. Further, an estimator for entropy based on recurrence times is proposed for which large deviation behavior is proved for stationary and ergodic sources satisfying similar mixing conditions.

On the optimality of Sliding Window Lempel-Ziv algorithm with side information

Several universal coding algorithms including LZ-78, MPM and CTW have been extended to the case with side information in the information theory literature. In this paper, we consider the side information version of sliding window Lempel-Ziv algorithm introduced by Subrahmanya and Berger. We prove the pointwise optimality of the algorithm for a class of sources with exponential rates for entropy using the asymptotic properties of recurrence times. We also study the asymptotic properties of a quantity which we call recurrence counts, introduced by Subrahmanya and Berger and prove a pointwise result regarding its convergence.

On redundancy rate of FDLZ algorithm and its variants

We use the machinery developed by Wyner [1], for the sources satisfying Markov condition, to obtain an upper bound on the contribution of pointer bits to the compression ratio for fixed database Lempel-Ziv (FDLZ) algorithm to be H + O(1/ log2n) which is an improvement from the previous bound of H + H(1 + o(1))log2 log2n/ log2n . We use the definition of compression ratio as in Yang and Kieffer [2]. Here H is the entropy rate of the source and n is the size of the database. Then using the same definition of compression ratio we obtain an upper bound on the contribution of phrase length bits for the variant of FDLZ suggested in [3] to be O(1/ log2n), which gives an upper bound of O(1/ log2n) on the redundancy rate itself for this version of FDLZ.

Point-wise analysis of redundancy in SWLZ algorithm for φ-mixing sources

In this paper, we bound the number of phrases of the sliding window Lempel-Ziv (SWLZ) algorithm using an upper bound on the expected number of phrases in the fixed database Lempel-Ziv (FDLZ) algorithm for a class of φ-mixing sources which includes Markov sources, unifilar sources and finite state sources as special cases, as developed by Yang and Kieffer [1]. We use this bound to obtain a point-wise upper bound on the redundancy rate of SWLZ algorithm to be 2H(log₂log₂n_w/log₂n_w) + O(log₂log₂log₂n_w/log₂n_w). Here H is the entropy rate of the source and n_w is the window size.

On point-wise redundancy rate of Bender-Wolf's variant of SWLZ algorithm

In this paper we analyse the redundancy rate of a variant of sliding window Lempel-Ziv (SWLZ) proposed by Bender and Wolf, which encodes phrase lengths differently from the original algorithm. We examine upper bound on the contribution of phrase length bits for this variant to get an overall upper bound of O(1/log nw) on the redundancy rate for Markov sources which is better than the lower bound of O(log log nw/log nw), established by the Lastras-Montano for SWLZ algorithm. Here nw denotes the window size.

On Match Lengths, Zero Entropy, and Large Deviations—With Application to Sliding Window Lempel–Ziv Algorithm

The sliding window Lempel-Ziv (SWLZ) algorithm that makes use of recurrence times and match lengths has been studied from various perspectives in information theory literature. In this paper, we undertake a finer study of these quantities under two different scenarios: 1) zero entropy sources that are characterized by strong long-term memory and 2) the processes with weak memory as described through various mixing conditions. For zero entropy sources, a general statement on match length is obtained. It is used in the proof of almost sure optimality of fixed shift variant of Lempel-Ziv (FSLZ) and SWLZ algorithms given in literature. Through an example of stationary and ergodic processes generated by an irrational rotation, we establish that for a window of size nw, a compression ratio given by O(log nw/nwa), where a depends on nw and approaches 1 as nw → ∞, is obtained under the application of FSLZ and SWLZ algorithms. In addition, we give a general expression for the compression ratio for a class of stationary and ergodic processes with zero entropy. Next, we extend the study of Ornstein and Weiss on the asymptotic behavior of the normalized version of recurrence times and establish the large deviation property for a class of mixing processes. In addition, an estimator of entropy based on recurrence times is proposed for which large deviation principle is proved for sources satisfying similar mixing conditions.

Large deviation property of waiting times for Markov and mixing processes

In this work, we study the asymptotic properties of the waiting time until the opening string in the realization of a process first appears in an independent realization of the same or a different process. We first establish that the normalized waiting time between two independent realizations of a single source obeys the large deviation property for a class of mixing processes. Using the method of Markov types, we extend the result to when both the sequences are realizations of two distinct irreducible and aperiodic Markov sources.