Travis Gagie

Alphabet Partitioning for Compressed Rank/Select and Applications

We present a data structure that stores a string s[1..n] over the alphabet [1..σ] in nH 0(s) + o(n)(H 0(s) + 1) bits, where H 0(s) is the zero-order entropy of s. This data structure supports the queries access and rank in time (({mathcal O}{{rm lg lg}sigma})), and the select query in constant time. This result improves on previously known data structures using (nH_0(s)+o(nlgsigma)) bits, where on highly compressible instances the redundancy (o(nlgsigma)) cease to be negligible compared to the nH 0(s) bits that encode the data. The technique is based on combining previous results through an ingenious partitioning of the alphabet, and practical enough to be implementable. It applies not only to strings, but also to several other compact data structures. For example, we achieve (i) faster search times and lower redundancy for the smallest existing full-text self-index; (ii) compressed permutations π with times for π() and π − 1() improved to log-logarithmic; and (iii) the first compressed representation of dynamic collections of disjoint sets.

Lightweight data indexing and compression in external memory

In this paper we describe algorithms for computing the BWT and for building (compressed) indexes in external memory. The innovative feature of our algorithms is that they are lightweight in the sense that, for an input of size n, they use only n bits of disk working space while all previous approaches use Θ(n logn) bits of disk working space. Moreover, our algorithms access disk data only via sequential scans, thus they take full advantage of modern disk features that make sequential disk accesses much faster than random accesses. n nWe also present a scan-based algorithm for inverting the BWT that uses Θ(n) bits of working space, and a lightweight internal-memory algorithm for computing the BWT which is the fastest in the literature when the available working space is o(n) bits. n nFinally, we prove lower bounds on the complexity of computing and inverting the BWT via sequential scans in terms of the classic product: internal-memory space × number of passes over the disk data, showing that our algorithms are within an O(logn) factor of the optimal.

Competitive Boolean function evaluation: Beyond monotonicity, and the symmetric case

We study the extremal competitive ratio of Boolean function evaluation. We provide the first non-trivial lower and upper bounds for classes of Boolean functions which are not included in the class of monotone Boolean functions. For the particular case of symmetric functions our bounds are matching and we exactly characterize the best possible competitiveness achievable by a deterministic algorithm. Our upper bound is obtained by a simple polynomial time algorithm.

Grammar-based compression in a streaming model

We show that, given a string s of length n, with constant memory and logarithmic passes over a constant number of streams we can build a context-free grammar that generates s and only s and whose size is within an

Fast and Compact Prefix Codes

{mathcal O}left({min left( g log g, sqrt{n / log n} right)}right)

A better bouncer's algorithm

-factor of the minimum g. This stands in contrast to our previous result that, with polylogarithmic memory and polylogarithmic passes over a single stream, we cannot build such a grammar whose size is within any polynomial of g.

Toward a Succinct Index for Order-Preserving Pattern Matching.

It is well-known that, given a probability distribution over n characters, in the worst case it takes ?(n logn) bits to store a prefix code with minimum expected codeword length. However, in this paper we first show that, for any ? with 0 1, it takes O(n 1 / c logn) bits to store a prefix code with expected codeword length at most c times the minimum. In both cases, our data structures allow us to encode and decode any character in O(1) time.

Compressed Spaced Suffix Arrays.

Suppose we have a set of materials -- e.g., drugs or genes -- some combinations of which react badly together. We can experiment to see whether subsets contain any bad combinations and we want to find a maximal subset that does not. This problem is equivalent to finding a maximal independent set (or minimal vertex cover) in a hypergraph using group tests on the vertices. Consider the simple greedy algorithm that adds vertices one by one; after adding each vertex, the algorithm tests whether the subset now contains any edges and, if so, removes that vertex and discards it. We call this the bouncers algorithm because it is reminiscent of how order is maintained as patrons are admitted to some bars. If this algorithm processes the vertices according to a given total preference order, then its solution is the unique optimum with respect to that order. Our main contribution is another algorithm that produces the same solution but uses fewer tests when few vertices are discarded: if the bouncers algorithm discards d of the n vertices in the hypergraph, then our algorithm uses at most d(⌈log2 n⌉+1)+1 tests. It follows that, given black-box access to a monotone Boolean formula on n variables, we can find a minimal satisfying truth assignment using at most d(⌈log2 n⌉ +1)+1 tests, where d is the number of variables set to true. We also prove some bounds for partially adaptive algorithms.