Shirou Maruyama

An Online Algorithm for Lightweight Grammar-Based Compression

Grammar-based compression is a well-studied technique for constructing a small context-free grammar (CFG) uniquely deriving a given text. In this paper, we present an online algorithm for lightweight grammar-based compression. Our algorithm is based on the LCA algorithm [Sakamoto et al. 2004]which guarantees nearly optimum compression ratio and space. LCA, however, is an offline algorithm and requires external space to save space consumption. Therefore, we present its online version which inherits most characteristics of the original LCA. Our algorithm guarantees

Context-Sensitive Grammar Transform: Compression and Pattern Matching

O(\log^2 n)

ESP-index: a compressed index based on edit-sensitive parsing

-approximation ratio for an optimum grammar size, and all work is carried out on a main memory space which is bounded by the output size. In addition, we propose more practical encoding based on parentheses representation of a binary tree. Experimental results for repetitive texts demonstrate that our algorithm achieves effective compression compared to other practical compressors and the space consumption of our algorithm is smaller than the input text size.

Improving time and space complexity for compressed pattern matching

A framework of context-sensitive grammar transform is proposed. A greedy compression algorithm with the transform model is presented as well as a Knuth-Morris-Pratt (KMP)-type compressed pattern matching (CPM) algorithm. The compression performance is a match for gzip and Re-Pair. The search speed of our CPM algorithm is almost twice faster than the KMP type CPM algorithm on Byte-Pair-Encoding by Shibata et al. (2000), and in the case of short patterns, faster than the Boyer-Moore-Horspool algorithm with the stopper encoding by Rautio et al. (2002), which is regarded as one of the best combinations that allows a practically fast search.

Scalable detection of frequent substrings by grammar-based compression

We propose a compressed self-index based the edit-sensitive parsing (ESP). Given a string S, its ESP tree is equivalent to a contextfree grammar deriving just S, which can be represented as a DAG G. Finding pattern P in S is reduced to embedding P into G. Succinct data structures are adopted and G is then decomposed into two LOUDS bit strings and a single array for permutation, requiring (1 + e)n log n + 4n + o(n) bits for any 0 < e < 1 where n corresponds to the number of different symbols in the grammar. The time to count the occurrences of P in S is in O(log*u/e (mlog n+occc(logmlog u))), where m = |P|, u = |S|, and occc is the number of occurrences of a maximal common subtree in ESP trees of P and S. Using an additional array in n log u bits of space, our index supports locating P and displaying substring of S. Locating time is the same as counting time and displaying time for a substring of length m is O(m + log u).

A Space-Saving Approximation Algorithm for Grammar-Based Compression

The compressed pattern matching problem is to find all occurrences of a given pattern in a compressed text. In this paper an efficient grammar-based compression algorithm is presented for the compressed pattern matching. The algorithm achieves the worst-case approximation ratio O(g*logg*logn) for the optimum grammar size g* with an input text of length n. This upper bound improves the complexity of the compressed pattern matching problem to

ESP-index: A compressed index based on edit-sensitive parsing

O(g_*\log g_*\log m + \frac{n}{m} + m^2 + r)

Fully-Online Grammar Compression

time and O(g*logg*logm + m2) space for any pattern shorter than m and the number r of pattern occurrences.

Fully Online Grammar Compression in Constant Space

A scalable pattern discovery by compression is proposed. A string is representable by a context-free grammar (CFG) deriving the string deterministically. In this framework of grammar-based compression, the aim of the algorithm is to output as small a CFG as possible. Beyond that, the optimization problem is approximately solvable. In such approximation algorithms, the compressor by Sakamoto et al. (2009) is especially suitable for detecting maximal common substrings as well as long frequent substrings. This is made possible thanks to the characteristics of edit-sensitive parsing (ESP) by Cormode and Muthukrishnan (2007), which was introduced to approximate a variant of edit distance. Based on ESP, we design a linear time algorithm to find all frequent patterns in a string approximately and prove a lower bound for the length of extracted frequent patterns. We also examine the performance of our algorithm by experiments in DNA sequences and other compressible real world texts. Compared to the practical algorithm developed by Uno (2008), our algorithm is faster with large and repetitive strings.