Shunsuke Kanda

Compressed double-array tries for string dictionaries supporting fast lookup

A string dictionary is a basic tool for storing a set of strings in many kinds of applications. Recently, many applications need space-efficient dictionaries to handle very large datasets. In this paper, we propose new compressed string dictionaries using improved double-array tries. The double-array trie is a data structure that can implement a string dictionary supporting extremely fast lookup of strings, but its space efficiency is low. We introduce approaches for improving the disadvantage. From experimental evaluations, our dictionaries can provide the fastest lookup compared to state-of-the-art compressed string dictionaries. Moreover, the space efficiency is competitive in many cases.

Trie compact representation using double-array structures with string labels

A trie is an ordered tree structure with a character on each edge. The trie provides efficient storing and retrieval of a keyword set in processing strings. The double-array has been proposed to provide fast retrieval in the trie. As its space usage depends on the number of trie nodes, the space usage decreases by reducing nodes. To reduce the number of trie nodes, an Minimal-Prefix (MP) trie and a double-trie have been proposed, and the double-array can represent these data structures efficiently. On the other hand, the data structures include many nodes that can be reduced by giving a string label to each edge. However, the double-array with string labels has not been proposed. This paper proposes a new double-array with string labels by using multiple arrays depending on label sizes. Moreover, we show its effectiveness by experiments.

Practical rearrangement methods for dynamic double‐array dictionaries

Double‐array structures have been widely used to implement dictionaries with string keys. Although the space efficiency of dynamic double‐array dictionaries tends to decrease with key updates, we can still maintain high efficiency using existing methods. However, these methods have practical problems of time and functionality. This paper presents several efficient rearrangement methods to solve these problems. Through experiments using real‐world datasets, we demonstrate that the proposed rearrangement methods are much more practical than existing methods.

Practical Implementation of Space-Efficient Dynamic Keyword Dictionaries

A keyword dictionary is an associative array with string keys. Although it is a classical data structure, recent applications require the management of massive string data using the keyword dictionary in main memory. Therefore, its space-efficient implementation is very important. If limited to static applications, there are a number of very compact dictionary implementations; however, existing dynamic implementations consume much larger space than static ones. In this paper, we propose a new practical implementation of space-efficient dynamic keyword dictionaries. Our implementation uses path decomposition, which is proposed for constructing cache-friendly trie structures, for dynamic construction in compact space with a different approach. Using experiments on real-world datasets, we show that our implementation can construct keyword dictionaries in spaces up to 2.8x smaller than the most compact existing dynamic implementation.

A construction method by divided double array structures

When keywords are retrieved, speed and compactness are important. A trie is the data structure for keyword retrieval. Also, the double array is a key retrieval method by the trie and has speed and compactness. Some construction methods for the double array are proposed, but the problem is that construction time is exponentially late with large key set. To solve the problem, two methods to divide and construct the double array depending on depth are proposed. The proposed methods are confirmed to construct the double array fast.

A new compression method for double-array structures by a hierarchical representation

To store and retrieve keyword sets, a trie that is a tree structure is utilised in many applications for processing strings. The double-array and level-order unary degree sequence LOUDS efficiently represent the trie. The double-array provides fast retrieval for the trie, but its space usage is not so compact. On the other hand, LOUDS represents the trie compactly, but its retrieval speed is not so fast. This paper presents a new compression method for the double-array. Our new method represents the double-array by a hierarchical structure and changes allocations of the double-array. Theoretical observations show that the new method reduces the space usage of the double-array to ∼60%. Moreover, experimental results for English keywords show that the new method reduces the space usage of the double-array to ∼60-62% without impairing the high-speed performance. The retrieval speed of the new method is ∼17-24 times faster than that of LOUDS.