-Ping Chung

Inverted file compression through document identifier reassignment

The inverted file is the most popular indexing mechanism for document search in an information retrieval system. Compressing an inverted file can greatly improve document search rate. Traditionally, the d-gap technique is used in the inverted file compression by replacing document identifiers with usually much smaller gap values. However, fluctuating gap values cannot be efficiently compressed by some well-known prefix-free codes. To smoothen and reduce the gap values, we propose a document-identifier reassignment algorithm. This reassignment is based on a similarity factor between documents. We generate a reassignment order for all documents according to the similarity to reassign closer identifiers to the documents having closer relationships. Simulation results show that the average gap values of sample inverted files can be reduced by 30%, and the compression rate of d-gapped inverted file with prefix-free codes can be improved by 15%.

ETAHM: an energy-aware task allocation algorithm for heterogeneous multiprocessor

In demand of more computing power and less energy use, multiprocessor with power management facility emerges in embedded system design. Dynamic voltage scaling is such a facility that varies clock speed and supply voltage to save more energy. In this paper, we propose ETAHM to allocate tasks on a target multiprocessor system. In pursuit of global optimal solution, it mixes task scheduling, mapping and DVS utilization in one phase and couples ant colony optimization algorithm. Extensive experiments show ETAHM could save 22.71% more energy than CASPER (V. Kianzad et al., 2005), a state-of-the-art integrated framework that tackles the identical problem with genetic algorithm instead.

A statistics-based approach to incrementally update inverted files

Many information retrieval systems use the inverted file as indexing structure. The inverted file, however, requires inefficient reorganization when new documents are to be added to an existing collection. Most studies suggest dealing with this problem by sparing free space in an inverted file for incremental updates. In this paper, we propose a run-time statistics-based approach to allocate the spare space. This approach estimates the space requirements in an inverted file using only a little most recent statistical data on space usage and document update request rate. For best indexing speed and space efficiency, the amount of the spare space to be allocated is determined by adaptively balancing the trade-offs between reorganization reduction and space utilization. Experiment results show that the proposed space-sparing approach significantly avoids reorganization in updating an inverted file, and in the meantime, unused free space can be well controlled such that the file access speed is not affected.

A Hierarchical Primitive Lists Structure for Tile-Based Rendering

Tile-based rendering has been widely used inresource-limited graphic processing environments, e.g., forhand-held devices. Since large primitives may cover a significantnumber of tiles, they need to be recorded in the primitive lists ofall related tiles. We propose a hierarchical primitive listsstructure, which also copes with misaligned and non-squareprimitive problems, to minimize the primitive recording.Intended advantages include: reduced storage pressure, listbuilding time, primitive retrieval counts for subsequentrendering, primitive data accesses (from external memory)during rendering, and possibly enhanced data locality/resourceutilization if layer-based rendering is exploited. Based on thisstructure, we propose a primitive-hierarchy fitting (hardware)algorithm which, for a given primitive of any size and shape,determines a best way of storing it in the structure. Experimentalresults on Doom3 and Quake4 show 10% to 33% storagereduction with our full capability, compared with using squarehierarchies only.

Unique-order interpolative coding for fast querying and space-efficient indexing in information retrieval systems

This paper presents a size reduction method for the inverted file, the most suitable indexing structure for an information retrieval system (IRS). We notice that in an inverted file the document identifiers for a given word are usually clustered. While this clustering property can be used in reducing the size of the inverted file, good compression as well as fast decompression must both be available. In this paper, we present a method that can facilitate coding and decoding processes for interpolative coding using recursion elimination and loop unwinding. We call this method the unique-order interpolative coding. It can calculate the lower and upper bounds of every document identifier for a binary code without using a recursive process, hence the decompression time can be greatly reduced. Moreover, it also can exploit document identifier clustering to compress the inverted file efficiently. Compared with the other well-known compression methods, our method provides fast decoding speed and excellent compression. This method can also be used to support a self-indexing strategy. Therefore our research work in this paper provides a feasible way to build a fast and space-economical IRS.

Posting file partitioning and parallel information retrieval

The rapid growth in Internet usages brings new challenges on designing a scalable information retrieval system. To reduce the response time of a query to a large database, we parallelize both CPU computation and disk access of Boolean query processing on a cluster of workstations. The key issue is to partition the inverted file such that, during parallel query processing, each workstation consults only its own locally resident data to complete its task. To achieve this goal, we treat the set of all postings referring to a document ID as an object to be allocated in the develop data placement problem. Following the partitioning by document ID principle, we develop posting file partitioning algorithms to transform a sequential information retrieval system to a parallel information retrieval system. The advantage is that a better speed-up can be achieved by deriving from the fast sequential approach – the compressed posting file. The partitioning schemes are designed to balance work-load of workstations in parallel query processing without increasing the average disk access time per posting. The experiment shows that almost linear speed-up can be achieved and the performance bottleneck in previous work, which parallelize only disk access, can be removed. This work shows that, by using parallel processing technique, it is feasible to build a scalable information retrieval system.

An analytical POC stack operations folding for continuous and discontinuous Java bytecodes

The execution performance of a stack-based Java virtual machine (JVM) is limited by the true data dependency. To enhance the performance of the JVM, a stack operations folding mechanism for the picoJava-I/II processor was proposed by Sun Microsystems to fold 42.3% stack operations. By comparing the continuous bytecodes with pre-defined folding patterns in instruction decoder, the number of push/pop operations in between the operand stack and the local variable could be reduced. In this study, an enhanced POC (EPOC) folding model is proposed to further fold the discontinuous bytecodes that cannot be folded in continuous bytecodes folding mechanisms. By proposing a stack re-order buffer (SROB) to help the folding check processes, the EPOC folding model can fold the stack operations perfectly with a small size of SROB implementation. Statistical data shows that the four-foldable strategy of the EPOC folding model can eliminate 98.8% of push/pop operations with an instruction buffer size of 7 bytes and the SROB size of eight entries.

Instruction set extension exploration in multiple-issue architecture

To satisfy high-performance computing demand in modern embedded devices, current embedded processor architectures provide designer with possibility either to define customized instruction set extension (ISE) or to increase instruction issue width. Previous studies have shown that deploying ISE in multiple-issue architecture can significantly improve performance. However, identifying ISE for multiple-issue architecture by using current ISE exploration algorithms will result in unnecessary waste of silicon area and limitation of performance improvement. This is because most algorithms overlook two important considerations: (1) only packing the operations lying on the critical path into ISE can improve performance; (2) the critical path usually changes after packing operations into an ISE. With these considerations, this paper presents an algorithm for ISE exploration based on list scheduling and Ant Colony Optimization (ACO), in which combines ISE exploration and the critical path identification (i.e. instruction scheduling). Results indicate that our approach outperforms the previous work in both performance improvement and area efficiency.

Design of an optimal folding mechanism for Java processors

Abstract Java has become the most important language in the Internet area, but its execution performance is severely limited by the true data dependency inherited from the stack architecture defined by the Suns Java Virtual Machine (JVM). To enhance the performance of the JVM, a stack operations folding mechanism for the picoJava-II processor was proposed by Sun Microsystems to fold 42.3% stack push/pop instructions. A systematic folding algorithm—Producer, Operator, and Consumer (POC) folding model was proposed in the earlier research to eliminate up to 82.9% of stack push/pop instructions. The remaining push and pop instructions cannot be folded due to the sequential checking characteristic of the POC folding model. A new folding algorithm—enhanced POC (EPOC) folding model is proposed in this paper to further fold the remaining push and pop instructions. In the EPOC folding model, stack push/pop instructions are folded with the proposed Stack Reorder Buffer (SROB) architecture. With a small SROB size of 584 bits, almost all of the stack push/pop instructions can be folded with the precise exception handling capability. Statistical data shows that 98.8% of the stack push/pop instructions can be folded, and the average execution performance speedup of a 4-foldable processor with a 7-byte instruction buffer is 1.74 as compared to a traditional single-pipelined stack machine without folding.

Load and storage balanced posting file partitioning for parallel information retrieval

Abstract: Many recent major search engines on Internet use a large-scale cluster to store a large database and cope with high query arrival rate. To design a large scale parallel information retrieval system, both performance and storage cost has to be taken into integrated consideration. Moreover, a quantitative method to design the cluster in systematical way is required. This paper proposes posting file partitioning algorithm for these requirements. The partitioning follows the partition-by-document-ID principle to eliminate communication overhead. The kernel of the partitioning is a data allocation algorithm to allocate variable-sized data items for both load and storage balancing. The data allocation algorithm is proven to satisfy a load balancing constraint with asymptotical 1-optimal storage cost. A probability model is established such that query processing throughput can be calculated from keyword popularities and data allocation result. With these results, we show a quantitative method to design a cluster systematically. This research provides a systematical approach to large-scale information retrieval system design. This approach has the following features: (1) the differences to ideal load balancing and storage balancing are negligible in real-world application. (2) Both load balancing and storage balancing can be taken into integrated consideration without conflicting. (3) The data allocation algorithm is capable to deal with data items of variable-sizes and variable loads. An algorithm having all these features together is never achieved before and is the key factor for achieving load and storage balanced workstation cluster in a real-world environment.