Soo Hyun Bae

Subjective Evaluation of Spatial Resolution and Quantization Noise Tradeoffs

Most full-reference fidelity/quality metrics compare the original image to a distorted image at the same resolution assuming a fixed viewing condition. However, in many applications, such as video streaming, due to the diversity of channel capacities and display devices, the viewing distance and the spatiotemporal resolution of the displayed signal may be adapted in order to optimize the perceived signal quality. For example, at low bitrate coding applications an observer may prefer to reduce the resolution or increase the viewing distance to reduce the visibility of the compression artifacts. The tradeoff between resolution/viewing conditions and visibility of compression artifacts requires new approaches for the evaluation of image quality that account for both image distortions and image size. In order to better understand such tradeoffs, we conducted subjective tests using two representative still image coders, JPEG and JPEG 2000. Our results indicate that an observer would indeed prefer a lower spatial resolution (at a fixed viewing distance) in order to reduce the visibility of the compression artifacts, but not all the way to the point where the artifacts are completely invisible. Moreover, the observer is willing to accept more artifacts as the image size decreases. The subjective test results we report can be used to select viewing conditions for coding applications. They also set the stage for the development of novel fidelity metrics. The focus of this paper is on still images, but it is expected that similar tradeoffs apply to video.

Multidimensional Incremental Parsing for Universal Source Coding

A multidimensional incremental parsing algorithm (MDIP) for multidimensional discrete sources, as a generalization of the Lempel-Ziv coding algorithm, is investigated. It consists of three essential component schemes, maximum decimation matching, hierarchical structure of multidimensional source coding, and dictionary augmentation. As a counterpart of the longest match search in the Lempel-Ziv algorithm, two classes of maximum decimation matching are studied. Also, an underlying behavior of the dictionary augmentation scheme for estimating the source statistics is examined. For an m-dimensional source, m augmentative patches are appended into the dictionary at each coding epoch, thus requiring the transmission of a substantial amount of information to the decoder. The property of the hierarchical structure of the source coding algorithm resolves this issue by successively incorporating lower dimensional coding procedures in the scheme. In regard to universal lossy source coders, we propose two distortion functions, the local average distortion and the local minimax distortion with a set of threshold levels for each source symbol. For performance evaluation, we implemented three image compression algorithms based upon the MDIP; one is lossless and the others are lossy. The lossless image compression algorithm does not perform better than the Lempel-Ziv-Welch coding, but experimentally shows efficiency in capturing the source structure. The two lossy image compression algorithms are implemented using the two distortion functions, respectively. The algorithm based on the local average distortion is efficient at minimizing the signal distortion, but the images by the one with the local minimax distortion have a good perceptual fidelity among other compression algorithms. Our insights inspire future research on feature extraction of multidimensional discrete sources.

Spatial Resolution and Quantization Noise Tradeoffs for Scalable Image Compression

Most full-reference quality metrics compare the original image to a distorted image at the same level of resolution assuming a fixed viewing distance. In video streaming applications, however, the transmitted or received signal may differ from the original in compression as well as spatiotemporal resolution. For example, at low bitrate coding applications the compressed image may be too distorted, and hence the observer may prefer to reduce the resolution or increase the viewing distance in order to reduce the visibility of the compression artifacts. The selection of the best tradeoff between resolution/viewing distance and visibility of compression artifacts requires a quality metric that accounts for both image distortions and image size. Such tradeoffs are not reflected in existing quality metrics, which ignore the signal visibility and only measure the visibility of compression distortions, which decrease with image size. In order to better understand such tradeoffs, with the goal of developing better quality metrics, we conducted subjective tests using a number of existing still image coders (JPEG2000 SPHIT, and JPEG). Our results indicate that the objective quality (perceptually weighted PSNR) of the images that the viewers select decreases with resolution, that is, the viewers are willing to accept more artifacts as image size decreases

IPSILON: Incremental Parsing for Semantic Indexing of Latent Concepts

A new framework for content-based image retrieval, which takes advantage of the source characterization property of a universal source coding scheme, is investigated. Based upon a new class of multidimensional incremental parsing algorithm, extended from the Lempel-Ziv incremental parsing code, the proposed method captures the occurrence pattern of visual elements from a given image. A linguistic processing technique, namely the latent semantic analysis (LSA) method, is then employed to identify associative ensembles of visual elements, which lay the foundation for intelligent visual information analysis. In 2-D applications, incremental parsing decomposes an image into elementary patches that are different from the conventional fixed square-block type patches. When used in compressive representations, it is amenable in schemes that do not rely on average distortion criteria, a methodology that is a departure from the conventional vector quantization. We call this methodology a parsed representation. In this article, we present our implementations of an image retrieval system, called IPSILON, with parsed representations induced by different perceptual distortion thresholds. We evaluate the effectiveness of the use of the parsed representations by comparing their performance with that of four image retrieval systems, one using the conventional vector quantization for visual information analysis under the same LSA paradigm, another using a method called SIMPLIcity which is based upon an image segmentation and integrated region matching, and the other two based upon query-by-semantic-example and query-by-visual-example. The first two of them were tested with 20 000 images of natural scenes, and the others were tested with a portion of the images. The experimental results show that the proposed parsed representation efficiently captures the salient features in visual images and the IPSILON systems outperform other systems in terms of retrieval precision and distortion robustness.

Incremental parsing for latent semantic indexing of images

A generalized latent semantic analysis framework using a universal source coding algorithm for content-based image retrieval is proposed. By the multidimensional incremental parsing algorithm which is considered as a multidimensional extension of the Lempel-Ziv data compression method, a given image is compressed at a moderate bitrate while constructing the dictionary which implicitly embeds source statistics. Instead of concatenating all the corresponding dictionaries of an image corpus, we sequentially compress images using a previously constructed dictionary and end up with a visual lexicon which contains the least number of visual words covering all the images in the corpus. From the latent semantic analysis of the co-occurrence pattern of visual words over the images, a similarity between a given query and an image from the corpus is measured. An application of the proposed technique on a database of 20,000 natural scene images has demonstrated that the performance of the proposed system is favorable to that of existing approaches.

Toward robust moment invariants for image registration

We apply pattern recognition techniques to enhance the robustness of moment-invariants-based image classifiers. Moment invariants exhibit variations under transformations that do not preserve the original image function, such as geometrical transformations involving interpolation. Such variations degrade the performance of classifiers due to the errors in the nearest neighbor search stage. We propose the use of linear discriminant analysis (LDA) and principal component analysis (PCA) to alleviate the variations and enhance the robustness of classification. We demonstrate the improved performance in image registration applications under spatial scaling and rotation transformations.

3CCD interpolation using selective projection

The emergence of HDTV accelerates the evolution of high-resolution imaging systems. A 3CCD digital camera system has been developed for higher resolution than one CCD imaging has. From the pixel correlation caused by a half-pixel shift of the green channel, we can interpolate pixels and get four times higher resolution of the color image. The proposed method involves three projection operators. The first is to reduce aliasing of image regions by selective projection in subband channels. The second projection makes an inverse of the MTF which generates blurring over the entire image. The last operator works for fast convergence. From experimental results, the proposed algorithm shows suppression of jagging effects and restoration of aliased image regions. It is experimentally shown that the projection process converges and is almost finished at the first iteration.

Feature extraction by incremental parsing for music indexing

In this paper, we employ a linguistic-processing approach to the content-based retrieval of music information. Central to the approach is the use of a lossy version of the Lempel-Ziv incremental parsing (LZIP) algorithm, which constructs a dictionary by incrementally parsing music feature vectors. LZIP is adopted as a source characterization technique owing to its universal-coding nature, and asymptotic convergence to the entropy of the source. The dictionary is composed of variable-length parsed representations, which are used to construct a highly sparse co-occurrence matrix, which counts the occurrence of the parsed representations in each music. As a feature analysis framework, Latent Semantic Analysis (LSA) is then applied to the co-occurrence matrix to generate a lower-dimensional approximation that exposes the most salient features of the represented audio documents. The aforementioned approach, in addition to adopting reduced sampling rates and quantized feature vectors, yields a system with reduced requirements in terms of processing and storage, and increases the tolerance to noisy queries. We demonstrate the performance of the system in the music genre classification problem, and analyze its robustness to perturbed queries. Moreover, we demonstrate that using the incremental parsing algorithm in forming the audio dictionary has superior retrieval performance compared to techniques yielding a dictionary with fixed-length entries such as vector quantization.

Parsed and fixed block representations of visual information for image retrieval

The theory of linguistics teaches us the existence of a hierarchical structure in linguistic expressions, from letter to word root, and on to word and sentences. By applying syntax and semantics beyond words, one can further recognize the grammatical relationship between among words and the meaning of a sequence of words. This layered view of a spoken language is useful for effective analysis and automated processing. Thus, it is interesting to ask if a similar hierarchy of representation of visual information does exist. A class of techniques that have a similar nature to the linguistic parsing is found in the Lempel-Ziv incremental parsing scheme. Based on a new class of multidimensional incremental parsing algorithms extended from the Lempel-Ziv incremental parsing, a new framework for image retrieval, which takes advantage of the source characterization property of the incremental parsing algorithm, was proposed recently. With the incremental parsing technique, a given image is decomposed into a number of patches, called a parsed representation. This representation can be thought of as a morphological interface between elementary pixel and a higher level representation. In this work, we examine the properties of two-dimensional parsed representation in the context of imagery information retrieval and in contrast to vector quantization; i.e. fixed square-block representations and minimum average distortion criteria. We implemented four image retrieval systems for the comparative study; three, called IPSILON image retrieval systems, use parsed representation with different perceptual distortion thresholds and one uses the convectional vector quantization for visual pattern analysis. We observe that different perceptual distortion in visual pattern matching does not have serious effects on the retrieval precision although allowing looser perceptual thresholds in image compression result poor reconstruction fidelity. We compare the effectiveness of the use of the parsed representations, as constructed under the latent semantic analysis (LSA) paradigm so as to investigate their varying capabilities in capturing semantic concepts. The result clearly demonstrates the superiority of the parsed representation.

Aspect modeling of parsed representation for image retrieval

A probabilistic framework based on a universal source coding for content-based image retrieval is proposed. By a multidimensional incremental parsing technique, which is an extension of the Lempel-Ziv incremental parsing algorithm, a given image is parsed into a number of variable-size rectangular blocks, called parsed representations. To achieve a semantically relevant pattern matching, we introduce a new similarity measure from the first- and second-order statistics of given image patches. Once the occurrence patterns of images in the corpus are analyzed, the term-document joint distribution is estimated by an aspect modeling technique under the assumption of latent aspects. To compare the performance of the proposed image retrieval framework based on the parsed representations, we implement a benchmark system based on the fixed-shape block representations trained by vector quantization. In addition to these two systems, we bring two content-based image retrieval systems into the performance evaluation. The experimental results on a database of 20,000 natural scene images demonstrate that the proposed image retrieval system significantly outperforms other existing and the benchmark systems.