Stephen P. Etz

A computational approach to determination of main subject regions in photographic images

Abstract We present a computational approach to main subject detection, which provides a measure of saliency or importance for different regions that are associated with different subjects in an image with unconstrained scene content. It is built primarily upon selected image semantics, with low-level vision features also contributing to the decision. The algorithm consists of region segmentation, perceptual grouping, feature extraction, and probabilistic reasoning. To accommodate the inherent ambiguity in the problem as reflected by the ground truth (probabilistic in nature), we have developed a novel training mechanism for Bayes nets based on fractional frequency counting. Using a set of images spanning the ‘photo space,’ experimental results have shown the promise of our approach in that most of the regions that independent observers ranked as the main subject are also labeled as such by our system. In addition, without reorganization and retraining, the Bayes net-based framework lends itself to performance scalable configurations to suit different applications that have different requirements of accuracy and speed. This paper focuses on a high level description of the complete system used to solve the overall problem, while providing necessary descriptions of the component algorithms.

Normalized Kemeny and Snell distance: a novel metric for quantitative evaluation of rank-order similarity of images

There are needs for evaluating rank order-based similarity between images. Region importance maps from image understanding algorithms or human observer studies are ordered rankings of the pixel locations. We address three problems with Kemeny and Snells distance (d/sub KS/), an existing measure from ordinal ranking theory, when applied to images: its high-computational cost, its bias in favor of images with sparse histograms, and its image-size dependent range of values. We present a novel computationally efficient algorithm for computing d/sub KS/ between two images and we derive a normalized form d/sub KS/ with no bias whose range is independent of image size. For evaluating similarity between images that can be considered as ordered rankings of pixels, d/sub KS/ is subjectively superior to cross correlation.

Quantitative evaluation of rank-order similarity of images

Region importance maps from image understanding algorithms and human observer studies are ordered rankings of the pixel locations. Kemeny and Snells distance (d/sub KS/), an existing measure from ordinal ranking theory, can thus be used as a similarity measure between images. We address three problems with d/sub KS/: its high computational cost, its bias in favor of images with sparse histograms, and its image-size dependent range of values. We present a novel computationally efficient algorithm for computing d/sub KS/ between two images, and we derive a normalized form d/sub KS/ with no bias whose range is independent of image size. For evaluating an algorithm where the reference data and algorithm output are ordered rankings of pixels, d/sub KS/ is subjectively superior to the correlation coefficient as a figure of merit.