Terrance E. Boult

Constraining object features using a polarization reflectance model

The authors present a polarization reflectance model that uses the Fresnel reflection coefficients. This reflectance model accurately predicts the magnitudes of polarization components of reflected light, and all the polarization-based methods presented follow from this model. The authors demonstrate the capability of polarization-based methods to segment material surfaces according to varying levels of relative electrical conductivity, in particular distinguishing dielectrics, which are nonconducting, and metals, which are highly conductive. Polarization-based methods can provide cues for distinguishing different intensity-edge types arising from intrinsic light-dark or color variations, intensity edges caused by specularities, and intensity edges caused by occluding contours where the viewing direction becomes nearly orthogonal to surface normals. Analysis of reflected polarization components is also shown to enable the separation of diffuse and specular components of reflection, unobscuring intrinsic surface detail saturated by specular glare. Polarization-based methods used for constraining surface normals are discussed. >

Separation of Reflection Components Using Color and Polarization

Specular reflections and interreflections produce strong highlights in brightness images. These highlights can cause vision algorithms for segmentation, shape from shading, binocular stereo, and motion estimation to produce erroneous results. A technique is developed for separating the specular and diffuse components of reflection from images. The approach is to use color and polarization information, simultaneously, to obtain constraints on the reflection components at each image point. Polarization yields local and independent estimates of the color of specular reflection. The result is a linear subspace in color space in which the local diffuse component must lie. This subspace constraint is applied to neighboring image points to determine the diffuse component. In contrast to previous separation algorithms, the proposed method can handle highlights on surfaces with substantial texture, smoothly varying diffuse reflectance, and varying material properties. The separation algorithm is applied to several complex scenes with textured objects and strong interreflections. The separation results are then used to solve three problems pertinent to visual perception; determining illumination color, estimating illumination direction, and shape recovery.

The eyes have it

This paper evaluates the impact of eye localization on face recognition accuracy. To investigate its importance, we present an eye perturbation sensitivity analysis, as well as empirical evidence that reinforces the notion that eye localization plays a key role in the accuracy of face recognition systems. In particular, correct measurement of eye separation is shown to be more important than correct eye location, highlighting the critical role of eye separation in the scaling and normalization of face images. Results suggest that significant gains in recognition accuracy may be achieved by focussing more effort on the eye localization stage of the face recognition process.

Vision of the unseen: Current trends and challenges in digital image and video forensics

Digital images are everywhere—from our cell phones to the pages of our online news sites. How we choose to use digital image processing raises a surprising host of legal and ethical questions that we must address. What are the ramifications of hiding data within an innocent image? Is this an intentional security practice when used legitimately, or intentional deception? Is tampering with an image appropriate in cases where the image might affect public behavior? Does an image represent a crime, or is it simply a representation of a scene that has never existed? Before action can even be taken on the basis of a questionable image, we must detect something about the image itself. Investigators from a diverse set of fields require the best possible tools to tackle the challenges presented by the malicious use of todays digital image processing techniques. In this survey, we introduce the emerging field of digital image forensics, including the main topic areas of source camera identification, forgery detection, and steganalysis. In source camera identification, we seek to identify the particular model of a camera, or the exact camera, that produced an image. Forgery detections goal is to establish the authenticity of an image, or to expose any potential tampering the image might have undergone. With steganalysis, the detection of hidden data within an image is performed, with a possible attempt to recover any detected data. Each of these components of digital image forensics is described in detail, along with a critical analysis of the state of the art, and recommendations for the direction of future research.

Multi-attribute spaces: Calibration for attribute fusion and similarity search

Recent work has shown that visual attributes are a powerful approach for applications such as recognition, image description and retrieval. However, fusing multiple attribute scores - as required during multi-attribute queries or similarity searches - presents a significant challenge. Scores from different attribute classifiers cannot be combined in a simple way; the same score for different attributes can mean different things. In this work, we show how to construct normalized “multi-attribute spaces” from raw classifier outputs, using techniques based on the statistical Extreme Value Theory. Our method calibrates each raw score to a probability that the given attribute is present in the image. We describe how these probabilities can be fused in a simple way to perform more accurate multiattribute searches, as well as enable attribute-based similarity searches. A significant advantage of our approach is that the normalization is done after-the-fact, requiring neither modification to the attribute classification system nor ground truth attribute annotations. We demonstrate results on a large data set of nearly 2 million face images and show significant improvements over prior work. We also show that perceptual similarity of search results increases by using contextual attributes.

Into the woods: visual surveillance of noncooperative and camouflaged targets in complex outdoor settings

Autonomous video surveillance and monitoring of human subjects in video has a rich history. Many deployed systems are able to reliably track human motion in indoor and controlled outdoor environments, e.g., parking lots and university campuses. A challenging domain of vital military importance is the surveillance of noncooperative and camouflaged targets within cluttered outdoor settings. These situations require both sensitivity and a very wide field of view and, therefore, are a natural application of omnidirectional video. Fundamentally, target finding is a change detection problem. Detection of camouflaged and adversarial targets implies the need for extreme sensitivity. Unfortunately, blind change detection in woods and fields may lead to a high fraction of false alarms, since natural scene motion and lighting changes produce highly dynamic scenes. Naturally, this desire for high sensitivity leads to a direct tradeoff between miss detections and false alarms. This paper discusses the current state of the art in video-based target detection, including an analysis of background adaptation techniques. The primary focus of the paper is the Lehigh Omnidirectional Tracking System (LOTS) and its components. This includes adaptive multibackground modeling, quasi-connected components (a novel approach to spatio-temporal grouping), background subtraction analyses, and an overall system evaluation.

Factorization-based segmentation of motions

The authors address the problem of motion segmentation using the singular value decomposition of a feature track matrix. It is shown that, under general assumptions, the number of numerically nonzero singular values can be used to determine the number of motions. Furthermore, motions can be separated using the right singular vectors associated with the nonzero singular values. A relationship is derived between a good segmentation, the number of nonzero singular values in the input and the sum of the number of nonzero singular values in the segments. The approach is demonstrated on real and synthetic examples. The paper ends with a critical analysis of the approach.<<ETX>>

Revocable fingerprint biotokens: accuracy and security analysis

This paper reviews the biometric dilemma, the pending threat that may limit the long-term value of biometrics in security applications. Unlike passwords, if a biometric database is ever compromised or improperly shared, the underlying biometric data cannot be changed. The concept of revocable or cancelable biometric-based identity tokens (biotokens), if properly implemented, can provide significant enhancements in both privacy and security and address the biometric dilemma. The key to effective revocable biotokens is the need to support the highly accurate approximate matching needed in any biometric system as well as protecting privacy/security of the underlying data. We briefly review prior work and show why it is insufficient in both accuracy and security. This paper adapts a recently introduced approach that separates each datum into two fields, one of which is encoded and one which is left to support the approximate matching. Previously applied to faces, this paper uses this approach to enhance an existing fingerprint system. Unlike previous work in privacy-enhanced biometrics, our approach improves the accuracy of the underlying svstem! The security analysis of these biotokens includes addressing the critical issue of protection of small fields. The resulting algorithm is tested on three different fingerprint verification challenge datasets and shows an average decrease in the Equal Error Rate of over 30% - providing improved security and improved privacy.

Toward Open Set Recognition

To date, almost all experimental evaluations of machine learning-based recognition algorithms in computer vision have taken the form of “closed set” recognition, whereby all testing classes are known at training time. A more realistic scenario for vision applications is “open set” recognition, where incomplete knowledge of the world is present at training time, and unknown classes can be submitted to an algorithm during testing. This paper explores the nature of open set recognition and formalizes its definition as a constrained minimization problem. The open set recognition problem is not well addressed by existing algorithms because it requires strong generalization. As a step toward a solution, we introduce a novel “1-vs-set machine,” which sculpts a decision space from the marginal distances of a 1-class or binary SVM with a linear kernel. This methodology applies to several different applications in computer vision where open set recognition is a challenging problem, including object recognition and face verification. We consider both in this work, with large scale cross-dataset experiments performed over the Caltech 256 and ImageNet sets, as well as face matching experiments performed over the Labeled Faces in the Wild set. The experiments highlight the effectiveness of machines adapted for open set evaluation compared to existing 1-class and binary SVMs for the same tasks.

Removal of specularities using color and polarization

An algorithm for separating the specular and diffuse components of reflection from images is presented. The method uses color and polarization simultaneously to obtain strong constraints on the reflection components at each image point. Polarization is used to locally determine the color of the specular component, constraining the diffuse color at a pixel to a one-dimensional linear subspace. This subspace is used to find neighboring pixels whose color is consistent with the pixel. Diffuse color information from consistent neighbors is used to determine the diffuse color of the pixel. In contrast to previous separation algorithms, the proposed method can handle highlights that have a varying diffuse component, as well as highlights that include regions with different reflectance and material properties. Experimental results obtained by applying the algorithm to complex scenes with textured objects and strong interreflections are presented.<<ETX>>