Cong Phuoc Huynh

A Solution of the Dichromatic Model for Multispectral Photometric Invariance

In this paper, we address the problem of photometric invariance in multispectral imaging making use of an optimisation approach based upon the dichromatic model. In this manner, we cast the problem of recovering the spectra of the illuminant, the surface reflectance and the shading and specular factors in a structural optimisation setting. Making use of the additional information provided by multispectral imaging and the structure of image patches, we recover the dichromatic parameters of the scene. To do this, we formulate a target cost function combining the dichromatic error and the smoothness priors for the surfaces under study. The dichromatic parameters are recovered through minimising this cost function in a coordinate descent manner. The algorithm is quite general in nature, admitting the enforcement of smoothness constraints and extending in a straightforward manner to trichromatic settings. Moreover, the objective function is convex with respect to the subset of variables to be optimised in each alternating step of the minimisation strategy. This gives rise to an optimal closed-form solution for each of the iterations in our algorithm. We illustrate the effectiveness of our method for purposes of illuminant spectrum recovery, skin recognition, material clustering and specularity removal. We also compare our results to a number of alternatives.

Shape and refractive index recovery from single-view polarisation images

In this paper, we propose an approach to the problem of simultaneous shape and refractive index recovery from multispectral polarisation imagery captured from a single viewpoint. The focus of this paper is on dielectric surfaces which diffusely polarise light transmitted from the dielectric body into the air. The diffuse polarisation of the reflection process is modelled using a Transmitted Radiance Sinusoid curve and the Fresnel transmission theory. We provide a method of estimating the azimuth angle of surface normals from the spectral variation of the phase of polarisation. Moreover, to render the problem of simultaneous estimation of surface orientation and index of refraction well-posed, we enforce a generative model on the material dispersion equations for the index of refraction. This generative model, together with the Fresnel transmission ratio, permit the recovery of the index of refraction and the zenith angle simultaneously. We show results on shape recovery and rendering for real world and synthetic imagery.

Imaging Spectroscopy for Scene Analysis

This book presents a detailed analysis of spectral imaging, describing how it can be used for the purposes of material identification, object recognition and scene understanding. The opportunities and challenges of combining spatial and spectral information are explored in depth, as are a wide range of applications. Features: discusses spectral image acquisition by hyperspectral cameras, and the process of spectral image formation; examines models of surface reflectance, the recovery of photometric invariants, and the estimation of the illuminant power spectrum from spectral imagery; describes spectrum representations for the interpolation of reflectance and radiance values, and the classification of spectra; reviews the use of imaging spectroscopy for material identification; explores the recovery of reflection geometry from image reflectance; investigates spectro-polarimetric imagery, and the recovery of object shape and material properties using polarimetric images captured from a single view.

Shape and Refractive Index from Single-View Spectro-Polarimetric Images

In this paper, we address the problem of the simultaneous recovery of the shape and refractive index of an object from a spectro-polarimetric image captured from a single view. Here, we focus on the diffuse polarisation process occuring at dielectric surfaces due to subsurface scattering and transmission from the object surface into the air. The diffuse polarisation of the reflection process is modelled by the Fresnel transmission theory. We present a method for estimating the azimuth angle of surface normals from the spectral variation of the phase of polarisation. Moreover, we estimate the zenith angle of surface normals and index of refraction simultaneously in a well-posed optimisation framework. We achieve well-posedness by introducing two additional constraints to the problem, including the surface integrability and the material dispersion equation. This yields an iterative solution which is computationally efficient due to the use of closed-form solutions for both the zenith angle and the refractive index in each iteration. To demonstrate the effectiveness of our approach, we show results of shape recovery and surface rendering for both real-world and synthetic imagery.

Hyperspectral imaging for skin recognition and biometrics

In this paper, we present a system for automatic spectral signature acquisition and recognition of skin from hyperspectral face imagery. In the acquisition step, hyperspectral cameras are used to capture multispectral or hyperspectral images of faces for skin recognition. The acquired signature may either be stored in a database for future testing or be used for purposes of identification. In the recognition step, the system accounts for variations in the illumination by recovering the light power spectrum in the scene and obtains the scene reflectance by normalising the input image radiance accordingly. Furthermore, incorporated into this system is a Non-Uniform Rational B-Spline (NURBS) compact descriptor of spectral reflectance for recognition purposes. We have employed this system as a profiling tool to classify a real-world multispectral face image database into separate ethnic groups.

Simultaneous photometric invariance and shape recovery

In this paper, we identify the constraints under which the generally ill-posed problem of simultaneous recovery of surface shape and its photometric invariants can be rendered tractable. We examine the cases where a single or more images are acquired using different lighting directions with known illuminant power. Given these conditions, we state the constraints upon which the recovery of the surface geometry and its photometric parameters can be estimated. With these constraints, we then show how the recovery process may be formulated as an optimisation algorithm which aims to fit the reflectance models under study to the image reflectance. The approach presented here is general and can be applied to a family of reflectance models that are based on the Fresnel reflection theory. Thus, we provide a theoretical and computational background for recovering shape, material index of refraction and microscopic roughness from multi-spectral images.

Supporting OO Design Heuristics

Heuristics have long been recognised as a way to tackle problems which are intractable because of their size or complexity. They have been used in software engineering for purposes such as identification of favourable regions of design space. Some heuristics in software engineering can be expressed in high-level abstract terms while others are more specific. Heuristics tend to be couched in terms which make them hard to automate. In our previous work we have developed robust semantic models of software in order to support the computation of metrics and the construction of visualisations which allow their interpretation by developers. In this paper, we show how software engineering heuristics can be supported by a semantic model infrastructure. Examples from our current work illustrate the value of combining the rigour of a semantic model with the human mental models associated with heuristics.

Class-Specific Image Deblurring

In image deblurring, a fundamental problem is that the blur kernel suppresses a number of spatial frequencies that are difficult to recover reliably. In this paper, we explore the potential of a class-specific image prior for recovering spatial frequencies attenuated by the blurring process. Specifically, we devise a prior based on the class-specific subspace of image intensity responses to band-pass filters. We learn that the aggregation of these subspaces across all frequency bands serves as a good class-specific prior for the restoration of frequencies that cannot be recovered with generic image priors. In an extensive validation, our method, equipped with the above prior, yields greater image quality than many state-of-the-art methods by up to 5 dB in terms of image PSNR, across various image categories including portraits, cars, cats, pedestrians and household objects.

Material-specific user colour profiles from imaging spectroscopy data

In this paper, we present a method which permits the creation of user colour preferences for object materials and lights in the scene making use of imaging spectroscopy data. To do this, we build upon the heterogeneous nature of the scene by imposing consistency over object materials so as to allow for small compositional variations across objects in the image. Once the consistency has been imposed, we aim at maximising the quality of the images under consideration based upon user input. This provides the flexibility necessary to utilise user profiles for the automatic processing of real world imagery while avoiding undesirable effects encountered when colour images are produced. We provide results on real-world imagery and illustrate how the method can be used to produce material-specific colours based upon user input.

A NURBS-based spectral reflectance descriptor with applications in computer vision and pattern recognition

In this paper, we present a surface reflectance descriptor based on the control points resulting from the interpolation of non-uniform rational B-spline (NURBS) curves to multispectral reflectance data. The interpolation is based upon a knot removal scheme in the parameter domain. Thus, we exploit the local support of NURBS so as to recover a compact descriptor robust to noise and local perturbation of the spectra. We demonstrate the utility of our NURBS-based descriptor for material identification. To this end, we perform skin spectra recognition making use of a support vector machine classifier. We also provide results on hyperspectral imagery and elaborate on the preprocessing step for skin segmentation. We compare our results with those obtained using an alternative descriptor.