Oana G. Cula

Compact representation of bidirectional texture functions

A bidirectional texture function (BTF) describes image texture as it varies with viewing and illumination direction. Many real world surfaces such as skin, fur, gravel, etc. exhibit fine-scale geometric surface detail. Accordingly, variations in appearance with viewing and illumination direction may be quite complex due to local foreshortening, masking and shadowing. Representations of surface texture that support robust recognition must account for these effects. We construct a representation which captures the underlying statistical distribution of features in the image texture as well as the variations in this distribution with viewing and illumination direction. The representation combines clustering to learn characteristic image features and principle components analysis to reduce the space of feature histograms. This representation is based on a core image set as determined by a quantitative evaluation of importance of individual images in the overall representation. The result is a compact representation and a recognition method where a single novel image of unknown viewing and illumination direction can be classified efficiently. The CUReT (Columbia-Utrecht reflectance and texture) database is used as a test set for evaluation of these methods.

3D Texture Recognition Using Bidirectional Feature Histograms

Textured surfaces are an inherent constituent of the natural surroundings, therefore efficient real-world applications of computer vision algorithms require precise surface descriptors. Often textured surfaces present not only variations of color or reflectance, but also local height variations. This type of surface is referred to as a 3D texture. As the lighting and viewing conditions are varied, effects such as shadowing, foreshortening and occlusions, give rise to significant changes in texture appearance. Accounting for the variation of texture appearance due to changes in imaging parameters is a key issue in developing accurate 3D texture models. The bidirectional texture function (BTF) is observed image texture as a function of viewing and illumination directions. In this work, we construct a BTF-based surface model which captures the variation of the underlying statistical distribution of local structural image features, as the viewing and illumination conditions are changed. This 3D texture representation is called the bidirectional feature histogram (BFH). Based on the BFH, we design a 3D texture recognition method which employs the BFH as the surface model, and classifies surfaces based on a single novel texture image of unknown imaging parameters. Also, we develop a computational method for quantitatively evaluating the relative significance of texture images within the BTF. The performance of our methods is evaluated by employing over 6200 texture images corresponding to 40 real-world surface samples from the CUReT (Columbia-Utrecht reflectance and texture) database. Our experiments produce excellent classification results, which validate the strong descriptive properties of the BFH as a 3D texture representation.

Skin Texture Modeling

Quantitative characterization of skin appearance is an important but difficult task. The skin surface is a detailed landscape, with complex geometry and local optical properties. In addition, skin features depend on many variables such as body location (e.g. forehead, cheek), subject parameters (age, gender) and imaging parameters (lighting, camera). As with many real world surfaces, skin appearance is strongly affected by the direction from which it is viewed and illuminated. Computational modeling of skin texture has potential uses in many applications including realistic rendering for computer graphics, robust face models for computer vision, computer-assisted diagnosis for dermatology, topical drug efficacy testing for the pharmaceutical industry and quantitative comparison for consumer products. In this work we present models and measurements of skin texture with an emphasis on faces. We develop two models for use in skin texture recognition. Both models are image-based representations of skin appearance that are suitably descriptive without the need for prohibitively complex physics-based skin models. Our models take into account the varied appearance of the skin with changes in illumination and viewing direction. We also present a new face texture database comprised of more than 2400 images corresponding to 20 human faces, 4 locations on each face (forehead, cheek, chin and nose) and 32 combinations of imaging angles. The complete database is made publicly available for further research.

Bidirectional imaging and modeling of skin texture

In this paper, we present a method of skin imaging called bidirectional imaging that captures significantly more properties of appearance than standard imaging. The observed structure of the skins surface is greatly dependent on the angle of incident illumination and the angle of observation. Specific protocols to achieve bidirectional imaging are presented and used to create the Rutgers Skin Texture Database (clinical component). This image database is the first of its kind in the dermatology community. Skin images of several disorders under multiple controlled illumination and viewing directions are provided publicly for research and educational use. Using this skin texture database, we employ computational surface modeling to perform automated skin texture classification. The classification experiments demonstrate the usefulness of the modeling and measurement methods.

Recognition methods for 3D textured surfaces

Texture as a surface representation is the subject of a wide body of computer vision and computer graphics literature. While texture is always associated with a form of repetition in the image, the repeating quantity may vary. The texture may be a color or albedo variation as in a checkerboard, a paisley print or zebra stripes. Very often in real-world scenes, texture is instead due to a surface height variation, e.g. pebbles, gravel, foliage and any rough surface. Such surfaces are referred to here as 3D textured surfaces. Standard texture recognition algorithms are not appropriate for 3D textured surfaces because the appearance of these surfaces changes in a complex manner with viewing direction and illumination direction. Recent methods have been developed for recognition of 3D textured surfaces using a database of surfaces observed under varied imaging parameters. One of these methods is based on 3D textons obtained using K-means clustering of multiscale feature vectors. Another method uses eigen-analysis originally developed for appearance-based object recognition. In this work we develop a hybrid approach that employs both feature grouping and dimensionality reduction. The method is tested using the Columbia-Utrecht texture database and provides excellent recognition rates. The method is compared with existing recognition methods for 3D textured surfaces. A direct comparison is facilitated by empirical recognition rates from the same texture data set. The current method has key advantages over existing methods including requiring less prior information on both the training and novel images.

Statistical and Hybrid Methods for Speech Recognition in Romanian

The present paper describes the evolution of our work concerning the problem of speech recognition. Beginning with a classical hidden Markov model (HMM), we have investigated two ways to improve the performance of this basic structure. The first way was to realize a neuro-statistical hybrid by integrating a multilayer perceptron (MLP) as a posteriori probability estimator. The system was further refined by adding supplementary discriminative training (DT) based on the minimum classification error (MCE). Tests performed on a 15,000 isolated spoken-word database, showed an increase in the recognition rate from 92.2% for the HMM-based recognition system, to 94.7% for the HMM-MLP system, and then to 98.1% for the refined HMM-MLP-DT system. The second way to improve the classical HMM was to build a fuzzy-statistical hybrid, FHMM, based on a fuzzy similarity measure instead of the probabilistic measure specific to the usual statistical model. The benefits of the fuzzy measure introduction were evaluated on a vowel recognition task, and a decrease of approximately 3% in the error rate is reported.

Polarization multiplexing for bidirectional imaging

Our goal is to incorporate polarization in appearance-based modeling in an efficient and meaningful way. Polarization has been used in numerous prior studies for separating diffuse and specular reflectance components, but in this work we show that it also can be used to separate surface reflectance contributions from individual light sources. Our approach is called polarization multiplexing and it has significant impact in appearance modeling and bidirectional imaging where the image as a function of illumination direction is needed. Multiple unknown light sources can illuminate the scene simultaneously, and the individual contributions to the overall surface reflectance can be estimated. To develop the method of polarization multiplexing, we use a relationship between light source direction and intensity modulation. Inverting this transformation enables the individual intensity contributions to be estimated. In addition to polarization multiplexing, we show that phase histograms from the intensity modulations can be used to estimate scene properties including the number of light sources.

Polarization Multiplexing and Demultiplexing for Appearance-Based Modeling

Polarization has been used in numerous prior studies for separating diffuse and specular reflectance components, but in this work we show that it also can be used to separate surface reflectance contributions from individual light sources. Our approach is called polarization multiplexing and it has a significant impact in appearance modeling where the image as a function of illumination direction is needed. Multiple unknown light sources can illuminate the scene simultaneously, and the individual contributions to the overall surface reflectance are estimated. Polarization multiplexing relies on the relationship between the light source direction and the intensity modulation. Inverting this transformation enables the individual intensity contributions to be estimated. In addition to polarization multiplexing, we show that phase histograms from the intensity modulations can be used to estimate scene properties including the number of light sources

Learning-based detection of acne-like regions using time-lapse features

Objective evaluation of acne treatment requires observing test subjects for multiple months. To capture the appearance of acne lesions during the treatment period, a subject is photographed at imaging sessions separated by time intervals of days or weeks. The efficacy of the treatment method is evaluated by counting the number of acne lesions in the acquired skin images. Traditionally, the counting of acne lesions has been done manually. However, manual counting is unreliable and time consuming; therefore in recent years there has been an increasing interest in automatically detecting and counting acne lesions using computer-based methods. In this paper we model acne-like and non-acne regions using spatio-temporal features, and use a supervised learning approach to find the separating hyperplane between the regions in the feature space. The temporal component is an important feature because acne lesions change over time, while scars and other marks remain constant. Precise alignment is a challenge in computing meaningful temporal features. The images must be aligned to a subpixel level, exceeding the requirements of typical face alignment algorithms. We have acquired and aligned a time series acne dataset by imaging a human subject with facial acne under the same illumination and pose on 39 different days over a period of three months. The resulting time-lapse video of skin with precision alignment is the first of its kind and impressively demonstrates the temporal evolution of acne lesions. We use this registered time-lapse set to train and test an acne lesion classifier.

Surface detail in computer models

Quantitative characterization of surface appearance is an important but difficult task. Surfaces of real world objects are detailed landscapes, with complex geometry and local optical properties. Surface appearance is strongly affected by the direction from which it is viewed and illuminated. Computational modeling of surface texture has potential uses in many applications including realistic rendering for computer graphics and robust recognition for computer vision. For recognition, the overall structure of the object is important, but fine-scale details can assist the recognition problem greatly. We develop models of surface texture and demonstrate their use in recognition tasks. We also describe a texture camera for capturing fine-scale surface details. Specifically, the texture camera measures reflectance and surface height variation using curved mirrors. We discuss why measurements and models of fine scale detail are important in modern industrial applications.