Abdul R. Farooq

Dynamic photometric stereo for on line quality control of ceramic tiles

The rapid and automated detection of manufacturing flaws is becoming increasingly important in order to maintain competitive advantage in many production environments. In the case of natural and ornamental materials, the presence of both surface colouration and surface topography is often such that manual inspection, along with many conventional imaging techniques, fails to isolate physical or structural defects in the presence of complex and random patterns. In this paper the concepts of photometric stereo are adapted and extended for application in manufacturing environments. A case study on the high speed inspection of ceramic tiles is presented for the analysis of surfaces at production line rates of up to 30 m/min. This new technique, for the first time, demonstrates a genuine and commercially attractive potential for the practical automated quality control of complex surfaces. A commercial system, based on this research, is currently being developed.

Examining the uncertainty of the recovered surface normal in three light photometric stereo

Although the three light photometric stereo technique has been used in many applications, there is little published work concerned with characterizing the uncertainty of these systems due to the involvement of a number of complicating factors. This paper presents a methodology used to analyze the uncertainty of the recovered unit surface normal with respect to irradiance variance. Illumination configurations and the values of the composite albedo are found to directly affect the stability of the photometric stereo technique. An orthogonally distributed illumination arrangement is proven to be the theoretically optimal configuration. Further practical considerations are also identified. The derived general uncertainty expression can be easily employed to optimize the location of the light sources. Hence, the work is of significance for the development of practical industrial applications of photometric stereo, including metrology, reverse engineering and various surface inspection tasks.

Gender and gaze gesture recognition for human-computer interaction

We introduce a gender recognition algorithm that adopts Fisher Vectors.We propose an unsupervised modular approach for eye centre localisation.We design gaze gestures intended for controlling a HCI system remotely.We develop a HCI system as a type of assistive technology.All the proposed methods are highly accurate, efficient and robust. The identification of visual cues in facial images has been widely explored in the broad area of computer vision. However theoretical analyses are often not transformed into widespread assistive Human-Computer Interaction (HCI) systems, due to factors such as inconsistent robustness, low efficiency, large computational expense or strong dependence on complex hardware. We present a novel gender recognition algorithm, a modular eye centre localisation approach and a gaze gesture recognition method, aiming to escalate the intelligence, adaptability and interactivity of HCI systems by combining demographic data (gender) and behavioural data (gaze) to enable development of a range of real-world assistive-technology applications.The gender recognition algorithm utilises Fisher Vectors as facial features which are encoded from low-level local features in facial images. We experimented with four types of low-level features: greyscale values, Local Binary Patterns (LBP), LBP histograms and Scale Invariant Feature Transform (SIFT). The corresponding Fisher Vectors were classified using a linear Support Vector Machine. The algorithm has been tested on the FERET database, the LFW database and the FRGCv2 database, yielding 97.7%, 92.5% and 96.7% accuracy respectively.The eye centre localisation algorithm has a modular approach, following a coarse-to-fine, global-to-regional scheme and utilising isophote and gradient features. A Selective Oriented Gradient filter has been specifically designed to detect and remove strong gradients from eyebrows, eye corners and self-shadows (which sabotage most eye centre localisation methods). The trajectories of the eye centres are then defined as gaze gestures for active HCI. The eye centre localisation algorithm has been compared with 10 other state-of-the-art algorithms with similar functionality and has outperformed them in terms of accuracy while maintaining excellent real-time performance.The above methods have been employed for development of a data recovery system that can be employed for implementation of advanced assistive technology tools. The high accuracy, reliability and real-time performance achieved for attention monitoring, gaze gesture control and recovery of demographic data, can enable the advanced human-robot interaction that is needed for developing systems that can provide assistance with everyday actions, thereby improving the quality of life for the elderly and/or disabled.

Machine vision 3D skin texture analysis for detection of melanoma

Purpose – The purpose of this paper is to describe innovative machine vision methods that have been employed for the capture and analysis of 3D skin textures; and the resulting potential for assisting with identification of suspicious lesions in the detection of skin cancer.Design/methodology/approach – A machine vision approach has been employed for analysis of 3D skin textures. This involves an innovative application of photometric stereo for the capture of the textures, and a range of methods for analysing and quantifying them, including statistical methods and neural networks.Findings – 3D skin texture has been identified as a useful indicator of skin cancer. It can be used to improve realism of virtual skin reconstructions in tele‐dermatology. 3D texture features can also be combined with 2D features to obtain a more robust classifier for improving diagnostic accuracy, thereby assisting with the long‐term goal of implementing computer‐aided diagnostics for skin cancer.Originality/value – The device d...

Surface texture inspection using conventional techniques applied to a photometrically acquired bump map

The paper presents a new approach to texture analysis. The need for a more formal definition of the term surface texture is first identified, and an appropriate texture taxonomy proposed. A method of analysis is described, synthesising innovative elements of machine vision and computer graphics to achieve an object‐centred inspection technique, which is both robust and flexible in application. A selection of experimental results is presented in the paper.

Eye center localization and gaze gesture recognition for human–computer interaction

This paper introduces an unsupervised modular approach for accurate and real-time eye center localization in images and videos, thus allowing a coarse-to-fine, global-to-regional scheme. The trajectories of eye centers in consecutive frames, i.e., gaze gestures, are further analyzed, recognized, and employed to boost the human-computer interaction (HCI) experience. This modular approach makes use of isophote and gradient features to estimate the eye center locations. A selective oriented gradient filter has been specifically designed to remove strong gradients from eyebrows, eye corners, and shadows, which sabotage most eye center localization methods. A real-world implementation utilizing these algorithms has been designed in the form of an interactive advertising billboard to demonstrate the effectiveness of our method for HCI. The eye center localization algorithm has been compared with 10 other algorithms on the BioID database and six other algorithms on the GI4E database. It outperforms all the other algorithms in comparison in terms of localization accuracy. Further tests on the extended Yale Face Database b and self-collected data have proved this algorithm to be robust against moderate head poses and poor illumination conditions. The interactive advertising billboard has manifested outstanding usability and effectiveness in our tests and shows great potential for benefiting a wide range of real-world HCI applications.

In vivo measurement of skin microrelief using photometric stereo in the presence of interreflections

This paper proposes and describes an implementation of a photometric stereo-based technique for in vivo assessment of three-dimensional (3D) skin topography in the presence of interreflections. The proposed method illuminates skin with red, green, and blue colored lights and uses the resulting variation in surface gradients to mitigate the effects of interreflections. Experiments were carried out on Caucasian, Asian, and African American subjects to demonstrate the accuracy of our method and to validate the measurements produced by our system. Our method produced significant improvement in 3D surface reconstruction for all Caucasian, Asian, and African American skin types. The results also illustrate the differences in recovered skin topography due to the nondiffuse bidirectional reflectance distribution function (BRDF) for each color illumination used, which also concur with the existing multispectral BRDF data available for skin.

Facial Reconstruction and Alignment Using Photometric Stereo and Surface Fitting

This paper presents a novel 3D face shape capture device suitable for practical face recognition applications. A new surface fitting based face alignment algorithm is then presented to normalise the pose in preparation for recognition. The 3D data capture consists of a photometric stereo rig capable of acquiring four images, each with a different light source direction, in just 15ms. This high-speed data acquisition process allows all images to be taken without significant movement between images, a previously highly restrictive disadvantage of photometric stereo. The alignment algorithm is based on fitting bivariate polynomials to the reconstructed faces and calculating the pitch, roll and yaw from the resulting polynomial parameters. Successful experiments are performed on a range of faces and pose variations.

Sampling Light Field for Photometric Stereo

—To implement the photometric stereo technique, the radiance distribution of the respective light sources from the different illumination directions must be accurately known. Most previous work has tended to assume distance point sources, so that a collimated and uniform illumination distribution can be approximated, thereby allowing the photometric stereo problem to be easily solved in a linear way. However, there can be significant practical difficulties in realizing such idealized light sources in real world applications. In addition, the strategy of using distant light sources produces a low signal/noise ratio for the system, and is also unsuitable for applications where setup space is limited. These problems potentially limit new opportunities for the wider applications of photometric stereo beyond the research laboratory in evolving areas such as industrial inspection, security and medical applications. This paper proposes a compensation method for illumination radiance to allow the possibility of employing normal low-cost commercial light sources. A flat diffuse surface with either homogeneous or heterogeneous albedo distribution is used to sample the radiance distribution before implementing photometric stereo. The unevenly distributed light radiance is eliminated by using the acquired reference information. The experimental results demonstrate the efficacy of the proposed method.

2.5D Facial Expression Recognition using Photometric Stereo and the Area Weighted Histogram of Shape Index

In this work we argue that the high frequency spatial variations in the topological information of the face are important for Facial Expression Recognition. Stereo and laser scanner based datasets currently used are inherently regularized, resulting in the loss of high frequency information. We test our hypothesis on the dense gradient field from Photometric Stereo which preserves this high frequency information. To overcome the geometric artefacts introduced through the integration of the gradient field we take a local approach and, assuming piecewise smoothness, we directly extract the second order differential geometry. We introduce the Area Weighted Histogram of Shape Index which is invariant to both scale and orientation and extend this to a localized histogram approach. Rather than using heuristically chosen areas of the face we use a data driven approach based on the Fisher Discriminant Ratio to identify the most discriminatory regions of the face. Using a non-linear Support Vector Machine we are able to recognize the six prototypic expressions of the face. We carry out analysis on the Binghamton BU4DFE database as well as a small Photometric Stereo dataset and show that the high frequency information preserved by Photometric Stereo may be highly useful for automatic Facial Expression Recognition.