Masood Mehmood Khan

Classifying pretended and evoked facial expressions of positive and negative affective states using infrared measurement of skin temperature

Earlier researchers were able to extract the transient facial thermal features from thermal infrared images (TIRIs) to make binary distinctions between the expressions of affective states. However, effective human-computer interaction would require machines to distinguish between the subtle facial expressions of affective states. This work, for the first time, attempts to use the transient facial thermal features for recognizing a much wider range of facial expressions. A database of 324 time-sequential, visible-spectrum, and thermal facial images was developed representing different facial expressions from 23 participants in different situations. A novel facial thermal feature extraction, selection, and classification approach was developed and invoked on various Gaussian mixture models constructed using: neutral and pretended happy and sad faces, faces with multiple positive and negative facial expressions, faces with neutral and six (pretended) basic facial expressions, and faces with evoked happiness, sadness, disgust, and anger. This work demonstrates that (1) infrared imaging can be used to observe the affective-state-specific facial thermal variations, (2) pixel-grey level analysis of TIRIs can help localise significant facial thermal feature points along the major facial muscles, and (3) cluster-analytic classification of transient thermal features can help distinguish between the facial expressions of affective states in an optimized eigenspace of input thermal feature vectors. The observed classification results exhibited influence of a Gaussian mixture models structure on classifier-performance. The work also unveiled some pertinent aspects of future research on the use of facial thermal features in automated facial expression classification and affect recognition.

Video shot boundary detection based on candidate segment selection and transition pattern analysis

Video shot boundary detection or shot segmentation is an integral part of semantic video analysis. The objective of this process is to automatically detect the boundary region in video that further segment the video into meaningful shot, scene and so on. Video frame feature representation therefore plays an important role in the process where it directly affects the overall performance of the system. The transition points between meaningful scenes can be emphasised by the extracted features. In this paper, a combination of global and local feature descriptors is implemented to represent the temporal characteristic in video. Motivated by computational efficiency and practical implementation, a video shot boundary detection scheme using adaptive thresholding is proposed. Candidate segment selection and transition pattern analysis are implemented by the dissimilarity score between video frames. The performance evaluation is constructed on a golf video dataset using the precision and recall performance measures.

Thermographic investigation of osseous stress pathology

The debilitating pathology of stress fracture accounts for 10% of all athletic injuries[2], with prevalence as high as 20% in modern military basic training cohorts [3]. Increasing concerns surrounding adverse effects of radiology [5], combined with the 12.5% contribution of diagnostic imaging to Australian Medicare benefits paid in 2009–10 [6], have prompted the search for alternative/adjunct electronic decision support systems[7]. Within conducive physioanatomic milieu, thermal infrared imaging (TIRI) may feasibly be used to remotely detect and topographically map diagnostically useful signs of suprathreshold thermodynamic pathophysiology. This paper details a three month clinical pilot study into TIRI-based detection of osseous stress pathology in the lower legs of Australian Army basic trainees. A dataset of over 500 TIRIs was amassed. The apparent ‘normal’ thermal profile of the anterior aspect of the asymptomatic lower leg is topographically defined and validated against current thermophysiological theory [8] via cadaveric dissection.

Multi-Modal Visual Features-Based Video Shot Boundary Detection

One of the essential pre-processing steps of semantic video analysis is the video shot boundary detection (SBD). It is the primary step to segment the sequence of video frames into shots. Many SBD systems using supervised learning have been proposed for years; however, the training process still remains its principal limitation. In this paper, a multi-modal visual features-based SBD framework is employed that aims to analyze the behaviors of visual representation in terms of the discontinuity signal. We adopt a candidate segment selection that performs without the threshold calculation but uses the cumulative moving average of the discontinuity signal to identify the position of shot boundaries and neglect the non-boundary video frames. The transition detection is structurally performed to distinguish candidate segment into a cut transition and a gradual transition, including fade in/out and logo occurrence. Experimental results are evaluated using the golf video clips and the TREC2001 documentary video data set. Results show that the proposed SBD framework can achieve good accuracy in both types of video data set compared with other proposed SBD methods.

A study of discriminant visual descriptors for sport video shot boundary detection

Video shot boundary detection is the process of automatically detecting the meaningful boundary content in video. Most shot boundary categorisation techniques use features extracted from the video frames to highlight the transition points between meaningful scenes. In this paper, a combination of global and local feature descriptors is proposed to represent the temporal characteristic in video. Motivated by the practical applications with moderate computational time, a video shot boundary detection scheme using supervised learning is proposed. The performance evaluation is constructed on a golf video dataset using the precision and recall performance measures.

Efficacy of biophysiological measurements at FTFPs for facial expression classification: A validation

Recent works suggest that thermal intensity values (TIVs) measured around the facial thermal feature points (FTFPs) can help in distinguishing between the facial expression of affective states. This work investigates if the average pixel grey-levels, instead of TIVs, measured in sub-image masks around the FTFPs allow classifying facial expressions. Thermal infrared images from the IEEE OTCBVS database were used to distinguish between facial expressions. The pixel grey-levels measured in sub-image masks were used to measure, for each individual, the Euclidean distance between images of different facial expressions. Linear discriminant analysis was performed to obtain hyper-planes for separating the clusters of sample images. Significant pixel grey-level differences were observed at FTFPs between three facial expressions; neutral, happy, and angry. More than 96 of the original images in a three-expression Gaussian mixture model were separable and clustered around distant centroids in a discriminant space.

Thermal infrared imaging: Toward diagnostic medical capability

Thermal infrared imaging (TIRI) employs a focal plane array (FPA) of infrared detectors, with associated optics and optoelectronics to remotely detect and topographically map thermal emittance. Thermal and optical properties of human physioanatomy are not fully understood yet confounding diagnostic interpretation of human TIRIs. Elucidation of the specific physical mechanism via which thermal emission arises from human anatomy in-vivo requires empirical investigation under objective clinical protocols. This paper characterizes the fundamental architecture of the clinical TIRI system with a view to facilitation of objective protocol development, elucidation of the mechanism/s of human thermal infrared emittance, and eventual validation of TIRI as a diagnostic medical tool. Relevant recent and ongoing empirical studies by the authors are also summarized.

Toward Use of Facial Thermal Features in Dynamic Assessment of Affect and Arousal Level

Automated assessment of affect and arousal level can help psychologists and psychiatrists in clinical diagnoses; and may enable affect-aware robot-human interaction. This work identifies major difficulties in automating affect and arousal assessment and attempts to overcome some of them. We first analyze thermal infrared images and examine how changes in affect and/or arousal level would cause hæmodynamic variations, concentrated along certain facial muscles. These concentrations are used to measure affect/arousal induced facial thermal variations. In step-1 of a 2-step pattern recognition schema, ‘between-affect’ and ‘between-arousal-level’ variations are used to derive facial thermal features as Principal Components (PCs) of the facial thermal measurements. The most influential of these PCs are used to cluster the feature space for different affects and subsequently assign a set of thermal features to an affect cluster. In step-2, affect clusters are partitioned into high, medium and mild arousal levels. The distance between a test face vector and the centroids of sub-clusters at three arousal levels belonging to a single affective state, identified from step-1, is used to determine the arousal level of the identified affective state.

A tongue-activated emergency beacon for immobile patients

This paper reports the design and implementation of an infrared signal transmitting tongue activated emergency beacon. This low-cost, simple and reliable device can help immobile patients communicate with the medical staff in the event of an emergency without interfering with other equipment. The physical dimensions of this device were minimized to provide flexibility and suit the most vulnerable and impaired patients. The presented sensor-microcontroller configuration results in a robust and intelligent functionality that would allow this device to outperform many of the commercially available systems used in similar environments.

Closed-loop Petri Net model for implementing an affective-state expressive robotic face

A closed-loop Petri Net (PN) model was developed to exhibit, maintain and, withdraw facial expressions of six basic affective states, in a human-like manner, on a robotic face. The PN model was aimed to enable execution of major facial muscles-like-interactions between segments of a latex-made facial mask. The muscle-like interactions were based upon the widely accepted and used facial action coding systems. In order to validate the PN model, the facial mask, mounted on a 3-D printed artificial skull, was used as a robotic face. Human facial muscles like movements, generated on the surface of the facial mask, were able to express positive and negative affective states. Audio signals were used as stimuli for eliciting expressions of affective states. Results show that an event driven discrete model would suffice deterministic representation of a finite number of affective states on an artificial robotic face.