Sezer Ulukaya

A comparative study of face landmarking techniques

Face landmarking, defined as the detection and localization of certain characteristic points on the face, is an important intermediary step for many subsequent face processing operations that range from biometric recognition to the understanding of mental states. Despite its conceptual simplicity, this computer vision problem has proven extremely challenging due to inherent face variability as well as the multitude of confounding factors such as pose, expression, illumination and occlusions. The purpose of this survey is to give an overview of landmarking algorithms and their progress over the last decade, categorize them and show comparative performance statistics of the state of the art. We discuss the main trends and indicate current shortcomings with the expectation that this survey will provide further impetus for the much needed high-performance, real-life face landmarking operating at video rates.

Gaussian mixture model based estimation of the neutral face shape for emotion recognition

When the goal is to recognize the facial expression of a person given an expressive image, there are mainly two types of information encoded in the image that we have to deal with: identity-related information and expression related information. Alleviating the identity-related information, for example by using an image of the same person with a neutral facial expression, increases the success of facial expression recognition algorithms. However, the neutral face image corresponding to an expressive face may not always be available or known, which is known as the baseline problem. In this work, we propose a general solution to the baseline problem by estimating the unknown neutral face shape of an expressive face image using a dictionary of neutral face shapes. The dictionary is formed using a Gaussian Mixture Model fitting method. We also present a method of fusing shape-based (geometrical) features with appearance based features by calculating them only around the most discriminative geometrical facial features, which have been selected automatically. Experimental results on three widely used facial expression databases as well as cross database analysis show that utilization of the estimated neutral face shapes increases the facial expression recognition rate significantly, when the person-specific neutral face information is not available.

A lung sound classification system based on the rational dilation wavelet transform

In this work, a wavelet based classification system that aims to discriminate crackle, normal and wheeze lung sounds is presented. While the previous works related with this problem use constant low Q-factor wavelets, which have limited frequency resolution and can not cope with oscillatory signals, in the proposed system, the Rational Dilation Wavelet Transform, whose Q-factors can be tuned, is employed. Proposed system yields an accuracy of 95 % for crackle, 97 % for wheeze, 93.50 % for normal and 95.17 % for total sound signal types using energy feature subset and proposed approach is superior to conventional low Q-factor wavelet analysis.

Estimation of the neutral face shape using Gaussian Mixture Models

We present a Gaussian Mixture Model (GMM) fitting method for estimating the unknown neutral face shape for frontal facial expression recognition using geometrical features. Subtracting the estimated neutral face, which is related to the identity-specific component of the shape leaves us with the component related to the variations resulting from facial expressions. Experimental results on the Extended Cohn-Kanade (CK+) database show that subtracting the estimated neutral face shape gives better emotion recognition rates as compared to classifying the geometrical facial features directly, when the person-specific neutral face shape is not available. We also experimentally evaluate two different geometric facial feature extraction methods for emotion recognition. The average emotion recognition rates achieved with the proposed neutral shape estimation method and coordinate based features is 88%, which is higher than the baseline results presented in the literature, although we do not use the person-specific neutral shapes (94% if we use), and any appearance based features.

Feature extraction using time-frequency analysis for monophonic-polyphonic wheeze discrimination

The aim of this study is monophonic-polyphonic wheeze episode discrimination rather than the conventional wheeze (versus non-wheeze) episode detection. We used two different methods for feature extraction to discriminate monophonic and polyphonic wheeze episodes. One of the methods is based on frequency analysis and the other is based on time analysis. Frequency analysis based method uses ratios of quartile frequencies to exploit the difference in the power spectrum. Time analysis based method uses mean crossing irregularity to exploit the difference in periodicity in the time domain. Both methods are applied on the data before and after an image processing based preprocessing step. Calculated features are used in classification both individually and in combinations. Support vector machine, k-nearest neighbor and Naive Bayesian classifiers are adopted in leave-one-out scheme. A total of 121 monophonic and 110 polyphonic wheeze episodes are used in the experiments, where the best classification performances are 71.45% for time domain based features, 68.43% for frequency domain based features, and 75.78% for a combination of selected best features.

Respiratory sound classification using perceptual linear prediction features for healthy - Pathological diagnosis

This study proposes a new model and feature extraction method for the classification of multi-channel respiratory sound data with the final aim of building a diagnosis aid tool for the medical doctor. Fourteen-channel data are processed separately and combined at feature level and fed to the support vector machines with radial basis kernel. Healthy-pathological subject based binary classification is employed where the recall rates for the healthy class and pathological class are 95 percent and 80 percent, respectively. The minimum precision rate is 80 percent. The method, when supported by additional features (adventitious sound frequency, type, etc.), may be employed in clinical practice as an aiding decision maker.

An Automated Lung Sound Preprocessing and Classification System Based OnSpectral Analysis Methods

In this work, respiratory sounds are classified into four classes in the presence of various noises (talking, coughing, motion artefacts, heart and intestinal sounds) using support vector machine classifier with radial basis function kernel. The four classes can be listed as normal, wheeze, crackle and crackle plus wheeze. Crackle and wheeze adventitious sounds have opposite behavior in the time-frequency domain. In order to better represent and resolve the discriminative characteristics of adventitious sounds, non-linear novel spectral feature extraction algorithms are proposed to be employed in four class classification problem. The proposed algorithm, which has achieved 49.86% accuracy on a very challenging and rich dataset, is a promising tool to be used as preprocessor in lung disease decision support systems.

Overcomplete discrete wavelet transform based respiratory sound discrimination with feature and decision level fusion

Abstract Background and objective Crackle, wheeze and normal lung sound discrimination is vital in diagnosing pulmonary diseases. Previous works suffer from limited frequency resolution and lack of the ability to deal with oscillatory signals (wheezes). The main objective of this study is to propose a novel wavelet based lung sound classification system that is capable of adaptively representing crackle, wheeze and normal lung sound signal time-frequency properties. Methods A method which is based on rational dilation wavelet transform is proposed to classify lung sounds into three main categories, namely, normal, wheeze and crackle. Six different feature extraction methods were used with five different classifiers all of which were compared with the proposed method on 600 lung sound episodes in a cross validation scheme. Six statistical subset features were extracted from raw features and fed into classifiers. After comparative evaluation of the proposed method, an ensemble learning scheme was built to increase the performance of the proposed method. Results It has been shown that performance of the proposed method was superior to previous methods in terms of accuracy. Moreover, its computational time was far less than its nearest competitor (S transform). It has also been shown that the proposed method was able to cope with oscillatory type signals as well as transient sounds performing 95.17% average accuracy for energy subset and 97.38% ensemble average accuracy showing a promising time-frequency tool for biological signals. Conclusions The proposed method has shown better performance even using only one subset of extracted features. It provides better time-frequency resolution for all types of signals of interest and is less redundant than continuous wavelet transform and significantly faster than its nearest competitor.

Resonance based respiratory sound decomposition aiming at localization of crackles in noisy measurements

In this work, resonance based decomposition of lung sounds that aims to separate wheeze, crackle and vesicular sounds into three individual channels while automatically localizing crackles for both synthetic and real data is presented. Previous works focus on stationary-non stationary discrimination to separate crackles and vesicular sounds disregarding wheezes which are stationary as compared to crackles. However, wheeze sounds include important cues about the underlying pathology. Using two different threshold methods and synthetic sound generation scenarios in the presence of wheezes, resonance based decomposition performs 89.5 % crackle localization recall rate for white Gaussian noise and 98.6 % crackle localization recall rate for healthy vesicular sound treated as noise at low signal-to-noise ratios. Besides, an adaptive threshold determination which is independent from the channel at which it will be applied is used and is found to be robust to noise.In this work, resonance based decomposition of lung sounds that aims to separate wheeze, crackle and vesicular sounds into three individual channels while automatically localizing crackles for both synthetic and real data is presented. Previous works focus on stationary-non stationary discrimination to separate crackles and vesicular sounds disregarding wheezes which are stationary as compared to crackles. However, wheeze sounds include important cues about the underlying pathology. Using two different threshold methods and synthetic sound generation scenarios in the presence of wheezes, resonance based decomposition performs 89.5 % crackle localization recall rate for white Gaussian noise and 98.6 % crackle localization recall rate for healthy vesicular sound treated as noise at low signal-to-noise ratios. Besides, an adaptive threshold determination which is independent from the channel at which it will be applied is used and is found to be robust to noise.

A novel method for determination of wheeze type

Among respiratory disorders, obstructive diseases such as asthma and chronic obstructive pulmonary disease (COPD) constitute an important group. To our knowledge, there does not exist a study in the literature that quantifies the relationship between the type of wheeze and the type or severity of the disease. This study, aims at classifying wheeze type rather than classical normal-wheeze sound classification studies in the literature. In this study, we propose a method based on Multiple Signal Classification (MUSIC) algorithm to differentiate between monophonic and polyphonic wheezes, without a need for pre-training the algorithm. The algorithm determines the true labels of monophonic and polyphonic wheezes with 100% and 78% accuracy, respectively. Since there does not exist a method in the literature that has been proposed specifically for this problem, only the results of the most relevant few studies have been presented. Since the proposed system can directly estimate the frequency, we consider the method proposed here would be a useful quantification method for further studies in medical literature, on finding correlations between wheezes and disorders.