Juan M. Fontana

A Sensor System for Automatic Detection of Food Intake Through Non-Invasive Monitoring of Chewing

Objective and automatic sensor systems to monitor ingestive behavior of individuals arise as a potential solution to replace inaccurate method of self-report. This paper presents a simple sensor system and related signal processing and pattern recognition methodologies to detect periods of food intake based on non-invasive monitoring of chewing. A piezoelectric strain gauge sensor was used to capture movement of the lower jaw from 20 volunteers during periods of quiet sitting, talking and food consumption. These signals were segmented into non-overlapping epochs of fixed length and processed to extract a set of 250 time and frequency domain features for each epoch. A forward feature selection procedure was implemented to choose the most relevant features, identifying from 4 to 11 features most critical for food intake detection. Support vector machine classifiers were trained to create food intake detection models. Twenty-fold cross-validation demonstrated per-epoch classification accuracy of 80.98% and a fine time resolution of 30 s. The simplicity of the chewing strain sensor may result in a less intrusive and simpler way to detect food intake. The proposed methodology could lead to the development of a wearable sensor system to assess eating behaviors of individuals.

Automatic Ingestion Monitor: A Novel Wearable Device for Monitoring of Ingestive Behavior

Objective monitoring of food intake and ingestive behavior in a free-living environment remains an open problem that has significant implications in study and treatment of obesity and eating disorders. In this paper, a novel wearable sensor system (automatic ingestion monitor, AIM) is presented for objective monitoring of ingestive behavior in free living. The proposed device integrates three sensor modalities that wirelessly interface to a smartphone: a jaw motion sensor, a hand gesture sensor, and an accelerometer. A novel sensor fusion and pattern recognition method was developed for subject-independent food intake recognition. The device and the methodology were validated with data collected from 12 subjects wearing AIM during the course of 24 h in which both the daily activities and the food intake of the subjects were not restricted in any way. Results showed that the system was able to detect food intake with an average accuracy of 89.8%, which suggests that AIM can potentially be used as an instrument to monitor ingestive behavior in free-living individuals.

Energy intake estimation from counts of chews and swallows

Current, validated methods for dietary assessment rely on self-report, which tends to be inaccurate, time-consuming, and burdensome. The objective of this work was to demonstrate the suitability of estimating energy intake using individually-calibrated models based on Counts of Chews and Swallows (CCS models). In a laboratory setting, subjects consumed three identical meals (training meals) and a fourth meal with different content (validation meal). Energy intake was estimated by four different methods: weighed food records (gold standard), diet diaries, photographic food records, and CCS models. Counts of chews and swallows were measured using wearable sensors and video analysis. Results for the training meals demonstrated that CCS models presented the lowest reporting bias and a lower error as compared to diet diaries. For the validation meal, CCS models showed reporting errors that were not different from the diary or the photographic method. The increase in error for the validation meal may be attributed to differences in the physical properties of foods consumed during training and validation meals. However, this may be potentially compensated for by including correction factors into the models. This study suggests that estimation of energy intake from CCS may offer a promising alternative to overcome limitations of self-report.

A robust classification scheme for detection of food intake through non-invasive monitoring of chewing

Automatic methods for food intake detection are needed to objectively monitor ingestive behavior of individuals in a free living environment. In this study, a pattern recognition system was developed for detection of food intake through the classification of jaw motion. A total of 7 subjects participated in laboratory experiments that involved several activities of daily living: talking, walking, reading, resting and food intake while being instrumented with a wearable jaw motion sensor. Inclusion of such activities provided a high variability to the sensor signal and thus challenged the classification task. A forward feature selection process decided on the most appropriate set of features to represent the chewing signal. Linear and RBF Support Vector Machine (SVM) classifiers were evaluated to find the most suitable classifier that can generalize the high variability of the input signal. Results showed that an average accuracy of 90.52% can be obtained using Linear SVM with a time resolution of 15 sec.

Estimation of feature importance for food intake detection based on Random Forests classification

Selection of the most representative features is important for any pattern recognition system. This paper investigates the importance of time domain (TD) and frequency domain (FD) features used for automatic food intake detection in a wearable sensor system by using Random Forests classification. Features were extracted from signals collected using 3 different sensor modalities integrated into the Automatic Ingestion Monitor (AIM): a jaw motion sensor, a hand gesture sensor and an accelerometer. Data was collected from 12 subjects wearing AIM in free-living for a 24-hr period where they experienced unrestricted intake. Features from the sensor signals were used to train the Random Forests classifier that estimated the importance of each feature as part of the training process. Results indicated that FD features from the jaw motion signal and TD features from the accelerometer signal were the most relevant features for food intake detection.

Automatic identification of the number of food items in a meal using clustering techniques based on the monitoring of swallowing and chewing

The number of distinct foods consumed in a meal is of significant clinical concern in the study of obesity and other eating disorders. This paper proposes the use of information contained in chewing and swallowing sequences for meal segmentation by food types. Data collected from experiments of 17 volunteers were analyzed using two different clustering techniques. First, an unsupervised clustering technique, Affinity Propagation (AP), was used to automatically identify the number of segments within a meal. Second, performance of the unsupervised AP method was compared to a supervised learning approach based on Agglomerative Hierarchical Clustering (AHC). While the AP method was able to obtain 90% accuracy in predicting the number of food items, the AHC achieved an accuracy >95%. Experimental results suggest that the proposed models of automatic meal segmentation may be utilized as part of an integral application for objective Monitoring of Ingestive Behavior in free living conditions.

Swallowing detection by sonic and subsonic frequencies: A comparison

The detection of swallowing events by acoustic means represents an important tool to assess and diagnose swallowing disorders as well as to objectively monitor ingestive behavior of individuals. Acoustic sensors used to register swallowing sounds may also capture sound artifacts arising from intrinsic speech and external noise affecting the detection. In this paper we tested if subsonic frequencies are less prone to artifacts from speech, chewing and other intrinsic sounds than sonic frequencies. A simple method using a throat and an ambient microphone was employed to compare the swallowing detection accuracy by acoustic signals acquired in the sonic (20–2500 Hz) and subsonic (≤ 5 Hz) ranges. Averaged recall values were higher than 85% for both ranges. However, averaged precision values of 50% for subsonic frequencies and of 42% for sonic frequencies were caused by a high number of false positives. These results indicated no significant difference between averaged precision values which may suggest that subsonic frequencies were not less prone to intrinsic sound artifacts than frequencies in the sonic range. Further examination with the addition of a signal classification layer is proposed as a future step to confirm this statement.

Detection and Characterization of Food Intake by Wearable Sensors

Food intake provides energy and nutrients to sustain human life. Studying the ingestive behavior of individuals is of particular interest for understanding and treatment of medical conditions strongly associated with food intake, such as obesity and eating disorders. Traditionally, ingestive behavior in humans has been assessed through self-monitoring of food intake. However, this approach is inaccurate, time consuming, and suffers from the observation and misreporting effects. Wearable sensors present a compelling alternative to overcome limitations of self-reporting methods. These sensors can potentially provide more objective measurements of food intake by monitoring physiological processes related to one or more stages of the food consumption process: hand-to-mouth gestures, bites, chewing, or swallowing. Specialized signal processing and pattern recognition methodologies use the sensor data to automatically detect and characterize each intake episode. Particularly, timing and duration of the meals, the mass and volume of ingestion, caloric and nutritional content of a meal, and the rate of ingestion could potentially be estimated from sensor data. This chapter presents an overview of the wearable sensors and accompanying methodologies that have been proposed for monitoring ingestive behavior in humans.

A Comparative Study of Food Intake Detection Using Artificial Neural Network and Support Vector Machine

In Machine Learning applications, the selection of the classification algorithm depends on the problem at hand. This paper provides a comparison of the performance of the Support Vector Machine (SVM) and the Artificial Neural Network (ANN) for food intake detection. A combination of time domain (TD) and frequency domain (FD) features, extracted from signals captured using a jaw motion sensor, were used to train both types of classifiers. Data were collected from 12 subjects in free-living for a period of 24-hrs under unrestricted conditions. ANN with a different number of hidden layer neurons and SVMs with different kernels were trained using a leave one out cross validation scheme. ANN achieved an average accuracy of 86.86 ± 6.5 % whereas SVM (with linear kernel) achieved an average classification accuracy of 81.93 ± 9.22 %. Data collected from an independent subject in a separate study were used to evaluate the performance of these classifiers in-terms of the number of meals detected per day resulting in an accuracy of 72.72% for ANN and 63.63% for SVM. The results suggest that ANN may perform better than SVM for this specific problem.

Analysis of Electrode Shift Effects on Wavelet Features Embedded in a Myoelectric Pattern Recognition System

Myoelectric pattern recognition systems can translate muscle contractions into prosthesis commands; however, the lack of long-term robustness of such systems has resulted in low acceptability. Specifically, socket misalignment may cause disturbances related to electrodes shifting from their original recording location, which affects the myoelectric signals (MES) and produce degradation of the classification performance. In this work, the impact of such disturbances on wavelet features extracted from MES was evaluated in terms of classification accuracy. Additionally, two principal component analysis frameworks were studied to reduce the wavelet feature set. MES from seven able-body subjects and one subject with congenital transradial limb loss were studied. The electrode shifts were artificially introduced by recording signals during six sessions for each subject. A small drop in classification accuracy from 93.8% (no disturbances) to 88.3% (with disturbances) indicated that wavelet features were able to adapt to the variability introduced by electrode shift disturbances. The classification performance of the reduced feature set was significantly lower than the performance of the full wavelet feature set. The results observed in this study suggest that the effect of electrode shift disturbances on the MES can potentially be mitigated by using wavelet features embedded in a pattern recognition system.