Jasper Snoek

Basset: Learning the regulatory code of the accessible genome with deep convolutional neural networks

The complex language of eukaryotic gene expression remains incompletely understood. Despite the importance suggested by many noncoding variants statistically associated with human disease, nearly all such variants have unknown mechanisms. Here, we address this challenge using an approach based on a recent machine learning advance-deep convolutional neural networks (CNNs). We introduce the open source package Basset to apply CNNs to learn the functional activity of DNA sequences from genomics data. We trained Basset on a compendium of accessible genomic sites mapped in 164 cell types by DNase-seq, and demonstrate greater predictive accuracy than previous methods. Basset predictions for the change in accessibility between variant alleles were far greater for Genome-wide association study (GWAS) SNPs that are likely to be causal relative to nearby SNPs in linkage disequilibrium with them. With Basset, a researcher can perform a single sequencing assay in their cell type of interest and simultaneously learn that cells chromatin accessibility code and annotate every mutation in the genome with its influence on present accessibility and latent potential for accessibility. Thus, Basset offers a powerful computational approach to annotate and interpret the noncoding genome.

Automated detection of unusual events on stairs

This paper presents a method for automatically detecting unusual human events on stairs from video data. The motivation is to provide a tool for biomedical researchers to rapidly find the events of interest within large quantities of video data. Our system identifies potential sequences containing anomalies, and reduces the amount of data that needs to be searched by a human. We compute two sets of features from a video of a person descending a stairwell. The first set of features are the foot positions and velocities. We track both feet using a mixed state particle filter with an appearance model based on histograms of oriented gradients. We compute expected (most likely) foot positions given the state of the filter at each frame. The second set of features are the parameters of the mean optical flow over a foreground region. Our final classification system inputs these two sets of features into a hidden Markov model (HMM) to analyse the spatio-temporal progression of the stair descent. A single HMM is trained on sequences of normal stair use, and a threshold on sequence likelihoods is used to detect unusual events in new data. We demonstrate our system on a data set with five people descending a set of stairs in a laboratory environment. We show how our system can successfully detect nearly all anomalous events, with a low false positive rate. We discuss limitations and suggest improvements to the system.

Towards a single sensor passive solution for automated fall detection

Falling in the home is one of the major challenges to independent living among older adults. The associated costs, coupled with a rapidly growing elderly population, are placing a burden on healthcare systems worldwide that will swiftly become unbearable. To facilitate expeditious emergency care, we have developed an artificially intelligent camera-based system that automatically detects if a person within the field-of-view has fallen. The system addresses concerns raised in earlier work and the requirements of a widely deployable in-home solution. The presented prototype utilizes a consumer-grade camera modified with a wide-angle lens. Machine learning techniques applied to carefully engineered features allow the system to classify falls at high accuracy while maintaining invariance to lighting, environment and the presence of multiple moving objects. This paper describes the system, outlines the algorithms used and presents empirical validation of its effectiveness.

Automated Detection of Unusual Events on Stairs

This paper presents a method for automatically detecting and recognising unusual events on stairs from video data. The motivation is to provide a tool for biomedical researchers to rapidly find and analyse the events of interest within large quantities of video data. Our system identifies potential sequences containing anomalies, and reduces the amount of data that needs to be searched by a human. We apply adaptive background subtraction to segment the person using the stairs, followed by affine flow computation over the segmented region. A hidden Markov model (HMM) is then used to analyse the temporal progression of the affine features. A single HMM is trained on sequences of normal stair use, and a threshold is used to detect unusual events in new data. We also introduce a temporal segmentation method using a conditional random field (CRF). We demonstrate our system on a data set with three persons.

Video analysis for identifying human operation difficulties and faucet usability assessment

As the world struggles to cope with a growing elderly population, concerns of how to preserve independence are becoming increasingly acute. A major hurdle to independent living is the inability to use everyday household objects. This work aims to automate the assessment of product usability for the elderly population using the tools of computer vision and machine learning. A novel video analysis technique is presented that performs temporal segmentation of video containing human-product interaction and automatically identifies time segments in which the human has difficulties in operating the product. The method has applications in the automatic assessment of the usability of various product designs via measuring the frequency of operation difficulties. The approach is applied to a case study of water faucet design for the older adult population with dementia. Experiments in the automatic analysis of a large database of real-world recorded videos confirm the effectiveness of the approach in providing valid temporal segmentation (accuracy 88.1%) and in the correct estimation of the relative advantage (or disadvantage) of one design over another in terms of operation difficulties in performing various actions.

Water Flow Detection in a Handwashing Task

Older adults suffering from Alzheimers disease often require assistance with performing simple activities of daily living, such as washing their hands in the bathroom. This severely limits their independence and places a heavy care giving burden on their family and the healthcare system. The motivation for developing a water detection algorithm is for it to be used within a system that provides reminding prompts for Alzheimers sufferers and to study product usability for older adults with cognitive impairments. Water detection in a video sequence poses a challenging computer vision problem since it is difficult to model the flow of water in a structured manner. A real-time detection system is presented here that estimates the presence of flowing water in a bathroom sink during a hand washing task based on classifying video and audio features with an overall accuracy of 88.76%. Visual features are extracted using temporal image derivatives and hand tracking is used to enhance the robustness in the visual features.

Data Mining in Bone Marrow Transplant Records to Identify Patients With High Odds of Survival

Patients undergoing a bone marrow stem cell transplant (BMT) face various risk factors. Analyzing data from past transplants could enhance the understanding of the factors influencing success. Records up to 120 measurements per transplant procedure from 1751 patients undergoing BMT were collected (Shariati Hospital). Collaborative filtering techniques allowed the processing of highly sparse records with 22.3% missing values. Ten-fold cross-validation was used to evaluate the performance of various classification algorithms trained on predicting the survival status. Modest accuracy levels were obtained in predicting the survival status (AUC = 0.69). More importantly, however, operations that had the highest chances of success were shown to be identifiable with high accuracy, e.g., 92% or 97% when identifying 74 or 31 recipients, respectively. Identifying the patients with the highest chances of survival has direct application in the prioritization of resources and in donor matching. For patients where high-confidence prediction is not achieved, assigning a probability to their survival odds has potential applications in probabilistic decision support systems and in combination with other sources of information.

Patient prognosis from vital sign time series: Combining convolutional neural networks with a dynamical systems approach

In this work, we propose a stacked switching vector-autoregressive (SVAR)-CNN architecture to model the changing dynamics in physiological time series for patient prognosis. The SVAR-layer extracts dynamical features (or modes) from the time-series, which are then fed into the CNN-layer to extract higher-level features representative of transition patterns among the dynamical modes. We evaluate our approach using 8-hours of minute-by-minute mean arterial blood pressure (BP) from over 450 patients in the MIMIC-II database. We modeled the time-series using a third-order SVAR process with 20 modes, resulting in first-level dynamical features of size 20×480 per patient. A fully connected CNN is then used to learn hierarchical features from these inputs, and to predict hospital mortality. The combined CNN/SVAR approach using BP time-series achieved a median and interquartile-range AUC of 0.74 [0.69, 0.75], significantly outperforming CNN-alone (0.54 [0.46, 0.59]), and SVAR-alone with logistic regression (0.69 [0.65, 0.72]). Our results indicate that including an SVAR layer improves the ability of CNNs to classify nonlinear and nonstationary time-series.

Towards Aging-in-Place: Automatic Assessment of Product Usability for Older Adults with Dementia

Considerations of how to facilitate aging-in-place are becoming increasingly pertinent as caregivers are overwhelmed by an aging population. A primary challenge to independent living is the inability to use products associated with tasks of daily living. As improving the usability of these products for the elderly will extend their independence, this work attempts to automate and expedite the assessment of usability using artificial intelligence. Video analysis is performed to temporally segment video of human-product interaction and automatically identify segments in which the human has difficulty operating the product. The approach is applied to the study of water faucet designs for older adults with dementia. Empirical analysis is performed on videos of dementia patients operating various faucet types, demonstrating the accuracy of the temporal segmentation (88.1%) and the ability to estimate the relative advantage of either design in terms of operational ease.

Unobtrusive Detection of Mild Cognitive Impairment in Older Adults Through Home Monitoring

The early detection of dementias such as Alzheimers disease can in some cases reverse, stop, or slow cognitive decline and in general greatly reduce the burden of care. This is of increasing significance as demographic studies are warning of an aging population in North America and worldwide. Various smart homes and systems have been developed to detect cognitive decline through continuous monitoring of high risk individuals. However, the majority of these smart homes and systems use a number of predefined heuristics to detect changes in cognition, which has been demonstrated to focus on the idiosyncratic nuances of the individual subjects, and thus, does not generalize. In this paper, we address this problem by building generalized linear models of home activity of older adults monitored using unobtrusive sensing technologies. We use inhomogenous Poisson processes to model the presence of the recruited older adults within different rooms throughout the day. We employ an information theoretic approach to compare the generalized linear models learned, and we observe significant statistical differences between the cognitively intact and impaired older adults. Using a simple thresholding approach, we were able to detect mild cognitive impairment in older adults with an average area under the ROC curve of 0.716 and an average area under the precision-recall curve of 0.706 using activity models estimated over a time window of 12 weeks.