Devrim Unay

Dendritic spine shape classification from two-photon microscopy images

Functional properties of a neuron are coupled with its morphology, particularly the morphology of dendritic spines. Spine volume has been used as the primary morphological parameter in order the characterize the structure and function coupling. However, this reductionist approach neglects the rich shape repertoire of dendritic spines. First step to incorporate spine shape information into functional coupling is classifying main spine shapes that were proposed in the literature. Due to the lack of reliable and fully automatic tools to analyze the morphology of the spines, such analysis is often performed manually, which is a laborious and time intensive task and prone to subjectivity. In this paper we present an automated approach to extract features using basic image processing techniques, and classify spines into mushroom or stubby by applying machine learning algorithms. Out of 50 manually segmented mushroom and stubby spines, Support Vector Machine was able to classify 98% of the spines correctly.

A joint classification and segmentation approach for dendritic spine segmentation in 2-photon microscopy images

Shape priors have been successfully used in challenging biomedical imaging problems. However when the shape distribution involves multiple shape classes, leading to a multimodal shape density, effective use of shape priors in segmentation becomes more challenging. In such scenarios, knowing the class of the shape can aid the segmentation process, which is of course unknown a priori. In this paper, we propose a joint classification and segmentation approach for dendritic spine segmentation which infers the class of the spine during segmentation and adapts the remaining segmentation process accordingly. We evaluate our proposed approach on 2-photon microscopy images containing dendritic spines and compare its performance quantitatively to an existing approach based on nonparametric shape priors. Both visual and quantitative results demonstrate the effectiveness of our approach in dendritic spine segmentation.

Dendritic spine classification using shape and appearance features based on two-photon microscopy

BACKGROUND Neuronal morphology and function are highly coupled. In particular, dendritic spine morphology is strongly governed by the incoming neuronal activity. The first step towards understanding the structure-function relationships is to classify spine shapes into the main spine types suggested in the literature. Due to the lack of reliable automated analysis tools, classification is mostly performed manually, which is a time-intensive task and prone to subjectivity. NEW METHOD We propose an automated method to classify dendritic spines using shape and appearance features based on challenging two-photon laser scanning microscopy (2PLSM) data. Disjunctive Normal Shape Models (DNSM) is a recently proposed parametric shape representation. We perform segmentation of spine images by applying DNSM and use the resulting representation as shape features. Furthermore, we use Histogram of oriented gradients (HOG) to extract appearance features. In this context, we propose a kernel density estimation (KDE) based framework for dendritic spine classification, which uses these shape and appearance features. RESULTS Our shape and appearance features based approach combined with Neural Network (NN) correctly classifies 87.06% of spines on a dataset of 456 spines. COMPARISON WITH EXISTING METHODS Our proposed method outperforms standard morphological feature based approaches. Our KDE based framework also enables neuroscientists to analyze the separability of spine shape classes in the likelihood ratio space, which leads to further insights about nature of the spine shape analysis problem. CONCLUSIONS Results validate that performance of our proposed approach is comparable to a human expert. It also enable neuroscientists to study shape statistics in the likelihood ratio space.

Dendritic spine shape analysis using disjunctive normal shape models

Analysis of dendritic spines is an essential task to understand the functional behavior of neurons. Their shape variations are known to be closely linked with neuronal activities. Spine shape analysis in particular, can assist neuroscientists to identify this relationship. A novel shape representation has been proposed recently, called Disjunctive Normal Shape Models (DNSM). DNSM is a parametric shape representation and has proven to be successful in several segmentation problems. In this paper, we apply this parametric shape representation as a feature extraction algorithm. Further, we propose a kernel density estimation (KDE) based classification approach for dendritic spine classification. We evaluate our proposed approach on a data set of 242 spines, and observe that it outperforms the classical morphological feature based approach for spine classification. Our probabilistic framework also provides a way to examine the separability of spine shape classes in the likelihood ratio space, which leads to further insights about the nature of the shape analysis problem in this context.

A tool for automatic dendritic spine detection and analysis. Part I: Dendritic spine detection using multi-level region-based segmentation

We propose an image processing pipeline for dendritic spine detection in two-photon fluorescence microscopy images. Spines of interest to neuroscientists often contain high intensity regions with respect to their surroundings. We find such maxima regions using morphological image reconstruction. These regions facilitate a multi-level segmentation algorithm to detect spines. First, watershed algorithm is applied to extract initial rough regions of spines. Then, these results are further refined using a graph-theoretic region-growing algorithm which incorporates segmentation on a sparse representation of image data and hierarchical clustering as a post-processing step. We compare our final results to segmentation results of the domain expert. Our pipeline produces promising segmentation results with practical run times for monitoring streaming data.

Nonparametric joint shape and feature priors for segmentation of dendritic spines

Multimodal shape density estimation is a challenging task in many biomedical image segmentation problems. Existing techniques in the literature estimate the underlying shape distribution by extending Parzen density estimator to the space of shapes. Such density estimates are only expressed in terms of distances between shapes which may not be sufficient for ensuring accurate segmentation when the observed intensities provide very little information about the object boundaries. In such scenarios, employing additional shape-dependent discriminative features as priors and exploiting both shape and feature priors can aid to the segmentation process. In this paper, we propose a segmentation algorithm that uses nonparametric joint shape and feature priors using Parzen density estimator. The joint prior density estimate is expressed in terms of distances between shapes and distances between features. We incorporate the learned joint shape and feature prior distribution into a maximum a posteriori estimation framework for segmentation. The resulting optimization problem is solved using active contours. We present experimental results on dendritic spine segmentation in 2-photon microscopy images which involve a multimodal shape density.

On comparison of manifold learning techniques for dendritic spine classification

Dendritic spines are one of the key functional components of neurons. Their morphological changes are correlated with neuronal activity. Neuroscientists study spine shape variations to understand their relation with neuronal activity. Currently this analysis performed manually, the availability of reliable automated tools would assist neuroscientists and accelerate this research. Previously, morphological features based spine analysis has been performed and reported in the literature. In this paper, we explore the idea of using and comparing manifold learning techniques for classifying spine shapes. We start with automatically segmented data and construct our feature vector by stacking and concatenating the columns of images. Further, we apply unsupervised manifold learning algorithms and compare their performance in the context of dendritic spine classification. We achieved 85.95% accuracy on a dataset of 242 automatically segmented mushroom and stubby spines. We also observed that ISOMAP implicitly computes prominent features suitable for classification purposes.

Dendritic Spine Shape Analysis: A Clustering Perspective

Functional properties of neurons are strongly coupled with their morphology. Changes in neuronal activity alter morphological characteristics of dendritic spines. First step towards understanding the structure-function relationship is to group spines into main spine classes reported in the literature. Shape analysis of dendritic spines can help neuroscientists understand the underlying relationships. Due to unavailability of reliable automated tools, this analysis is currently performed manually which is a time-intensive and subjective task. Several studies on spine shape classification have been reported in the literature, however, there is an on-going debate on whether distinct spine shape classes exist or whether spines should be modeled through a continuum of shape variations. Another challenge is the subjectivity and bias that is introduced due to the supervised nature of classification approaches. In this paper, we aim to address these issues by presenting a clustering perspective. In this context, clustering may serve both confirmation of known patterns and discovery of new ones. We perform cluster analysis on two-photon microscopic images of spines using morphological, shape, and appearance based features and gain insights into the spine shape analysis problem. We use histogram of oriented gradients (HOG), disjunctive normal shape models (DNSM), morphological features, and intensity profile based features for cluster analysis. We use x-means to perform cluster analysis that selects the number of clusters automatically using the Bayesian information criterion (BIC). For all features, this analysis produces 4 clusters and we observe the formation of at least one cluster consisting of spines which are difficult to be assigned to a known class. This observation supports the argument of intermediate shape types.

Nonparametric Joint Shape and Feature Priors for Image Segmentation

In many image segmentation problems involving limited and low-quality data, employing statistical prior information about the shapes of the objects to be segmented can significantly improve the segmentation result. However, defining probability densities in the space of shapes is an open and challenging problem, especially if the object to be segmented comes from a shape density involving multiple modes (classes). Existing techniques in the literature estimate the underlying shape distribution by extending Parzen density estimator to the space of shapes. In these methods, the evolving curve may converge to a shape from a wrong mode of the posterior density when the observed intensities provide very little information about the object boundaries. In such scenarios, employing both shape- and class-dependent discriminative feature priors can aid the segmentation process. Such features may involve, e.g., intensity-based, textural, or geometric information about the objects to be segmented. In this paper, we propose a segmentation algorithm that uses nonparametric joint shape and feature priors constructed by Parzen density estimation. We incorporate the learned joint shape and feature prior distribution into a maximum a posteriori estimation framework for segmentation. The resulting optimization problem is solved using active contours. We present experimental results on a variety of synthetic and real data sets from several fields involving multimodal shape densities. Experimental results demonstrate the potential of the proposed method.

Retrieval From and Understanding of Large-Scale Multi-modal Medical Datasets: A Review

Content-based multimedia retrieval (CBMR) has been an active research domain since the mid 1990s. In medicine visual retrieval started later and has mostly remained a research instrument and less a clinical tool. The limited size of data sets due to privacy constraints is often mentioned as reason for these limitations. Nevertheless, much work has been done in CBMR, including the availability of increasingly large data sets and scientific challenges. Annotated data sets and clinical data for images have now become available and can be combined for multi-modal retrieval. Much has been learned on user behavior and application scenarios. This text is motivated by the advances in medical image analysis and the availability of public large data sets that often include clinical data. It is a systematic review of recent work (concentrating on the period 2011–2017) on multi-modal CBMR and image understanding in the medical domain, where image understanding includes techniques such as detection, localization, and classification for leveraging visual content. With the objective of summarizing the current state of research for multimedia researchers outside the medical field, the text provides ways to get data sets and identifies current limitations and promising research directions. The text highlights advances in the past six years and a trend to use larger scale training data and deep learning approaches that can replace/complement hand-crafted features. Using images alone will likely only work in limited domains but combining multiple sources of data for multi-modal retrieval has the biggest chances of success, particularly for clinical impact.