Ullrich Koethe

Automated Detection and Segmentation of Synaptic Contacts in Nearly Isotropic Serial Electron Microscopy Images

We describe a protocol for fully automated detection and segmentation of asymmetric, presumed excitatory, synapses in serial electron microscopy images of the adult mammalian cerebral cortex, taken with the focused ion beam, scanning electron microscope (FIB/SEM). The procedure is based on interactive machine learning and only requires a few labeled synapses for training. The statistical learning is performed on geometrical features of 3D neighborhoods of each voxel and can fully exploit the high z-resolution of the data. On a quantitative validation dataset of 111 synapses in 409 images of 1948×1342 pixels with manual annotations by three independent experts the error rate of the algorithm was found to be comparable to that of the experts (0.92 recall at 0.89 precision). Our software offers a convenient interface for labeling the training data and the possibility to visualize and proofread the results in 3D. The source code, the test dataset and the ground truth annotation are freely available on the website http://www.ilastik.org/synapse-detection.

Globally optimal closed-surface segmentation for connectomics

We address the problem of partitioning a volume image into a previously unknown number of segments, based on a likelihood of merging adjacent supervoxels. Towards this goal, we adapt a higher-order probabilistic graphical model that makes the duality between supervoxels and their joint faces explicit and ensures that merging decisions are consistent and surfaces of final segments are closed. First, we propose a practical cutting-plane approach to solve the MAP inference problem to global optimality despite its NP-hardness. Second, we apply this approach to challenging large-scale 3D segmentation problems for neural circuit reconstruction (Connectomics), demonstrating the advantage of this higher-order model over independent decisions and finite-order approximations.

3D segmentation of SBFSEM images of neuropil by a graphical model over supervoxel boundaries

The segmentation of large volume images of neuropil acquired by serial sectioning electron microscopy is an important step toward the 3D reconstruction of neural circuits. The only cue provided by the data at hand is boundaries between otherwise indistinguishable objects. This indistinguishability, combined with the boundaries becoming very thin or faint in places, makes the large body of work on region-based segmentation methods inapplicable. On the other hand, boundary-based methods that exploit purely local evidence do not reach the extremely high accuracy required by the application domain that cannot tolerate the global topological errors arising from false local decisions. As a consequence, we propose a supervoxel merging method that arrives at its decisions in a non-local fashion, by posing and approximately solving a joint combinatorial optimization problem over all faces between supervoxels. The use of supervoxels allows the extraction of expressive geometric features. These are used by the higher-order potentials in a graphical model that assimilate knowledge about the geometry of neural surfaces by automated training on a gold standard. The scope of this improvement is demonstrated on the benchmark dataset E1088 (Helmstaedter et al., 2011) of 7.5billionvoxels from the inner plexiform layer of rabbit retina. We provide C++ source code for annotation, geometry extraction, training and inference.

Graphical model for joint segmentation and tracking of multiple dividing cells

MOTIVATION To gain fundamental insight into the development of embryos, biologists seek to understand the fate of each and every embryonic cell. For the generation of cell tracks in embryogenesis, so-called tracking-by-assignment methods are flexible approaches. However, as every two-stage approach, they suffer from irrevocable errors propagated from the first stage to the second stage, here from segmentation to tracking. It is therefore desirable to model segmentation and tracking in a joint holistic assignment framework allowing the two stages to maximally benefit from each other. RESULTS We propose a probabilistic graphical model, which both automatically selects the best segments from a time series of oversegmented images/volumes and links them across time. This is realized by introducing intra-frame and inter-frame constraints between conflicting segmentation and tracking hypotheses while at the same time allowing for cell division. We show the efficiency of our algorithm on a challenging 3D+t cell tracking dataset from Drosophila embryogenesis and on a 2D+t dataset of proliferating cells in a dense population with frequent overlaps. On the latter, we achieve results significantly better than state-of-the-art tracking methods. AVAILABILITY AND IMPLEMENTATION Source code and the 3D+t Drosophila dataset along with our manual annotations will be freely available on http://hci.iwr.uni-heidelberg.de/MIP/Research/tracking/

A discrete chain graph model for 3d+t cell tracking with high misdetection robustness

Tracking by assignment is well suited for tracking a varying number of divisible cells, but suffers from false positive detections. We reformulate tracking by assignment as a chain graph---a mixed directed-undirected probabilistic graphical model---and obtain a tracking simultaneously over all time steps from the maximum a-posteriori configuration. The model is evaluated on two challenging four-dimensional data sets from developmental biology. Compared to previous work, we obtain improved tracks due to an increased robustness against false positive detections and the incorporation of temporal domain knowledge.

Automated detection of synapses in serial section transmission electron microscopy image stacks.

We describe a method for fully automated detection of chemical synapses in serial electron microscopy images with highly anisotropic axial and lateral resolution, such as images taken on transmission electron microscopes. Our pipeline starts from classification of the pixels based on 3D pixel features, which is followed by segmentation with an Ising model MRF and another classification step, based on object-level features. Classifiers are learned on sparse user labels; a fully annotated data subvolume is not required for training. The algorithm was validated on a set of 238 synapses in 20 serial 7197×7351 pixel images (4.5×4.5×45 nm resolution) of mouse visual cortex, manually labeled by three independent human annotators and additionally re-verified by an expert neuroscientist. The error rate of the algorithm (12% false negative, 7% false positive detections) is better than state-of-the-art, even though, unlike the state-of-the-art method, our algorithm does not require a prior segmentation of the image volume into cells. The software is based on the ilastik learning and segmentation toolkit and the vigra image processing library and is freely available on our website, along with the test data and gold standard annotations (http://www.ilastik.org/synapse-detection/sstem).

Learning to segment dense cell nuclei with shape prior

We study the problem of segmenting multiple cell nuclei from GFP or Hoechst stained microscope images with a shape prior. This problem is encountered ubiquitously in cell biology and developmental biology. Our work is motivated by the observation that segmentations with loose boundary or shrinking bias not only jeopardize feature extraction for downstream tasks (e.g. cell tracking), but also prevent robust statistical analysis (e.g. modeling of fluorescence distribution). We therefore propose a novel extension to the graph cut framework that incorporates a “blob”-like shape prior. The corresponding energy terms are parameterized via structured learning. Extensive evaluation and comparison on 2D/3D datasets show substantial quantitative improvement over other state-of-the-art methods. For example, our method achieves an 8.2% Rand index increase and a 4.3 Hausdorff distance decrease over the second best method on a public hand-labeled 2D benchmark.

Multicut brings automated neurite segmentation closer to human performance

Reference EPFL-ARTICLE-226946doi:10.1038/nmeth.4151View record in Web of Science Record created on 2017-03-27, modified on 2017-07-13

DALSA: Domain Adaptation for Supervised Learning From Sparsely Annotated MR Images

We propose a new method that employs transfer learning techniques to effectively correct sampling selection errors introduced by sparse annotations during supervised learning for automated tumor segmentation. The practicality of current learning-based automated tissue classification approaches is severely impeded by their dependency on manually segmented training databases that need to be recreated for each scenario of application, site, or acquisition setup. The comprehensive annotation of reference datasets can be highly labor-intensive, complex, and error-prone. The proposed method derives high-quality classifiers for the different tissue classes from sparse and unambiguous annotations and employs domain adaptation techniques for effectively correcting sampling selection errors introduced by the sparse sampling. The new approach is validated on labeled, multi-modal MR images of 19 patients with malignant gliomas and by comparative analysis on the BraTS 2013 challenge data sets. Compared to training on fully labeled data, we reduced the time for labeling and training by a factor greater than 70 and 180 respectively without sacrificing accuracy. This dramatically eases the establishment and constant extension of large annotated databases in various scenarios and imaging setups and thus represents an important step towards practical applicability of learning-based approaches in tissue classification.

Automated segmentation of synapses in 3D EM data

This contribution presents a method for automatic detection of excitatory, asymmetric synapses and segmentation of synaptic junctional complexes in stacks of serial electron microscopy images with nearly isotropic resolution. The method uses a Random Forest classifier in the space of generic image features, computed directly in the 3D neighborhoods of each pixel, and an additional step of interactive probability maps thresholding. On the test dataset, the algorithm missed considerably less synapses than the human expert during the ground truth creation, while maintaining an equivalent false positive rate. The algorithm is implemented as an extension to the Interactive Learning and Segmentation Toolkit “ilastik” and is freely available on our website (www.ilastik.org/synapse-detection).