Anne Jorstad

NeuroMorph: A Toolset for the Morphometric Analysis and Visualization of 3D Models Derived from Electron Microscopy Image Stacks

Serialelectron microscopy imaging is crucial for exploring the structure of cells and tissues. The development of block face scanning electron microscopy methods and their ability to capture large image stacks, some with near isotropic voxels, is proving particularly useful for the exploration of brain tissue. This has led to the creation of numerous algorithms and software for segmenting out different features from the image stacks. However, there are few tools available to view these results and make detailed morphometric analyses on all, or part, of these 3D models. We have addressed this issue by constructing a collection of software tools, called NeuroMorph, with which users can view the segmentation results, in conjunction with the original image stack, manipulate these objects in 3D, and make measurements of any region. This approach to collecting morphometric data provides a faster means of analysing the geometry of structures, such as dendritic spines and axonal boutons. This bridges the gap that currently exists between rapid reconstruction techniques, offered by computer vision research, and the need to collect measurements of shape and form from segmented structures that is currently done using manual segmentation methods.

Correlative In Vivo 2-Photon Imaging and Focused Ion Beam Scanning Electron Microscopy: 3D Analysis of Neuronal Ultrastructure

This protocol describes how dendrites and axons, imaged in vivo, can subsequently be analyzed in 3D using focused ion beam scanning electron microscopy (FIBSEM). The fluorescent structures are identified after chemical fixation and their position highlighted using the 2-photon laser to burn fiducial marks around the region. Once the section has been stained and resin embedded, a small block is trimmed close to these marks. Serially aligned EM images are acquired through this region, using FIBSEM, and the neurites of interest then reconstructed semi-automatically using the Ilastik software (ilastik.org). This fast and reliable imaging and reconstruction technique avoids the use of specific labels to identify the features of interest in the electron microscope and optimizes their preservation for high-quality imaging and 3D analysis.

A deformation and lighting insensitive metric for face recognition based on dense correspondences

Face recognition is a challenging problem, complicated by variations in pose, expression, lighting, and the passage of time. Significant work has been done to solve each of these problems separately. We consider the problems of lighting and expression variation together, proposing a method that accounts for both variabilities within a single model. We present a novel deformation and lighting insensitive metric to compare images, and we present a novel framework to optimize over this metric to calculate dense correspondences between images. Typical correspondence cost patterns are learned between face image pairs and a Naïve Bayes classifier is applied to improve recognition accuracy. Very promising results are presented on the AR Face Database, and we note that our method can be extended to a broad set of applications.

Dense Image Registration and Deformable Surface Reconstruction in Presence of Occlusions and Minimal Texture

Deformable surface tracking from monocular images is well-known to be under-constrained. Occlusions often make the task even more challenging, and can result in failure if the surface is not sufficiently textured. In this work, we explicitly address the problem of 3D reconstruction of poorly textured, occluded surfaces, proposing a framework based on a template-matching approach that scales dense robust features by a relevancy score. Our approach is extensively compared to current methods employing both local feature matching and dense template alignment. We test on standard datasets as well as on a new dataset (that will be made publicly available) of a sparsely textured, occluded surface. Our framework achieves state-of-the-art results for both well and poorly textured, occluded surfaces.

Distributed Consensus on Camera Pose

Our work addresses pose estimation in a distributed camera framework. We examine how processing cameras can best reach a consensus about the pose of an object when they are each given a model of the object, defined by a set of point coordinates in the object frame of reference. The cameras can only see a subset of the object feature points in the midst of background clutter points, not knowing which image points match with which object points, nor which points are object points or background points. The cameras individually recover a prediction of the objects pose using their knowledge of the model, and then exchange information with their neighbors, performing consensus updates locally to obtain a single estimate consistent across all cameras, without requiring a common centralized processor. Our main contributions are: 1) we present a novel algorithm performing consensus updates in 3-D world coordinates penalized by a 3-D model, and 2) we perform a thorough comparison of our method with other current consensus methods. Our method is consistently the most accurate, and we confirm that the existing consensus method based upon calculating the Karcher mean of rotations is also reliable and fast. Experiments on simulated and real imagery are reported.

Refining Mitochondria Segmentation in Electron Microscopy Imagery with Active Surfaces

We present an active surface-based method for refining the boundary surfaces of mitochondria segmentation data. We exploit thefact that mitochondria have thick dark membranes, so referencing the image data at the inner membrane can help drive a more accurate delineation of the outer membrane surface. Given the initial boundary prediction from a machine learning-based segmentation algorithm as input, we compare several cost functions used to drive an explicit update scheme to locally refine 3D mesh surfaces, and results are presented on electron microscopy imagery. Our resulting surfaces are seen to fit very accurately to the mitochondria membranes, more accurately even than the available hand-annotations of the data.

Model-based pose estimation by consensus

We present a system for determining a consensus estimate of the pose of an object, as seen from multiple cameras in a distributed network. The cameras are pointed towards a 3D object defined by a configuration of points, which are assumed to be visible and detected in all camera images. The cameras are given a model defining the 3D configuration of these object points, but do not know which image point corresponds to which object point. Each camera estimates the pose of the object, then iteratively exchanges information with its neighbors to arrive at a common decision of the pose over the network. We consider eight variations of the consensus algorithm, and find that each converges to a more accurate result than do the individual cameras alone on average. The method exchanging 3D world coordinates penalized to agree with the input model provides the most accurate results. If bandwidth is limited, performing consensus over rotations and translations requires cameras to exchange only the six values specifying the six degrees of freedom of the object pose, and performing consensus in SE(3) using the Karcher mean is generally the best choice. We show further that interleaving pose calculation with the consensus iterations improves the final result when the image noise is large.

A fast illumination and deformation insensitive image comparison algorithm using wavelet-based geodesics

We present a fast image comparison algorithm for handling variations in illumination and moderate amounts of deformation using an efficient geodesic framework. As the geodesic is the shortest path between two images on a manifold, it is a natural choice to use the length of the geodesic to determine the image similarity. Distances on the manifold are defined by a metric that is insensitive to changes in scene lighting. This metric is described in the wavelet domain where it is able to handle moderate amounts of deformation, and can be calculated extremely fast (less than 3ms per image comparison). We demonstrate the similarity between our method and the illumination insensitivity achieved by the Gradient Direction. Strong results are presented on the AR Face Database.

The effects of aging on neuropil structure in mouse somatosensory cortex—A 3D electron microscopy analysis of layer 1

This study has used dense reconstructions from serial EM images to compare the neuropil ultrastructure and connectivity of aged and adult mice. The analysis used models of axons, dendrites, and their synaptic connections, reconstructed from volumes of neuropil imaged in layer 1 of the somatosensory cortex. This shows the changes to neuropil structure that accompany a general loss of synapses in a well-defined brain region. The loss of excitatory synapses was balanced by an increase in their size such that the total amount of synaptic surface, per unit length of axon, and per unit volume of neuropil, stayed the same. There was also a greater reduction of inhibitory synapses than excitatory, particularly those found on dendritic spines, resulting in an increase in the excitatory/inhibitory balance. The close correlations, that exist in young and adult neurons, between spine volume, bouton volume, synaptic size, and docked vesicle numbers are all preserved during aging. These comparisons display features that indicate a reduced plasticity of cortical circuits, with fewer, more transient, connections, but nevertheless an enhancement of the remaining connectivity that compensates for a generalized synapse loss.

Computer assisted detection of axonal bouton structural plasticity in in vivo time-lapse images

The ability to measure minute structural changes in neural circuits is essential for long-term in vivo imaging studies. Here, we propose a methodology for detection and measurement of structural changes in axonal boutons imaged with time-lapse two-photon laser scanning microscopy (2PLSM). Correlative 2PLSM and 3D electron microscopy (EM) analysis, performed in mouse barrel cortex, showed that the proposed method has low fractions of false positive/negative bouton detections (2/0 out of 18), and that 2PLSM-based bouton weights are correlated with their volumes measured in EM (r = 0.93). Next, the method was applied to a set of axons imaged in quick succession to characterize measurement uncertainty. The results were used to construct a statistical model in which bouton addition, elimination, and size changes are described probabilistically, rather than being treated as deterministic events. Finally, we demonstrate that the model can be used to quantify significant structural changes in boutons in long-term imaging experiments.