Qiwei Xie

Wrinkle Image Registration for Serial Microscopy Sections

3D Reconstruction from the microscopy images of serial sections plays an important role in analysis structure of biological specimens, such as neuronal circuits in brain tissue. During specimen sectioning and collecting, it is very hard to prevent wrinkle from these sections, which introduce distortion when imaging in electron microscopy and cause failure of structure reconstruction. In this paper, we propose a pipeline for registration of serial sections with wrinkle. First, Scale Invariant Feature Transform (SIFT) is used to detect corresponding landmarks across adjacent sections. Second, wrinkle areas are labeled manually in microscopy images, which is easy to distinguish by eye. Finally, a modified Moving-Least-Square deformation algorithm is applied to register adjacent sections with wrinkle. The algorithm reflects the discontinuity around wrinkle areas while keeps the smoothness in other regions. Experimental results demonstrate the effectiveness of our method.

An automated pipeline for mitochondrial segmentation on ATUM-SEM stacks

It is possible now to look more closely into mitochondrial physical structures due to the rapid development of electron microscope (EM). Mitochondrial physical structures play important roles in both cellular physiology and neuronal functions. Unfortunately, the segmentation of mitochondria from EM images has proven to be a difficult and challenging task, due to the presence of various subcellular structures, as well as image distortions in the sophisticated background. Although the current state-of-the-art algorithms have achieved some promising results, they have demonstrated poor performances on these mitochondria which are in close proximity to vesicles or various membranes. In order to overcome these limitations, this study proposes explicitly modelling the mitochondrial double membrane structures, and acquiring the image edges by way of ridge detection rather than by image gradient. In addition, this study also utilizes group-similarity in context to further optimize the local misleading segmentation. Then, the experimental results determined from the images acquired by automated tape-collecting ultramicrotome scanning electron microscopy (ATUM-SEM) demonstrate the effectiveness of this studys proposed algorithm.

Deep learning and shapes similarity for joint segmentation and tracing single neurons in SEM images

Extracting the structure of single neurons is critical for understanding how they function within the neural circuits. Recent developments in microscopy techniques, and the widely recognized need for openness and standardization provide a community resource for automated reconstruction of dendritic and axonal morphology of single neurons. In order to look into the fine structure of neurons, we use the Automated Tape-collecting Ultra Microtome Scanning Electron Microscopy (ATUM-SEM) to get images sequence of serial sections of animal brain tissue that densely packed with neurons. Different from other neuron reconstruction method, we propose a method that enhances the SEM images by detecting the neuronal membranes with deep convolutional neural network (DCNN) and segments single neurons by active contour with group shape similarity. We joint the segmentation and tracing together and they interact with each other by alternate iteration that tracing aids the selection of candidate region patch for active contour segmentation while the segmentation provides the neuron geometrical features which improve the robustness of tracing. The tracing model mainly relies on the neuron geometrical features and is updated after neuron being segmented on the every next section. Our method enables the reconstruction of neurons of the drosophila mushroom body which is cut to serial sections and imaged under SEM. Our method provides an elementary step for the whole reconstruction of neuronal networks.

Automatically Segmenting and Reconstructing Neurons in SEM images

Neuronal networks reconstruction is a big challenge in the neuroscience. Recent developments in volume electron microscopy (EM) imaging have enabled us to obtain large amounts of brain tissues image data. Analysis of the tremendously huge neuronal EM images based on automated method would be of vital importance. In this paper, we propose a method that Deep Convolutional Neural Network (DCNN) is used for neuronal boundary detection; and then, with the membrane detection probability map (MDPM) generated by DCNN, a marker-controlled watershed method is applied to segment neurons in the EM images over the MDPM. Semi-automated and fully automated 3D reconstruction methods are employed to connect the sections of the corresponding segmentations belonging to each neuron. Finally, we have reconstructed dense neurons in 8000×8000×1796 consecutive EM images stack of drosophila mushroom body with resolution of 4nm×4nm×50nm with automated method and several sparse neurons with semi-automated method.

3D-reconstruction of synapses based on EM images

3D reconstruction from the microscopy images of serial sections plays an important role in analyzing structure of biological specimens, such as neuronal circuits in brain tissue. This paper is focusing on the 3D reconstruction of synapses which is a branch of micro reconstruction of brain. Having analysed the structure of synapses, we first detect and locate them in serial sections with cascade AdaBoost algorithm, then optimize the shape of synapses by constructing suitable fitting functions and segment them with morphology processing method. Taking the connection of adjacent sections and synaptic extensibility in space into consideration, we complete the reconstruction of synapses which is based on the segmentation in serial sections. Actually all our works are based on the registration of serial sections.

Correction of SEM image distortion based on non-equidistant GM(1,1) model

Scanning Electron Microscope (SEM) is widely used in many fields to look into the nanometer world. However, there is non-linear distortion near the left boundary of high resolution images captured by SEM, which has bad effect on image processing such as image mosaic. In this paper, we propose a method to correct such distortion based on non-equidistant GM(1,1) model. We first extract pair-wise points which are the same position on target scene from two overlapped images respectively. Then the distortion is modeled using non-equidistant GM(1,1) which maps pixels of distorted image to their true positions. Finally, the corrected image is obtained with image interpolation. There is no need to pre-define the distortion model or parameters for the proposed method, and it is easy to use and flexible. Experiments show that the proposed method yields accurate correction results.

3D reconstruction of synapses with deep learning based on EM Images

Recently, due to the rapid development of electron microscope (EM) with its high resolution, stacks delivered by EM can be used to analyze a variety of components that are critical to understand brain function. Since synaptic study is essential in neurobiology and can be analyzed by EM stacks, the automated routines for reconstruction of synapses based on EM Images can become a very useful tool for analyzing large volumes of brain tissue and providing the ability to understand the mechanism of brain. In this article, we propose a novel automated method to realize 3D reconstruction of synapses for Automated Tapecollecting Ultra Microtome Scanning Electron Microscopy (ATUM-SEM) with deep learning. Being different from other reconstruction algorithms, which employ classifier to segment synaptic clefts directly. We utilize deep learning method and segmentation algorithm to obtain synaptic clefts as well as promote the accuracy of reconstruction. The proposed method contains five parts: (1) using modified Moving Least Square (MLS) deformation algorithm and Scale Invariant Feature Transform (SIFT) features to register adjacent sections, (2) adopting Faster Region Convolutional Neural Networks (Faster R-CNN) algorithm to detect synapses, (3) utilizing screening method which takes context cues of synapses into consideration to reduce the false positive rate, (4) combining a practical morphology algorithm with a suitable fitting function to segment synaptic clefts and optimize the shape of them, (5) applying the plugin in FIJI to show the final 3D visualization of synapses. Experimental results on ATUM-SEM images demonstrate the effectiveness of our proposed method.

An automated detection for axonal boutons in vivo two-photon imaging of mouse

Activity-dependent changes in the synaptic connections of the brain are tightly related to learning and memory. Previous studies have shown that essentially all new synaptic contacts were made by adding new partners to existing synaptic elements. To further explore synaptic dynamics in specific pathways, concurrent imaging of pre and postsynaptic structures in identified connections is required. Consequently, considerable attention has been paid for the automated detection of axonal boutons. Different from most previous methods proposed in vitro data, this paper considers a more practical case in vivo neuron images which can provide real time information and direct observation of the dynamics of a disease process in mouse. Additionally, we present an automated approach for detecting axonal boutons by starting with deconvolving the original images, then thresholding the enhanced images, and reserving the regions fulfilling a series of criteria. Experimental result in vivo two-photon imaging of mouse demonstrates the effectiveness of our proposed method.

Fully Automatic Synaptic Cleft Detection and Segmentation from EM Images Based on Deep Learning

The synapse, which is the carrier of neurotransmitter molecules to transmit and store information, is believed to be the key to the reconstruction of the neural circuit. To date, electron microscope (EM) is considered as one of the most important tools for observing and analyzing synaptic structures because they can clearly observe the internal structure of cells. Consequently, many meaningful researches are focused on how to detect and segment the synapses from EM images. In this paper, we propose a novel and effective method to automatically detect and segment the synaptic clefts by using Mask R-CNN. On this base, we utilize the context cues in adjacent sections to eliminate the misleading results. We apply the method to the CREMI challenge and the results demonstrate that our method is effective in segmenting the synaptic clefts of the drosophila. Specifically, we rank first in sample B+ dataset, and the CREMI score is 86.50 which outperforms most of state-of-the-art methods by a large margin.

An effective fully deep convolutional neural network for mitochondria segmentation based on ATUM-SEM.

Recent studies have empowered that the relation between mitochondrial function and degenerative disorder- s is related to aging diseases. Due to the rapid development of electron microscope (EM), stacks delivered by EM can be used to investigate the link between mitochondrial function and physical structure. Whereas, one of the main challenges in mitochondria research is developing suitable segmentation algorithms to obtain the shapes of mitochondria. Nowadays, Deep Neural Network (DNN) has been widely applied in solving the neuron membrane segmentation problems in virtue of its exceptional performance. For this reason, its appli- cation to mitochondria segmentation holds great promise. In this paper, we propose an effective deep learning approach to realize mitochondria segmentation in Automated Tape-Collecting Ultra Microtome Scanning Elec- tron Microscopy (ATUM-SEM) stacks. The proposed algorithm contains three parts: (1) utilizing histogram equalization algorithm as image preprocessing to keep the consistency of dataset; (2) putting forward a fusion fully convolution network (FCN), which is motivated by the principle the deeper, the better, to build a much deeper network architecture for more accurate mitochondria segmentation; and (3) employing fully connected conditional random field (CRF) to optimize segmentation results. Evaluation was performed on a dataset of a stack of 31 slices from ATUM-SEM, with 20 images used for training and 11 images for testing. For comparison, U-Net approach was evaluated through the same dataset. Jaccard index between the automated segmentation and expert manual segmentations indicates that our method (90.1%) outperforms U-Net (87.9%) and has a preferable performance on mitochondria segmentation with different shapes and sizes.