Zhennan Yan

Multi-Instance Deep Learning: Discover Discriminative Local Anatomies for Bodypart Recognition

In general image recognition problems, discriminative information often lies in local image patches. For example, most human identity information exists in the image patches containing human faces. The same situation stays in medical images as well. “Bodypart identity” of a transversal slice-which bodypart the slice comes from-is often indicated by local image information, e.g., a cardiac slice and an aorta arch slice are only differentiated by the mediastinum region. In this work, we design a multi-stage deep learning framework for image classification and apply it on bodypart recognition. Specifically, the proposed framework aims at: 1) discover the local regions that are discriminative and non-informative to the image classification problem, and 2) learn a image-level classifier based on these local regions. We achieve these two tasks by the two stages of learning scheme, respectively. In the pre-train stage, a convolutional neural network (CNN) is learned in a multi-instance learning fashion to extract the most discriminative and and non-informative local patches from the training slices. In the boosting stage, the pre-learned CNN is further boosted by these local patches for image classification. The CNN learned by exploiting the discriminative local appearances becomes more accurate than those learned from global image context. The key hallmark of our method is that it automatically discovers the discriminative and non-informative local patches through multi-instance deep learning. Thus, no manual annotation is required. Our method is validated on a synthetic dataset and a large scale CT dataset. It achieves better performances than state-of-the-art approaches, including the standard deep CNN.

Is Interactional Dissynchrony a Clue to Deception? Insights From Automated Analysis of Nonverbal Visual Cues

Detecting deception in interpersonal dialog is challenging since deceivers take advantage of the give-and-take of interaction to adapt to any sign of skepticism in an interlocutors verbal and nonverbal feedback. Human detection accuracy is poor, often with no better than chance performance. In this investigation, we consider whether automated methods can produce better results and if emphasizing the possible disruption in interactional synchrony can signal whether an interactant is truthful or deceptive. We propose a data-driven and unobtrusive framework using visual cues that consists of face tracking, head movement detection, facial expression recognition, and interactional synchrony estimation. Analysis were conducted on 242 video samples from an experiment in which deceivers and truth-tellers interacted with professional interviewers either face-to-face or through computer mediation. Results revealed that the framework is able to automatically track head movements and expressions of both interlocutors to extract normalized meaningful synchrony features and to learn classification models for deception recognition. Further experiments show that these features reliably capture interactional synchrony and efficiently discriminate deception from truth.

Accurate segmentation of brain images into 34 structures combining a non-stationary adaptive statistical atlas and a multi-atlas with applications to Alzheimer'S disease

Accurate segmentation of the 30+ subcortical structures in MR images of whole diseased brains is challenging due to inter-subject variability and complex geometry of brain anatomy. However a clinically viable solution yielding precise segmentation of the structures would enable: 1) accurate, objective measurement of structure volumes many of which are associated with diseases such as Alzheimers, 2) therapy monitoring and 3) drug development. Our contributions are two-fold. First we construct an extended adaptive statistical atlas method (EASA) to use a non-stationary relaxation factor rather than a global one. This permits finer control over adaptivity allowing 34 structures to be simultaneously segmented rather than just 4 as in [13]. Second we use the output of a weighted majority voting (WMV) label fusion multi-atlas method as the input to EASA in a hybrid WMV-EASA approach. We assess our proposed approaches on 18 healthy subjects in the public IBSR database and on 9 subjects with Alzheimers disease in the AIBL database. EASA is shown to produce state-of-the-art accuracy on healthy brains in a fraction of the time of comparable methods, while our hybrid WMV-EASA visibly improves segmentation accuracy for structures throughout the diseased brains.

Automated anatomical landmark detection ondistal femur surface using convolutional neural network

Accurate localization of the anatomical landmarks on distal femur bone in the 3D medical images is very important for knee surgery planning and biomechanics analysis. However, the landmark identification process is often conducted manually or by using the inserted auxiliaries, which is time-consuming and lacks of accuracy. In this paper, an automatic localization method is proposed to determine positions of initial geometric landmarks on femur surface in the 3D MR images. Based on the results from the convolutional neural network (CNN) classifiers and shape statistics, we use the narrow-band graph cut optimization to achieve the 3D segmentation of femur surface. Finally, the anatomical landmarks are located on the femur according to the geometric cues of surface mesh. Experiments demonstrate that the proposed method is effective, efficient, and reliable to segment femur and locate the anatomical landmarks.

Atlas-based liver segmentation and hepatic fat-fraction assessment for clinical trials

Automated assessment of hepatic fat-fraction is clinically important. A robust and precise segmentation would enable accurate, objective and consistent measurement of hepatic fat-fraction for disease quantification, therapy monitoring and drug development. However, segmenting the liver in clinical trials is a challenging task due to the variability of liver anatomy as well as the diverse sources the images were acquired from. In this paper, we propose an automated and robust framework for liver segmentation and assessment. It uses single statistical atlas registration to initialize a robust deformable model to obtain fine segmentation. Fat-fraction map is computed by using chemical shift based method in the delineated region of liver. This proposed method is validated on 14 abdominal magnetic resonance (MR) volumetric scans. The qualitative and quantitative comparisons show that our proposed method can achieve better segmentation accuracy with less variance comparing with two other atlas-based methods. Experimental results demonstrate the promises of our assessment framework.

Accurate thigh inter-muscular adipose quantification using a data-driven and sparsity-constrained deformable model

The thigh inter-muscular adipose tissue (IMAT) quantification plays a critical role in various medical analysis tasks, e.g., the analysis of physical performance or the diagnose of knee osteoarthritis. In recent years, several techniques have been proposed to perform automated thigh tissues quantification. However, nobody has provided effective methods to track fascia lata, which is an important anatomic trail to distinguish between subcutaneous adipose tissue (SAT) and I-MAT in thigh. As a result, the estimation of IMAT may not be accurate for subjects with pathological conditions. On the other hand, tissue prior information, e.g., intensity, orientation and scale, becomes critical to infer and refine the fascia lata boundary from image appearance cues. In this paper, we propose a novel data-driven and sparsity-constrained de-formable model to obtain accurate fascia lata labeling. The model deformation is driven by the target points on fascia lata detected by a local discriminative classifier in a narrowband fashion. By using a sparsity-constrained optimization, the deformation is solved with errors and outliers suppression. The proposed approach has been evaluated on a set of 3D MR thigh volumes. In a comparison with another state-of-art framework, our approach produces superior performance.

Modulus reconstruction from prostate ultrasound images using finite element modeling

In medical diagnosis, use of elastography is becoming increasingly more useful. However, treatments usually assume a planar compression applied to tissue surfaces and measure the deformation. The stress distribution is relatively uniform close to the surface when using a large, flat compressor but it diverges gradually along tissue depth. Generally in prostate elastography, the transrectal probes used for scanning and compression are cylindrical side-fire or rounded end-fire probes, and the force is applied through the rectal wall. These make it very difficult to detect cancer in prostate, since the rounded contact surfaces exaggerate the non-uniformity of the applied stress, especially for the distal, anterior prostate. We have developed a preliminary 2D Finite Element Model (FEM) to simulate prostate deformation in elastography. The model includes a homogeneous prostate with a stiffer tumor in the proximal, posterior region of the gland. A force is applied to the rectal wall to deform the prostate, strain and stress distributions can be computed from the resultant displacements. Then, we assume the displacements as boundary condition and reconstruct the modulus distribution (inverse problem) using linear perturbation method. FEM simulation shows that strain and strain contrast (of the lesion) decrease very rapidly with increasing depth and lateral distance. Therefore, lesions would not be clearly visible if located far away from the probe. However, the reconstructed modulus image can better depict relatively stiff lesion wherever the lesion is located.

An Automated and Robust Framework for Quantification of Muscle and Fat in the Thigh

The tissue quantification in the thigh (e.g. cross-sectional areas of adipose tissue and muscle) is important, since their quantities reflect adverse metabolic effects and muscle function. Traditional manual analysis is time-consuming and operator-dependent, especially in the case of multi-slices or 3D datasets. In clinical trials, there are a large amount of datasets acquired from magnetic resonance imaging (MRI) or X-ray computed tomography (CT) that requires automatic labeling of individual tissues. Since most segmentation algorithms are not suited for different modalities, we present an automatic and robust framework for the quantitative assessment of muscle and fat tissues on 3D MR or CT data. In our framework, a variational Bayesian Gaussian mixture model is used to cluster regions of interest in images into adipose tissues (fat and marrow), muscle, bone and background. The identification of each cluster is based on marrow detection. Furthermore, we use a combination of parametric and geodesic active contour models to distinguish different adipose tissues in 3D images. To validate our proposed framework, we have conducted preliminary experiments on five volumetric mid-thigh axial datasets of MR and CT images from clinical trials.

Accurate Whole-Brain Segmentation for Alzheimer's Disease Combining an Adaptive Statistical Atlas and Multi-atlas

Accurate segmentation of whole brain MR images including the cortex, white matter and subcortical structures is challenging due to inter-subject variability and the complex geometry of brain anatomy. However a precise solution would enable accurate, objective measurement of structure volumes for disease quantification. Our contribution is three-fold. First we construct an adaptive statistical atlas that combines structure specific relaxation and spatially varying adaptivity. Second we integrate an isotropic pairwise class-specific MRF model of label connectivity. Together these permit precise control over adaptivity, allowing many structures to be segmented simultaneously with superior accuracy. Third, we develop a framework combining the improved adaptive statistical atlas with a multi-atlas method which achieves simultaneous accurate segmentation of the cortex, ventricles, and sub-cortical structures in severely diseased brains, a feat not attained in [18]. We test the proposed method on 46 brains including 28 diseased brain with Alzheimers and 18 healthy brains. Our proposed method yields higher accuracy than state-of-the-art approaches on both healthy and diseased brains.

Automated compromised right lung segmentation method using a robust atlas-based active volume model with sparse shape composition prior in CT

To resolve challenges in image segmentation in oncologic patients with severely compromised lung, we propose an automated right lung segmentation framework that uses a robust, atlas-based active volume model with a sparse shape composition prior. The robust atlas is achieved by combining the atlas with the output of sparse shape composition. Thoracic computed tomography images (n=38) from patients with lung tumors were collected. The right lung in each scan was manually segmented to build a reference training dataset against which the performance of the automated segmentation method was assessed. The quantitative results of this proposed segmentation method with sparse shape composition achieved mean Dice similarity coefficient (DSC) of (0.72, 0.81) with 95% CI, mean accuracy (ACC) of (0.97, 0.98) with 95% CI, and mean relative error (RE) of (0.46, 0.74) with 95% CI. Both qualitative and quantitative comparisons suggest that this proposed method can achieve better segmentation accuracy with less variance than other atlas-based segmentation methods in the compromised lung segmentation.