Qiyuan Tian

Automating the design of image processing pipelines for novel color filter arrays: local, linear, learned (L3) method

The high density of pixels in modern color sensors provides an opportunity to experiment with new color filter array (CFA) designs. A significant bottleneck in evaluating new designs is the need to create demosaicking, denoising and color transform algorithms tuned for the CFA. To address this issue, we developed a method(local, linear, learned or L3) for automatically creating an image processing pipeline. In this paper we describe the L3 algorithm and illustrate how we created a pipeline for a CFA organized as a 2×2 RGB/Wblock containing a clear (W) pixel. Under low light conditions, the L3 pipeline developed for the RGB/W CFA produces images that are superior to those from a matched Bayer RGB sensor. We also use L3 to learn pipelines for other RGB/W CFAs with different spatial layouts. The L3 algorithm shortens the development time for producing a high quality image pipeline for novel CFA designs.

Q-space truncation and sampling in diffusion spectrum imaging

To characterize the q‐space truncation and sampling on the spin‐displacement probability density function (PDF) in diffusion spectrum imaging (DSI).

Local histogram modification based contrast enhancement

This paper presents a novel contrast enhancement algorithm based on local histogram modification. A global method, which works by striking a balance between linear contrast enhancement and histogram equalization, is firstly introduced. We then segment images into rectangle regions and adaptively stretch contrast using our global enhancement algorithmin in each region to reproduce local contrast and details. In order to avoid boundary artifacts, we propose a bilateral weighting scheme. Our method is easy to use, and the experiment results show that the technique can produce good results on a variety of images.

Double diffusion encoding MRI for the clinic: DDE MRI for the Clinic

The purpose of this study is to develop double diffusion encoding (DDE) MRI methods for clinical use. Microscopic diffusion anisotropy measurements from DDE promise greater specificity to changes in tissue microstructure compared with conventional diffusion tensor imaging, but implementation of DDE sequences on whole‐body MRI scanners is challenging because of the limited gradient strengths and lengthy acquisition times.

The separate effects of lipids and proteins on brain MRI contrast revealed through tissue clearing

&NA; Despite the widespread use of magnetic resonance imaging (MRI) of the brain, the relative contribution of different biological components (e.g. lipids and proteins) to structural MRI contrasts (e.g., T1, T2, T2*, proton density, diffusion) remains incompletely understood. This limitation can undermine the interpretation of clinical MRI and hinder the development of new contrast mechanisms. Here, we determine the respective contribution of lipids and proteins to MRI contrast by removing lipids and preserving proteins in mouse brains using CLARITY. We monitor the temporal dynamics of tissue clearance via NMR spectroscopy, protein assays and optical emission spectroscopy. MRI of cleared brain tissue showed: 1) minimal contrast on standard MRI sequences; 2) increased relaxation times; and 3) diffusion rates close to free water. We conclude that lipids, present in myelin and membranes, are a dominant source of MRI contrast in brain tissue. HighlightsCharacterized lipid removal and protein retention of CLARITY tissue clearing method.Evidence that lipids are more significant contributor to MRI contrast than proteins.MRI of cleared tissue had increased relaxation times and increased diffusion rates.

Learning the Image Processing Pipeline

Many creative ideas are being proposed for image sensor designs, and these may be useful in applications ranging from consumer photography to computer vision. To understand and evaluate each new design, we must create a corresponding image processing pipeline that transforms the sensor data into a form, that is appropriate for the application. The need to design and optimize these pipelines is time-consuming and costly. We explain a method that combines machine learning and image systems simulation that automates the pipeline design. The approach is based on a new way of thinking of the image processing pipeline as a large collection of local linear filters. We illustrate how the method has been used to design pipelines for novel sensor architectures in consumer photography applications.

Efficient illuminant correction in the local, linear, learned (L3) method

To speed the development of novel camera architectures we proposed a method, L3 (Local, Linear and Learned),that automatically creates an optimized image processing pipeline. The L3 method assigns each sensor pixel into one of 400 classes, and applies class-dependent local linear transforms that map the sensor data from a pixel and its neighbors into the target output (e.g., CIE XYZ rendered under a D65 illuminant). The transforms are precomputed from training data and stored in a table used for image rendering. The training data are generated by camera simulation, consisting of sensor responses and rendered CIE XYZ outputs. The sensor and rendering illuminant can be equal (same-illuminant table) or different (cross-illuminant table). In the original implementation, illuminant correction is achieved with cross-illuminant tables, and one table is required for each illuminant. We find, however, that a single same-illuminant table (D65) effectively converts sensor data for many different same-illuminant conditions. Hence, we propose to render the data by applying the same-illuminant D65 table to the sensor data, followed by a linear illuminant correction transform. The mean color reproduction error using the same-illuminant table is on the order of 4▵E units, which is only slightly larger than the cross-illuminant table error. This approach reduces table storage requirements significantly without substantially degrading color reproduction accuracy.

Automatically designing an image processing pipeline for a five-band camera prototype using the local, linear, learned (L3) method

The development of an image processing pipeline for each new camera design can be time-consuming. To speed camera development, we developed a method named L3 (Local, Linear, Learned) that automatically creates an image processing pipeline for any design. In this paper, we describe how we used the L3 method to design and implement an image processing pipeline for a prototype camera with five color channels. The process includes calibrating and simulating the prototype, learning local linear transforms and accelerating the pipeline using graphics processing units (GPUs).

Segmentation based tone-mapping for high dynamic range images

In this paper, we present a novel segmentation based method for displaying high dynamic range image. We segment images into regions and then carry out adaptive contrast and brightness adjustment using global tone mapping operator in the local regions to reproduce local contrast and brightness and ensure better quality. We propose a weighting scheme to eliminate the boundary artifacts caused by the segmentation and decrease the local contrast enhancement adaptively in the uniform area to eliminate the noise introduced. We demonstrate that our methods are easy to use and a fixed set of parameter values produces good results for a wide variety of images.

Diffusion MRI tractography for improved transcranial MRI-guided focused ultrasound thalamotomy targeting for essential tremor

Purpose To evaluate the use of diffusion magnetic resonance imaging (MRI) tractography for neurosurgical guidance of transcranial MRI-guided focused ultrasound (tcMRgFUS) thalamotomy for essential tremor (ET). Materials and methods Eight patients with medication-refractory ET were treated with tcMRgFUS targeting the ventral intermediate nucleus (Vim) of the thalamus contralateral to their dominant hand. Diffusion and structural MRI data and clinical evaluations were acquired pre-treatment and post-treatment. To identify the optimal target location, tractography was performed on pre-treatment diffusion MRI data between the treated thalamus and the hand-knob region of the ipsilateral motor cortex, the entire ipsilateral motor cortex and the contralateral dentate nucleus. The tractography-identified locations were compared to the lesion location delineated on 1 year post-treatment T2-weighted MR image. Their overlap was correlated with the clinical outcomes measured by the percentage change of the Clinical Rating Scale for Tremor scores acquired pre-treatment, as well as 1 month, 3 months, 6 months and 1 year post-treatment. Results The probabilistic tractography was consistent from subject-to-subject and followed the expected anatomy of the thalamocortical radiation and the dentatothalamic tract. Higher overlap between the tractography-identified location and the tcMRgFUS treatment-induced lesion highly correlated with better treatment outcome (r = −0.929, −0.75, −0.643, p = 0.00675, 0.0663, 0.139 for the tractography between the treated thalamus and the hand-knob region of the ipsilateral motor cortex, the entire ipsilateral motor cortex and the contralateral dentate nucleus, respectively, at 1 year post-treatment). The correlation for the tractography between the treated thalamus and the hand-knob region of the ipsilateral motor cortex is the highest for all time points (r = −0.719, −0.976, −0.707, −0.929, p = 0.0519, 0.000397, 0.0595, 0.00675 at 1 month, 3 months, 6 months and 1 year post-treatment, respectively). Conclusion Our data support the use of diffusion tractography as a complementary approach to current targeting methods for tcMRgFUS thalamotomy.