Terry S. Yoo

Engineering and algorithm design for an image processing API: A technical report on ITK - The Insight Toolkit

We present the detailed planning and execution of the Insight Toolkit (ITK), an application programmers interface (API) for the segmentation and registration of medical image data. This public resource has been developed through the NLM Visible Human Project, and is in beta test as an open-source software offering under cost-free licensing. The toolkit concentrates on 3D medical data segmentation and registration algorithms, multimodal and multiresolution capabilities, and portable platform independent support for Windows, Linux/Unix systems. This toolkit was built using current practices in software engineering. Specifically, we embraced the concept of generic programming during the development of these tools, working extensively with C++ templates and the freedom and flexibility they allow. Software development tools for distributed consortium-based code development have been created and are also publicly available. We discuss our assumptions, design decisions, and some lessons learned.

Ion-Abrasion Scanning Electron Microscopy Reveals Surface-Connected Tubular Conduits in HIV-Infected Macrophages

HIV-1-containing internal compartments are readily detected in images of thin sections from infected cells using conventional transmission electron microscopy, but the origin, connectivity, and 3D distribution of these compartments has remained controversial. Here, we report the 3D distribution of viruses in HIV-1-infected primary human macrophages using cryo-electron tomography and ion-abrasion scanning electron microscopy (IA-SEM), a recently developed approach for nanoscale 3D imaging of whole cells. Using IA-SEM, we show the presence of an extensive network of HIV-1-containing tubular compartments in infected macrophages, with diameters of ∼150–200 nm, and lengths of up to ∼5 µm that extend to the cell surface from vesicular compartments that contain assembling HIV-1 virions. These types of surface-connected tubular compartments are not observed in T cells infected with the 29/31 KE Gag-matrix mutant where the virus is targeted to multi-vesicular bodies and released into the extracellular medium. IA-SEM imaging also allows visualization of large sheet-like structures that extend outward from the surfaces of macrophages, which may bend and fold back to allow continual creation of viral compartments and virion-lined channels. This potential mechanism for efficient virus trafficking between the cell surface and interior may represent a subversion of pre-existing vesicular machinery for antigen capture, processing, sequestration, and presentation.

Direct visualization of volume data

A combination of segmentation tools and fast volume renderers that provides an interactive exploration environment for volume visualization is discussed. The tools and renderers include mechanisms that distribute volume data across multiple processors, as well as image compositing techniques and solutions to representation problems in the selection and display of subregions within bounding volumes. A volume visualization technique using the interactive control of images rendered directly from volume data coupled with a user-controlled semantic classification tool is described. The variations of parallel volume rendering being explored on the Pixel-Planes 5 system and the region-of-interest selection methods and the interactive tools used by the system are presented. The flexibility and power of combining volume rendering with region-of-interest selection techniques are demonstrated using examples of medical imaging applications.<<ETX>>

Designing effective transfer functions for volume rendering from photographic volumes

Photographic volumes present a unique, interesting challenge for volume rendering. In photographic volumes, the voxel color is pre-determined, making color selection through transfer functions unnecessary. However, photographic data does not contain a clear mapping from the multi-valued color values to a scalar density or opacity, making projection and compositing much more difficult than with traditional volumes. Moreover, because of the nonlinear nature of color spaces, there is no meaningful norm for the multi-valued voxels. Thus, the individual color channels of photographic data must be treated as incomparable data tuples rather than as vector values. Traditional differential geometric tools, such as intensity gradients, density and Laplacians, are distorted by the nonlinear non-orthonormal color spaces that are the domain of the voxel values. We have developed different techniques for managing these issues while directly rendering volumes from photographic data. We present and justify the normalization of color values by mapping RGB values to the CIE L*u*v* color space. We explore and compare different opacity transfer functions that map three-channel color values to opacity. We apply these many-to-one mappings to the original RGB values as well as to the voxels after conversion to L*u*v* space. Direct rendering using transfer functions allows us to explore photographic volumes without having to commit to an a-priori segmentation that might mask fine variations of interest. We empirically compare the combined effects of each of the two color spaces with our opacity transfer functions using source data from the Visible Human project.

NIH-NSF visualization research challenges report summary

The US National Science Foundation (NSF) convened a panel to report on the potential of visualization as a new technology. The NSF and US National Institutes of Health (NIH) convened the Visualization Research Challenges (VRC) Executive Committee to write a new report. Here, we summarize that new VRC report. We explore the state of the field, examine the potential impact of visualization on areas of national and international importance, and present our findings and recommendations for the future of our growing discipline. Our audience is twofold: the supporters, sponsors, and application users of visualization research on the one hand, and researchers and practitioners in visualization on the other. We direct our discussion toward solving key problems of national interest and helping this works sponsors to concentrate resources to the greatest effect. Our findings and recommendations reflect information gathered from visualization and applications scientists during two workshops on VRC, as well as input from the larger visualization community.

Active contours using a constraint-based implicit representation

We present a new constraint-based implicit active contour, which shares desirable properties of both parametric and implicit active contours. Like parametric approaches, their representation is compact and can be manipulated interactively. Like other implicit approaches, they can naturally adapt to nonsimple topologies. Unlike implicit approaches using level-set methods, representation of the contour does not require a dense mesh. Instead, it is based on specified on-curve and off-curve constraints, which are interpolated using radial basis functions. These constraints are evolved according to specified forces drawn from the relevant literature of both parametric and implicit approaches. This new type of active contour is demonstrated through synthetic images, photographs, and medical images with both simple and nonsimple topologies. For complex input, this approach produces results comparable to those of level set or parameterized finite-element active models, but with a compact analytic representation. As with other active contours they can also be used for tracking, especially for multiple objects that split or merge.

Achieving direct volume visualization with interactive semantic region selection

The authors have achieved rates as high as 15 frames per second for interactive direct visualization of 3D data by trading some function for speed, while volume rendering with a full complement of ramp classification capabilities is performed at 1.4 frames per second. These speeds have made the combination of region selection with volume rendering practical for the first time. Semantic-driven selection, rather than geometric clipping, has proved to be a natural means of interacting with 3D data. Internal organs in medical data or other regions of interest can be built from preprocessed region primitives. The resulting combined system has been applied to real 3D medical data with encouraging results.<<ETX>>

Toward a Common Validation Methodology for Segmentation and Registration Algorithms

The National Library of Medicine and its partners are sponsoring Insight, a public software toolkit for segmentation and registration of high dimensional medical data. An essential element of this initiative is the development of a validation methodology, a common means of comparing the precision, accuracy, and efficiency of segmentation and registration methods. The goal is to make accessible the data, protocol standards, and support software necessary for a common platform for the whole medical image processing community. This paper outlines the issues and design principles for the test and training data and the supporting software that comprise the proposed Insight Validation Suite. We present the methods for establishing the functional design requirements. We also present a framework for the validation of segmentation and registration software and make some suggestions for validation trials. We conclude with some specific recommendations to improve the infrastructure for validating medical image processing research.

Open source software for medical image processing and visualization

Societies often create smaller subsets or communities that connect with one another for commerce and intellectual exchange over mutual interests. In science and engineering, the need for communication among researchers is often hampered by artificial barriers of university politics, economic market forces, and the sheer momentum of an academic reward structure that values individual discovery over joint development. Recent initiatives have attempted to reduce some of these barriers, encouraging collaborative multidisciplinary research programs. Through this effort, we have studied the processes that lead to the successful foundation of new communities.

Using Statistical Pattern Recognition Techniques to Control Variable Conductance Diffusion

We present an approach for controlling relaxation parameters in variable conductance diffusion. This approach incorporates a Bayesian classifier to perform a partial labeling of an image, followed by a diffusion step. Conductance values between pixels are controlled by statistical measurements made of the partial classification. Several iterations follow, interleaving partial labeling with diffusion steps until a convergence or stopping criterion is met. The method is suitable for performing diffusion within multi-valued images. It consistently controls relaxation parameters, even in the presence of noise. The method is presented along with results on phantom and MR images.