Mark D. Zarella

Whither the hypercolumn

Among the crowning achievements of Hubel and Wiesels highly influential studies on primary visual cortex is the description of the cortical hypercolumn, a set of cortical columns with functional properties spanning a particular parameter space. This fundamental concept laid the groundwork for the notion of a modular sensory cortex, canonical cortical circuits and an understanding of visual field coverage beyond simple retinotopy. Surprisingly, the search for and description of analogous hypercolumnar organizations in other cortical areas to date has been limited. In the present work, we have applied the hypercolumn concept to the functional organization of the second visual area, V2. We found it important to separate out the original definition of the hypercolumn from other associated observations and concepts, not all of which are applicable to V2. We present results indicating that, as in V1, the V2 hypercolumns for orientation and binocular interaction (disparity) run roughly orthogonal to each other. We quantified the ‘nearest neighbour’ periodicities for the hypercolumns for ocular dominance, orientation, colour and disparity, and found a marked similarity in the periodicities of all of these hypercolumns, both across hypercolumn type and across visual areas V1 and V2. The results support an underlying common mechanism that constrains the anatomical extent of hypercolumn systems, and highlight the original definition of the cortical hypercolumn.

An optimized color transformation for the analysis of digital images of hematoxylin & eosin stained slides

Hematoxylin and eosin (H&E) staining is ubiquitous in pathology practice and research. As digital pathology has evolved, the reliance of quantitative methods that make use of H&E images has similarly expanded. For example, cell counting and nuclear morphometry rely on the accurate demarcation of nuclei from other structures and each other. One of the major obstacles to quantitative analysis of H&E images is the high degree of variability observed between different samples and different laboratories. In an effort to characterize this variability, as well as to provide a substrate that can potentially mitigate this factor in quantitative image analysis, we developed a technique to project H&E images into an optimized space more appropriate for many image analysis procedures. We used a decision tree-based support vector machine learning algorithm to classify 44 H&E stained whole slide images of resected breast tumors according to the histological structures that are present. This procedure takes an H&E image as an input and produces a classification map of the image that predicts the likelihood of a pixel belonging to any one of a set of user-defined structures (e.g., cytoplasm, stroma). By reducing these maps into their constituent pixels in color space, an optimal reference vector is obtained for each structure, which identifies the color attributes that maximally distinguish one structure from other elements in the image. We show that tissue structures can be identified using this semi-automated technique. By comparing structure centroids across different images, we obtained a quantitative depiction of H&E variability for each structure. This measurement can potentially be utilized in the laboratory to help calibrate daily staining or identify troublesome slides. Moreover, by aligning reference vectors derived from this technique, images can be transformed in a way that standardizes their color properties and makes them more amenable to image processing.

An alternative reference space for H&E color normalization

Digital imaging of H&E stained slides has enabled the application of image processing to support pathology workflows. Potential applications include computer-aided diagnostics, advanced quantification tools, and innovative visualization platforms. However, the intrinsic variability of biological tissue and the vast differences in tissue preparation protocols often lead to significant image variability that can hamper the effectiveness of these computational tools. We developed an alternative representation for H&E images that operates within a space that is more amenable to many of these image processing tools. The algorithm to derive this representation operates by exploiting the correlation between color and the spatial properties of the biological structures present in most H&E images. In this way, images are transformed into a structure-centric space in which images are segregated into tissue structure channels. We demonstrate that this framework can be extended to achieve color normalization, effectively reducing inter-slide variability.

Image processing to extend effective OCT penetration depth in tissue

Innovative approaches in tissue imaging in an in vivo setting have included the use of optical coherence tomography (OCT) as a substrate for providing high resolution images at depths approaching 1.5 mm. This technology has offered the possibility of analyzing many tissues that are presently only evaluated using histologic methods after excision or biopsy. Despite the relatively high penetration depths of OCT, it is unclear whether the images acquired approximately 0.5 mm beyond the tissue surface maintain sufficient resolution and signal-to-noise ratio to provide useful information. Furthermore, there are relatively few studies that evaluate whether advanced image processing can be harnessed to improve the effective depth capabilities of OCT in tissue. We tested a tissue phantom designed to mimic the prostate as a model system, and independently modulated its refractive index and transmittance. Using dynamic focusing, and with the aid of an image analysis paradigm designed to improve signal detection in a model of tissue, we tested potential improvements in the ability to resolve structures at increasing penetration depths. We found that co-registered signal averaging and wavelet denoising improved overall image quality. B-spline interpolation made it possible to integrate dynamic focus images in a way that improved the effective penetration depth without significant loss in overall image quality. These results support the notion that image processing can refine OCT images for improved diagnostic capabilities to support in vivo microscopy.

A Template Matching Model for Nuclear Segmentation in Digital Images of H&E Stained Slides

Pathology has become increasingly more reliant on digital imaging as a means for viewing, sharing, and archiving slides, and as an essential first step for the application of advanced image analysis to support cancer diagnostics. In H&E stained tissue, cell nuclei are especially prominent, and their shapes, staining attributes, and distributions within the tissue serve as important diagnostic and prognostic features. Therefore, the ability to accurately identify and segment nuclei from other tissue structures is paramount toward developing a reliable analytical tool. We developed an algorithm that rapidly identifies candidate nuclei and segments them in a manner that retains much of the shape information and location precision. The algorithm uses color analysis, template matching based on shape, and clump splitting to demarcate individual nuclei and to segregate overlapping nuclei. Given its speed and relative simplicity, this method is especially amenable to processing large image regions at high magnification, making high throughput and on-demand analysis realizable.

Contextual modulation revealed by optical imaging exhibits figural asymmetry in macaque V1 and V2

Neurons in early visual cortical areas are influenced by stimuli presented well beyond the confines of their classical receptive fields, endowing them with the ability to encode fine-scale features while also having access to the global context of the visual scene. This property can potentially define a role for the early visual cortex to contribute to a number of important visual functions, such as surface segmentation and figure–ground segregation. It is unknown how extraclassical response properties conform to the functional architecture of the visual cortex, given the high degree of functional specialization in areas V1 and V2. We examined the spatial relationships of contextual activations in macaque V1 and V2 with intrinsic signal optical imaging. Using figure–ground stimulus configurations defined by orientation or motion, we found that extraclassical modulation is restricted to the cortical representations of the figural component of the stimulus. These modulations were positive in sign, suggesting a relative enhancement in neuronal activity that may reflect an excitatory influence. Orientation and motion cues produced similar patterns of activation that traversed the functional subdivisions of V2. The asymmetrical nature of the enhancement demonstrated the capacity for visual cortical areas as early as V1 to contribute to figure–ground segregation, and the results suggest that this information can be extracted from the population activity constrained only by retinotopy, and not the underlying functional organization.

Cue combination encoding via contextual modulation of V1 and V2 neurons

Neurons in early visual cortical areas encode the local properties of a stimulus in a number of different feature dimensions such as color, orientation, and motion. It has been shown, however, that stimuli presented well beyond the confines of the classical receptive field can augment these responses in a way that emphasizes these local attributes within the greater context of the visual scene. This mechanism imparts global information to cells that are otherwise considered local feature detectors and can potentially serve as an important foundation for surface segmentation, texture representation, and figure–ground segregation. The role of early visual cortex toward these functions remains somewhat of an enigma, as it is unclear how surface segmentation cues are integrated from multiple feature dimensions. We examined the impact of orientation- and motion-defined surface segmentation cues in V1 and V2 neurons using a stimulus in which the two features are completely separable. We find that, although some cells are modulated in a cue-invariant manner, many cells are influenced by only one cue or the other. Furthermore, cells that are modulated by both cues tend to be more strongly affected when both cues are presented together than when presented individually. These results demonstrate two mechanisms by which cue combinations can enhance salience. We find that feature-specific populations are more frequently encountered in V1, while cue additivity is more prominent in V2. These results highlight how two strongly interconnected areas at different stages in the cortical hierarchy can potentially contribute to scene segmentation.