Nicholas A. Ramey

Stereo-Based Endoscopic Tracking of Cardiac Surface Deformation

We propose an image-based motion tracking algorithm that can be used with stereo endoscopic and microscope systems. The tracking problem is considered to be a time-varying optimization of a parametric function describing the disparity map. This algorithm could be used as part of a virtual stabilization system that can be employed to compensate residual motion of the heart during robot-assisted off-pump coronary artery bypass surgery (CABG). To test the appropriateness of our methods for this application, we processed an image sequence of a beating pig heart obtained by the stereo endoscope used in the da Vinci robotic surgery system. The tracking algorithm was able to detect the beating of the heart itself as well as the respiration of the lungs.

Navigating inner space: 3-D assistance for minimally invasive surgery

Abstract Since its inception about three decades ago, modern minimally invasive surgery has made huge advances in both technique and technology. However, the minimally invasive surgeon is still faced with daunting challenges in terms of visualization and hand-eye coordination. At the Center for Computer Integrated Surgical Systems and Technology (CISST) we have been developing a set of techniques for assisting surgeons in navigating and manipulating the three-dimensional space within the human body. In order to develop such systems, a variety of challenging visual tracking, reconstruction and registration problems must be solved. In addition, this information must be tied to methods for assistance that improve surgical accuracy and reliability but allow the surgeon to retain ultimate control of the procedure and do not prolong time in the operating room. In this article, we present two problem areas, eye microsurgery and thoracic minimally invasive surgery, where computational vision can play a role. We then describe methods we have developed to process video images for relevant geometric information, and related control algorithms for providing interactive assistance. Finally, we present results from implemented systems.

Real-time 3D Surface Tracking and Its Applications

We present a general technique for directly estimating and tracking surfaces from a stream of rectified stereo pairs in real-time. These techniques are based on the iterative updating of surface representations directly from image information and use no disparity search except during initialization. We perform the tracking through an iteratively re-weighted least squares minimization wherein a mask is incorporated to increase robustness to occlusion. The algorithms are formulated for a general family of linear in parameters surface models and discussed for the cases of planar surfaces and tensor product surfaces. These algorithms have been implemented on standard hardware and run at or near frame rate, with accuracy on the order of 1/20 of a pixel. We discuss applications of the technique including mobile robot localization, general deforming surface tracking, and biometry of biological surfaces.

Imaging Mitochondria in Living Corneal Endothelial Cells Using Autofluorescence Microscopy

We report noninvasive autofluorescence mitochondrial imaging in cultured human corneal endothelial cells (HCECs). HCECs harvested from eye bank corneas were cultured in thin glass‐bottom plates. Mitochondria were imaged with an autofluorescence microscope using a DAPI filter set (excitation: G365, emission: band pass 445/50) and then, after fixation with 4% paraformaldehyde, cells were stained with MitoTracker™ Green FM (MTG). Both images were aligned using a linear conformal algorithm for image mapping based on manually selected corresponding feature points, and then mathematically compared using two‐dimensional spatial image correlation coefficients. Autofluorescence imaging provided highly resolved mitochondrial signals from living HCECs, comparable to those taken with MTG. Both techniques yielded very similar images at high magnification and high resolution, demonstrating the tubular morphology and cytoplasmic distribution that are characteristic of mitochondria. Image registration using a linear conformal mapping technique and cross‐correlations showed high correlation of overlapping autofluorescence and MTG images. This study validates the novel use of autofluorescence vital imaging as a noninvasive, inexpensive and functional alternative to the mitochondria‐specific dyes in cultured HCEC. This noninvasive mitochondrial imaging technique can be useful in future applications studying mitochondrial biology of ocular cells.

Vertical vergence adaptation produces an objective vertical deviation that changes with head tilt.

PURPOSE To document the cyclovertical ocular motor mechanism used for vertical fusion in healthy subjects, and to explore whether vertical vergence training in healthy individuals can produce objectively confirmed vertical deviations that change with head tilt, revealing a basic mechanism that can produce a pattern of misalignment in an otherwise normal ocular motor system that is similar to superior oblique muscle paresis (SOP). METHODS Seven subjects with normal orthoptic examinations were adapted to vertical image disparities using our tilting haploscopic eye-tracking apparatus presenting concentric circle targets without torsional cues. Static eye positions were recorded with head straight and when tilted 45 degrees to the left and right, during both binocular and monocular viewing. RESULTS Vertical fusional vergence was accompanied by a cycloversion, with the downward-moving eye intorting and the upward-moving eye extorting, implicating primary involvement of the oblique extraocular muscles. After adaptation to the slowly increasing vertical target separation, all subjects developed a temporary vertical deviation in the straight ahead position that increased with head tilt to one side and decreased with head tilt to the other side. CONCLUSIONS These results not only show that head-tilt-dependent changes in vertical deviation are not necessarily pathognomonic for SOP, but also, and more importantly, suggest mechanisms that can mimic SOP and suggest a possible role for vertical vergence training in reducing deviations and thus the amount of head tilt required for fusion. Ultimately, vertical vergence training may provide an adjunct or alternative to extraocular muscle surgery in selected cases.