Seth Billings

System for robot-assisted real-time laparoscopic ultrasound elastography

Surgical robots provide many advantages for surgery, including minimal invasiveness, precise motion, high dexterity, and crisp stereovision. One limitation of current robotic procedures, compared to open surgery, is the loss of haptic information for such purposes as palpation, which can be very important in minimally invasive tumor resection. Numerous studies have reported the use of real-time ultrasound elastography, in conjunction with conventional B-mode ultrasound, to differentiate malignant from benign lesions. Several groups (including our own) have reported integration of ultrasound with the da Vinci robot, and ultrasound elastography is a very promising image guidance method for robotassisted procedures that will further enable the role of robots in interventions where precise knowledge of sub-surface anatomical features is crucial. We present a novel robot-assisted real-time ultrasound elastography system for minimally invasive robot-assisted interventions. Our system combines a da Vinci surgical robot with a non-clinical experimental software interface, a robotically articulated laparoscopic ultrasound probe, and our GPU-based elastography system. Elasticity and B-mode ultrasound images are displayed as picture-in-picture overlays in the da Vinci console. Our system minimizes dependence on human performance factors by incorporating computer-assisted motion control that automatically generates the tissue palpation required for elastography imaging, while leaving high-level control in the hands of the user. In addition to ensuring consistent strain imaging, the elastography assistance mode avoids the cognitive burden of tedious manual palpation. Preliminary tests of the system with an elasticity phantom demonstrate the ability to differentiate simulated lesions of varied stiffness and to clearly delineate lesion boundaries.

Iterative Most-Likely Point Registration (IMLP): A Robust Algorithm for Computing Optimal Shape Alignment

We present a probabilistic registration algorithm that robustly solves the problem of rigid-body alignment between two shapes with high accuracy, by aptly modeling measurement noise in each shape, whether isotropic or anisotropic. For point-cloud shapes, the probabilistic framework additionally enables modeling locally-linear surface regions in the vicinity of each point to further improve registration accuracy. The proposed Iterative Most-Likely Point (IMLP) algorithm is formed as a variant of the popular Iterative Closest Point (ICP) algorithm, which iterates between point-correspondence and point-registration steps. IMLP’s probabilistic framework is used to incorporate a generalized noise model into both the correspondence and the registration phases of the algorithm, hence its name as a most-likely point method rather than a closest-point method. To efficiently compute the most-likely correspondences, we devise a novel search strategy based on a principal direction (PD)-tree search. We also propose a new approach to solve the generalized total-least-squares (GTLS) sub-problem of the registration phase, wherein the point correspondences are registered under a generalized noise model. Our GTLS approach has improved accuracy, efficiency, and stability compared to prior methods presented for this problem and offers a straightforward implementation using standard least squares. We evaluate the performance of IMLP relative to a large number of prior algorithms including ICP, a robust variant on ICP, Generalized ICP (GICP), and Coherent Point Drift (CPD), as well as drawing close comparison with the prior anisotropic registration methods of GTLS-ICP and A-ICP. The performance of IMLP is shown to be superior with respect to these algorithms over a wide range of noise conditions, outliers, and misalignments using both mesh and point-cloud representations of various shapes.

Iterative Most Likely Oriented Point Registration

A new algorithm for model based registration is presented that optimizes both position and surface normal information of the shapes being registered. This algorithm extends the popular Iterative Closest Point (ICP) algorithm by incorporating the surface orientation at each point into both the correspondence and registration phases of the algorithm. For the correspondence phase an efficient search strategy is derived which computes the most probable correspondences considering both position and orientation differences in the match. For the registration phase an efficient, closed-form solution provides the maximum likelihood rigid body alignment between the oriented point matches. Experiments by simulation using human femur data demonstrate that the proposed Iterative Most Likely Oriented Point (IMLOP) algorithm has a strong accuracy advantage over ICP and has increased ability to robustly identify a successful registration result.

Automated diagnosis of myositis from muscle ultrasound: Exploring the use of machine learning and deep learning methods

Objective To evaluate the use of ultrasound coupled with machine learning (ML) and deep learning (DL) techniques for automated or semi-automated classification of myositis. Methods Eighty subjects comprised of 19 with inclusion body myositis (IBM), 14 with polymyositis (PM), 14 with dermatomyositis (DM), and 33 normal (N) subjects were included in this study, where 3214 muscle ultrasound images of 7 muscles (observed bilaterally) were acquired. We considered three problems of classification including (A) normal vs. affected (DM, PM, IBM); (B) normal vs. IBM patients; and (C) IBM vs. other types of myositis (DM or PM). We studied the use of an automated DL method using deep convolutional neural networks (DL-DCNNs) for diagnostic classification and compared it with a semi-automated conventional ML method based on random forests (ML-RF) and “engineered” features. We used the known clinical diagnosis as the gold standard for evaluating performance of muscle classification. Results The performance of the DL-DCNN method resulted in accuracies ± standard deviation of 76.2% ± 3.1% for problem (A), 86.6% ± 2.4% for (B) and 74.8% ± 3.9% for (C), while the ML-RF method led to accuracies of 72.3% ± 3.3% for problem (A), 84.3% ± 2.3% for (B) and 68.9% ± 2.5% for (C). Conclusions This study demonstrates the application of machine learning methods for automatically or semi-automatically classifying inflammatory muscle disease using muscle ultrasound. Compared to the conventional random forest machine learning method used here, which has the drawback of requiring manual delineation of muscle/fat boundaries, DCNN-based classification by and large improved the accuracies in all classification problems while providing a fully automated approach to classification.

Generalized iterative most likely oriented-point (G-IMLOP) registration.

PurposeThe need to align multiple representations of anatomy is a problem frequently encountered in clinical applications. A new algorithm for feature-based registration is presented that solves this problem by aligning both position and orientation information of the shapes being registered.MethodsThe iterative most likely oriented-point (IMLOP) algorithm and its generalization (G-IMLOP) to the anisotropic noise case are described. These algorithms may be understood as probabilistic variants of the popular iterative closest point (ICP) algorithm. A probabilistic model provides the framework, wherein both position information and orientation information are simultaneously optimized. Like ICP, the proposed algorithms iterate between correspondence and registration subphases. Efficient and optimal solutions are presented for implementing each subphase of the proposed methods.ResultsExperiments based on human femur data demonstrate that the IMLOP and G-IMLOP algorithms provide a strong accuracy advantage over ICP, with G-IMLOP providing additional accuracy improvement over IMLOP for registering data characterized by anisotropic noise. Furthermore, the proposed algorithms have increased ability to robustly identify an accurate versus inaccurate registration result.ConclusionThe IMLOP and G-IMLOP algorithms provide a cohesive framework for incorporating orientation data into the registration problem, thereby enabling improvement in accuracy as well as increased confidence in the quality of registration outcomes. For shape data having anisotropic uncertainty in position and/or orientation, the anisotropic noise model of G-IMLOP enables further gains in registration accuracy to be achieved.

A hybrid surface/image-based approach to facilitate ultrasound/CT registration

Registration of intra-operative ultrasound with preoperative CT is highly desirable as a navigational aid for surgeons and interventional radiologists. Image-based solutions generally achieve poor results due to substantially different image appearance of ultrasound and CT. A method is presented that uses surface information and tracked ultrasound to improve registration results. Tracked ultrasound is combined with surface and image-based registration techniques to register ultrasound to CT. Surface data is acquired using an optically tracked range sensor, for example time-of-flight camera. Range data is registered to CT using robust point-set registration; this registration provides an approximate transformation from tracker to CT coordinates. The ultrasound probe is also optically tracked. The probe position and surface-based registration provide a first estimate for the position of the ultrasound image in CT coordinates. This estimate is subsequently refined by a final image-based registration stage. Initial tests using Coherent Point Drift algorithm for registering surface data to CT show favorable results. Tests using both simulated and real time-of-flight range data have good convergence over a wide initial translation and rotation misalignment domain. Preliminary testing using time-of-flight surface data suggests that surface to CT registration may be useful as an initial guess enabling later more precise (but less robust) image based methods for registering ultrasound images to CT. We believe this method will enable image-based algorithms to robustly converge to an optimal registration solution.

Elastography Using Multi-Stream GPU: An Application to Online Tracked Ultrasound Elastography, In-Vivo and the da Vinci Surgical System

A system for real-time ultrasound (US) elastography will advance interventions for the diagnosis and treatment of cancer by advancing methods such as thermal monitoring of tissue ablation. A multi-stream graphics processing unit (GPU) based accelerated normalized cross-correlation (NCC) elastography, with a maximum frame rate of 78 frames per second, is presented in this paper. A study of NCC window size is undertaken to determine the effect on frame rate and the quality of output elastography images. This paper also presents a novel system for Online Tracked Ultrasound Elastography (O-TRuE), which extends prior work on an offline method. By tracking the US probe with an electromagnetic (EM) tracker, the system selects in-plane radio frequency (RF) data frames for generating high quality elastograms. A novel method for evaluating the quality of an elastography output stream is presented, suggesting that O-TRuE generates more stable elastograms than generated by untracked, free-hand palpation. Since EM tracking cannot be used in all systems, an integration of real-time elastography and the da Vinci Surgical System is presented and evaluated for elastography stream quality based on our metric. The da Vinci surgical robot is outfitted with a laparoscopic US probe, and palpation motions are autonomously generated by customized software. It is found that a stable output stream can be achieved, which is affected by both the frequency and amplitude of palpation. The GPU framework is validated using data from in-vivo pig liver ablation; the generated elastography images identify the ablated region, outlined more clearly than in the corresponding B-mode US images.

Active multispectral illumination and image fusion for retinal microsurgery

It has been shown that white light exposure during retinal microsurgeries is detrimental to patients. To address this problem, we present a novel device and image processing tool, which can be used to significantly reduce the amount of phototoxicity induced in the eye. Our device alternates between illuminating the retina using white and limited spectrum light, while a fully automated image processing algorithm produces a synthetic white light video by colorizing non-white light images. We show qualitatively and quantitatively that our system can provide reliable images using far less toxic light when compared to traditional systems. In addition, the method proposed in this paper may be adapted to other clinical applications in order to give surgeons more flexibility when visualizing areas of interest.

Anatomically Constrained Video-CT Registration via the V-IMLOP Algorithm

Functional endoscopic sinus surgery (FESS) is a surgical procedure used to treat acute cases of sinusitis and other sinus diseases. FESS is fast becoming the preferred choice of treatment due to its minimally invasive nature. However, due to the limited field of view of the endoscope, surgeons rely on navigation systems to guide them within the nasal cavity. State of the art navigation systems report registration accuracy of over 1mm, which is large compared to the size of the nasal airways. We present an anatomically constrained video-CT registration algorithm that incorporates multiple video features. Our algorithm is robust in the presence of outliers. We also test our algorithm on simulated and in-vivo data, and test its accuracy against degrading initializations.

Ultrasound image analysis for myopathy detection

This study focuses on using ultrasound (US) biomarkers for characterizing myopathies and in particular myositis. US offers an opportunity to deliver diagnostics in clinical settings at a fraction of the cost and discomfort entailed in current workflows. US is also better suited for usage in under-resourced environments. This paper is focused on studying the link between biomarkers related to absolute and relative echo intensity of muscle tissue and the presence and severity of myositis disease. We show that there is good correlation between these biomarkers and the severity of muscle disease rated by the Heckmatt criteria. A moderate correlation is also found between these biomarkers and muscles categorized by healthy vs. diseased status of each patient. Experimental data involving 37 patients (9 polymyositis, 3 dermatomyositis, 9 inclusion body myositis, and 16 healthy patients) and seven muscle groups show correlations up to 0.91.