William A. Barrett

Intelligent scissors for image composition

We present a new, interactive tool called Intelligent Scissors which we use for image segmentation and composition. Fully automated segmentation is an unsolved problem, while manual tracing is inaccurate and laboriously unacceptable. However, Intelligent Scissors allow objects within digital images to be extracted quickly and accurately using simple gesture motions with a mouse. When the gestured mouse position comes in proximity to an object edge, a live-wire boundary “snaps” to, and wraps around the object of interest. Live-wire boundary detection formulates discrete dynamic programming (DP) as a two-dimensional graph searching problem. DP provides mathematically optimal boundaries while greatly reducing sensitivity to local noise or other intervening structures. Robustness is further enhanced with on-the-fly training which causes the boundary to adhere to the specific type of edge currently being followed, rather than simply the strongest edge in the neighborhood. Boundary cooling automatically freezes unchanging segments and automates input of additional seed points. Cooling also allows the user to be much more free with the gesture path, thereby increasing the efficiency and finesse with which boundaries can be extracted. Extracted objects can be scaled, rotated, and composited using live-wire masks and spatial frequency equivalencing. Frequency equivalencing is performed by applying a Butterworth filter which matches the lowest frequency spectra to all other image components. Intelligent Scissors allow creation of convincing compositions from existing images while dramatically increasing the speed and precision with which objects can be extracted.

Interactive segmentation with Intelligent Scissors

Abstract We present a new, interactive tool called Intelligent Scissors which we use for image segmentation. Fully automated segmentation is an unsolved problem, while manual tracing is inaccurate and laboriously unacceptable. However, Intelligent Scissors allow objects within digital images to be extracted quickly and accurately using simple gesture motions with a mouse. When the gestured mouse position comes in proximity to an object edge, a live-wire boundary “snaps” to, and wraps around the object of interest. Live-wire boundary detection formulates boundary detection as an optimal path search in a weighted graph. Optimal graph searching provides mathematically piece-wise optimal boundaries while greatly reducing sensitivity to local noise or other intervening structures. Robustness is further enhanced with on-the-fly training which causes the boundary to adhere to the specific type of edge currently being followed, rather than simply the strongest edge in the neighborhood. Boundary cooling automatically freezes unchanging segments and automates input of additional seed points. Cooling also allows the user to be much more free with the gesture path, thereby increasing the efficiency and finesse with which boundaries can be extracted.

Adaptive boundary detection using 'live-wire' two-dimensional dynamic programming

An adaptive boundary detection algorithm that uses two-dimensional dynamic programming (DP) is presented. The algorithm is less constrained than previous one-dimensional dynamic programming algorithms and allows the user to interactively determine the mathematically optimal boundary between a user-selected seed point and any other dynamically selected free point in the image. Interactive movement of the free point by the cursor causes the boundary to behave like a live wire as it adapts to the new minimum cost path between the seed point and the currently selected free point. The algorithm can also be adapted or customized to learn boundary-defining features for a particular class of images. Adaptive 2-D DP performs well on a variety of images. It accurately detects the boundaries of low contrast objects, which occur with intravenous injections, as well as those found in noisy, low SNR images.<<ETX>>

Toboggan-based intelligent scissors with a four-parameter edge model

Intelligent Scissors is an interactive image segmentation tool that allows a user to select piece-wise globally optimal contour segments that correspond to a desired object boundary. We present a new and faster method of computing the optimal path by over-segmenting the image using tobogganing and then imposing a weighted planar graph on top of the resulting region boundaries. The resulting region-based graph is many times smaller than the previous pixel-based graph, thus providing faster graph searches and immediate user interaction. Further tobogganing provides an new systematic and predictable framework for computing edge model parameters, allowing subpixel localization as well as a measure of edge blur.

Boundary detection via dynamic programming

This paper reports a new method for detecting optimal boundaries in multidimensional scene data via dynamic programming (DP). In its current form the algorithm detects 2-D contours on slices and differs from other reported DP-based algorithms in an essential way in that it allows freedom in 2-D for finding optimal contour paths (as opposed to a single degree of freedom in the published methods). The method is being successfully used in segmenting object boundaries in a variety of medical applications including orbital volume from CT images (for craniofacial surgical planning), segmenting bone in MR images for kinematic analysis of the joints of the foot, segmenting the surface of the brain from the inner surface of the cranial vault, segmenting pituitary gland tumor for following the effect of a drug on the tumor, segmenting the boundaries of the heart in MR images, and segmenting the olfactory bulb for verifying hypotheses related to the size of this bulb in certain disease states.

In-Vivo Alignment Comparing Patient Specific Instrumentation with both Conventional and Computer Assisted Surgery (CAS) Instrumentation in Total Knee Arthroplasty

Patient specific instrumentation (PSI) was developed to increase total knee arthroplasty (TKA) accuracy and efficiency. The study purpose was to compare immediate post-operative mechanical alignment, achieved using PSI, with conventional and computer assisted surgery (CAS) instruments in high volume TKA practices. This prospective, multicenter, non-randomized study accrued 66 TKA patients using PSI. A computed tomography (CT) based algorithm was used to develop the surgical plan. Sixty-two percent were females, 99% were diagnosed with osteoarthritis, average age at surgery was 66 years, and 33 was the average body mass index. A historical control group was utilized that underwent TKA using conventional instruments (n=86) or CAS (n=81), by the same set of surgeons. Postoperative mechanical alignment was comparable across the groups. Operative time mean and variance were significant.

Interactive segmentation of image volumes with Live Surface

Live Surface allows users to segment and render complex surfaces from 3D image volumes at interactive (sub-second) rates using a novel, cascading graph cut (CGC). Live Surface consists of two phases: (1) preprocessing for generation of a complete 3D hierarchy of tobogganed regions followed by tracking of all region surfaces; (2) user interaction in which, with each mouse movement, the volume is segmented and the 3D object is rendered at interactive rates. Interactive segmentation is accomplished by cascading through the 3D hierarchy from the top, applying graph cut successively, at each level, only to regions bordering the segmented surface from the previous level. CGC allows the entire image volume to be segmented an order of magnitude faster than existing techniques that make use of graph cut. OpenGL rendering provides for display and update of the segmented surface at interactive rates. The user selects objects by tagging voxels with either foreground (object) or background seeds. Seeds can be placed on image cross-sections or directly on the 3D rendered surface. Interaction with the rendered surface improves the users ability to steer the segmentation, augmenting or subtracting from the current selection. Segmentation and rendering, combined, is accomplished in about 0.35s, allowing 3D surfaces to be displayed and updated dynamically as each additional seed is deposited. The immediate feedback of Live Surface allows the segmentation of 3D image volumes using an interaction paradigm similar to the Live Wire (Intelligent Scissors) tool used in 2D images.

Fast, Accurate, and Reproducible Live-Wire Boundary Extraction

We present an interactive tool for efficient, accurate, and reproducible boundary extraction which requires minimal user input with a mouse. Optimal boundaries are computed and selected at interactive rates as the user moves the mouse starting from a user-selected seed point. When the mouse position comes in proximity to an object edge, a “live-wire” boundary snaps to, and wraps around the object of interest. Input of a new seed point “freezes” the selected boundary segment, and the process is repeated until the boundary is complete. Data-driven boundary cooling generates seed points automatically and further reduces user input. On-the-fly training adapts the dynamic boundary to edges of current interest.

Separating lines of text in free-form handwritten historical documents

We present an approach to finding (and separating) lines of text in free-form handwritten historical document images. After preprocessing, our method uses the count of foreground/background transitions in a binarized image to determine areas of the document that are likely to be text lines. Alternatively, an adaptive local connectivity map (ALCM) found in the literature can be used for this step of the process. We then use a min-cut/max-flow graph cut algorithm to split up text areas that appear to encompass more than one line of text. After removing text lines containing relatively little text information (or merging them with nearby text lines), we create output images for each line. A grayscale output image is created, as well as a special mask image containing both the foreground and information flagging ambiguous pixels. Foreground pixels that belong to other text lines are removed from the output images to provide cleaner line images useful for further processing. While some refinement is still necessary, the result of early experimentation with our method is encouraging

Object-based vectorization for interactive image editing

We present a technique for creating an editable vector graphic from an object in a raster image. Object selection is performed interactively in subsecond time by calling graph cut with each mouse movement. A renderable mesh is then computed automatically for the selected object and each of its subobjects by (1) generating a coarse object mesh; (2) performing recursive graph cut segmentation and hierarchical ordering of subobjects; (3) applying error-driven mesh refinement to each (sub)object. The fully layered object hierarchy compares favorably with current approaches and is computed in a few 10s of seconds, facilitating object-level editing without leaving holes.