Kevin Novins

On Benchmarking Optical Flow

Evaluating the performance of optical flow algorithms has been difficult because of the lack of ground truth data sets for complex scenes. We present a new method for generating motion fields from real sequences containing polyhedral objects and present a test suite for benchmarking optical flow algorithms consisting of complex synthetic sequences and real scenes with ground truth. We provide a preliminary quantitative evaluation of seven optical flow algorithms using these synthetic and real sequences. Ultimately, we feel that researchers should benchmark their own algorithms using a standard suite. To that end, we offer our Web site as a repository for standard sequences and results.

Recovering Motion Fields: An Evaluation of Eight Optical Flow Algorithms

Evaluating the performance of optical flow algorithms has been difficult because of the lack of ground-truth data sets for complex scenes. We describe a simple modification to a ray tracer that allows us to generate ground-truth motion fields for scenes of arbitrary complexity. The resulting flow maps are used to assist in the comparison of eight optical flow algorithms using three complex, synthetic scenes. Our study found that a modified version of Lucas and Kanade’s algorithm has superior performance but produces sparse flow maps. Proesmans et al.’s algorithm performs slightly worse, on average, but produces a very dense depth map.

Computation of digitally reconstructed radiographs for use in radiotherapy treatment design

The increasing use of 3-dimensional radiotherapy treatment design has created greater reliance on methods for computing images from CT data which correspond to the conventional simulation film. These images, known as computed or digitally reconstructed radiographs, serve as reference images for verification of computer-designed treatments. Used with software that registers graphic overlays of target and anatomic structures, digitally reconstructed radiographs are also valuable tools for designing portal shape. We have developed radiograph reconstruction software that takes full advantage of the contrast and spatial detail inherent in the original CT data. This goal has been achieved by using a ray casting algorithm which explicitly takes into account every intersected voxel, and a heuristic approach for approximating the images that would result from purely photoelectric or Compton interactions. The software also offers utilities to superimpose outlines of anatomic structures, field edges, beam crosshairs, and linear scales on digitally reconstructed radiographs. The pixel size of the computed image can be controlled, and several methods of interslice interpolation are offered. The software is written in modular format in the C language, and can stand alone or interface with other treatment planning software.

Fluid sketches: continuous recognition and morphing of simple hand-drawn shapes

We describe a new sketching interface in which shape recognition and morphing are tightly coupled. Raw input strokes are continuously morphed into ideal geometric shapes, even before the pen is lifted. By means of smooth and continual shape transformations the user is apprised of recognition progress and the appearance of the final shape, yet always retains a sense of control over the process. At each time t the system uses the trajectory traced out thus far by the pen coupled with the current appearance of the time-varying shape to classify the sketch as one of several pre-defined basic shapes. The recognition operation is performed using shape-specific fits based on least-squares or relaxation, which are continuously updated as the user draws. We describe the time-dependent transformation of the sketch, beginning with the raw pen trajectory, using a family of first-order ordinary differential equations that depend on time and the current shape of the sketch. Using this formalism, we describe several possible behaviors that result by varying the relative significance of new and old portions of a stroke, changing the “viscosity” of the morph, and enforcing different end conditions. A preliminary user study suggests that the new interface is particularly effective for rapidly constructing diagrams consisting of simple shapes.

Equation entry and editing via handwriting and gesture recognition

We describe a system for freehand entry and editing of mathematical expressions using a pen and tablet. The expressions are entered in the same way that they would be written on paper. The system interprets the results and generates output in a form suitable for use in other applications, such as word processors or symbolic manipulators. Interpretation includes character segmentation, character recognition, and formula parsing. Our interface incorporates easy to use tools for correcting interpretation errors at any stage. The user can also edit the handwritten representation and ask the system to reinterpret the results. By recovering the formulas structure directly from its handwritten form, the user is free to use common conventions of mathematical notation without regard to internal representation. We report the results of a small user study, which indicate that the new style of interaction is effective.

Algorithmic Fusion for More Robust Feature Tracking

We present a framework for merging the results of independent feature-based motion trackers using a classification based approach. We demonstrate the efficacy of the framework using corner trackers as an example. The major problem with such systems is generating ground truth data for training. We show how synthetic data can be used effectively to overcome this problem. Our combined system performs better in both dropouts and errors than a correspondence tracker, and had less than half the dropouts at the cost of moderate increase in error compared to a relaxation tracker.

Echocardiographic Three-Dimensional Visualization of the Heart

To perform three-dimensional (3-D) reconstruction of the heart by ultrasound, we developed a novel rotating echocardiographic probe which, with computer assistance, allows “real” 3-D reconstruction of the beating heart from 62 standard fan shaped two-dimensional (2-D) images acquired at 2.903 degree increments of rotation around its central axis. To reconstruct 3-D images of the beating heart, an entire cardiac cycle was recorded from each transducer position with electrocardiographic gating; acquisition time is 75 to 123 seconds in normal sinus rhythm. For each frame of the cardiac cycle, the 62 images digitized in cylindrical coordinates were processed by a scan converter algorithm to reconstruct a 3-D cone of information in cartesian coordinates. From the 3-D matrices stored in the computer, 2-D echocardiographic images in any plane at specified times in the cardiac cycle, or throughout the entire cardiac cycle, can be derived and visualized. A computer workstation-based system was developed to create full 3-D perspective projections of the echocardiographic data based on a technique called ray tracing, and adapted for use in visualizing 3-D scalar fields. The 3-D images obtained in normal volunteers demonstrated that our system permits an accurate reconstruction of the heart with the same spatial and temporal resolution as the original 2-D echocardiograms without cumbersome external reference systems.

Controlled precision volume integration

Traditional methods for evaluating the low-albedo volume rendering integral do not include bounds on the magnitude of approximation error. In this paper, we examine three techniques for solving this integral with error bounds: trapezoid rule, Simpson’s rule, and a power series method. In each case, the expression for the error bound provides a mechanism for computing the integral to any specified precision. The formulations presented are appropriate for polynomial reconstruction from point samples; however, the approach is considerably mom general. The three techniques we present differ in relative efficiency for computing results to a given precision. The trapezoid rule and Simpson’s rule are most efficient for lowto medium-precision solutions. The power series method converges rapidly to a machine precision solution, providing both an efficient means for high-accuracy volume rendering, and a reference standard by which other approximations may be measured. CR

Appearance-preserving manipulation of hand-drawn graphs

We describe a sketching system that allows users to create and manipulate directed graphs, such as those depicting state diagrams, using pen-input alone. The system exactly preserves the users strokes, which may be entered in any order, and depicts them with a chalk texture to evoke a blackboard metaphor. The system automatically interprets the geometry of the sketch, distinguishing vertices, edges, and arrow heads, then tacitly imparts the intended graph semantics based on the two-dimensional placement of these elements. Once drawn, the user can manipulate the directed graph gesturally using the pen. The system responds to vertices or edges being picked and dragged by adjusting all adjacent edges appropriately. The original appearance of the hand-drawn vertices and edges is maintained even while their shapes are continually morphed in response to rearrangement of these elements. All edges exhibit shape memory, which is the proclivity to return to their original hand-drawn shape despite repeated stretching and compression.

Polygon recognition in sketch-based interfaces with immediate and continuous feedback

Sketch-based drawing interfaces attempt to combine the ease of freehand drawing with the advantages of computer processing by analysing hand-drawn pen strokes in real time and replacing them with perfect geometric representations. This paper describes the development of a sketch-based interface for drawing polygons. An investigation into how people draw polygons revealed that it is possible to detect the corners of a polygon while it is being drawn by examining the time stamps of the sample points on the users stroke. Results of a pilot user study suggest that this method is more pleasant and convenient to use than conventional polygon drawing tools and would be useful for composing drawings quickly. However, users still prefer the traditional method of clicking in the location of each corner for drawing accurately.