Thomas F. Stahovich

An image-based, trainable symbol recognizer for hand-drawn sketches

We describe a trainable, hand-drawn symbol recognizer based on a multi-layer recognition scheme. Symbols are internally represented as binary templates. An ensemble of four different classifiers compares and ranks definition symbols according to their similarity to the unknown symbol. The scores of the individual classifiers are aggregated to produce a combined score for each definition. The definition with the best combined score is assigned to the unknown symbol. All four classifiers use template-matching techniques to compute similarity (and dissimilarity) between symbols. Ordinarily, template-matching is sensitive to rotation, and existing solutions for rotation invariance are too expensive for interactive performance. We have developed a fast technique that uses a polar coordinate representation to achieve rotational invariance. This technique is applied prior to the multi-classifier recognition step to determine the best alignment of the unknown with each definition. One advantage of this technique is that it filters out the bulk of unlikely definitions, thereby reducing the number of definitions the multi-classifier recognition step must consider.

Combining geometry and domain knowledge to interpret hand-drawn diagrams

One main challenge in building interpreters for hand-drawn sketches is the task of parsing a sketch to locate the individual symbols. Many existing pen-based systems avoid this problem by requiring the user to explicitly indicate the partitioning of the sketch with button clicks or pauses in drawing. We have created a parser that automatically locates symbols by looking for areas of high ink density, and for points at which the characteristics of the pen strokes change. To demonstrate our techniques, we have developed AC-SPARC, a sketch-based interface for the SPICE electric circuit analysis program. An evaluation of our interface has indicated that, even for novice users, our system can successfully locate and identify most of the circuit components in hand-drawn circuit diagrams.

Generating multiple new designs from a sketch

Abstract We describe a program called Sketch IT that transforms a single sketch of a mechanical device into multiple families of new designs. It represents each of these families with a “BEP-Model”, a parametric model augmented with constraints that ensure the device produces the desired behavior. The program is based on qualitative configuration space (qc-space), a novel representation that captures mechanical behavior while abstracting away its implementation. The program employs a paradigm of abstraction and resynthesis: it abstracts the initial sketch into qc-space, then uses a library of primitive mechanical interactions to map from qc-space to new implementations.

RedesignIT—A Model-Based Tool for Managing Design Changes

RedesignIT is a computer program that uses model-based reasoning to generate and evaluate proposals of redesign plans for engineered devices. These proposals describe how the design parameters could be changed to achieve a specified performance goal. Equally important, the program proposes complementary modifications that may be necessary to counteract the undesirable side effects of the primary changes. RedesignIT is intended for use during the first stages of a redesign project, when engineers need to make a quick, yet accurate assessment of the overall effects of a particular design change. The program uses qualitative device models, which allow it to compute redesign plans efficiently. With its ability to predict the collateral, and probably undesirable, effects of a design change, the program is well suited to aid product designers in deciding on the feasibility of introducing design changes to a product.

Computerized planning for multiprobe cryosurgery using a force-field analogy.

Cryosurgery is the destruction of undesired biological tissues by freezing. For internal organs, multiple cryoprobes are inserted into the tissue with the goal of maximizing cryoinjury within a predefined target region, while minimizing cryoinjury to the surrounding tissues. The objective of this study is to develop a computerized planning tool to determine the best locations to insert the cryoprobes, based on bioheat transfer simulations. This tool is general and suitable for all available cooling techniques and hardware. The planning procedure employs a novel iterative optimization technique based on a force-field analogy. In each iteration, a single transient bioheat transfer simulation of the cryoprocedure is computed. At the end of the simulation, regions of tissue that would have undesired temperatures apply “forces” to the cryoprobes directly moving them to better locations. This method is more efficient than traditional numerical optimization techniques, because it requires significantly fewer bioheat transfer simulations for each iteration of planning. For demonstration purposes, 2D examples on cross sections typical of prostate cryosurgery are given.

Computerized Planning of Cryosurgery Using Cryoprobes and Cryoheaters

In a typical minimally invasive cryoprocedure, multiple cryoprobes are inserted into the tissue with the goal of maximizing cryoinjury within a predefined target region, while minimizing cryoinjury to the surrounding tissues. A temperature-controlled electrical heater has been developed recently by this research team, in order to assist in limiting the cryoinjury to the target region. The new device has been termed a ‘cryoheater,’ and it can work with any cryosurgical cooling technique. A prototype computerized planning tool has been presented recently by this research team, which helps to determine the best locations in which to insert the cryoprobes. This prototype was designed for cryoprobes only. The planning procedure utilized a novel iterative optimization technique, based on a force-field analogy. The combination of cryoheaters with computerized planning is the subject matter of this report. The current report includes a review of cryoheater development, and presents an improved cryosurgery planning tool which incorporates cryoheaters.

An efficient graph-based recognizer for hand-drawn symbols

We describe a trainable, multi-stroke symbol recognizer for pen-based user interfaces. The approach is insensitive to orientation, non-uniform scaling, and drawing order. Symbols are represented internally as attributed relational graphs describing both the geometry and topology of the symbols. Symbol definitions are statistical models, which makes the approach robust to variations common in hand-drawn shapes. Symbol recognition requires finding the definition symbol whose attributed relational graph best matches that of the unknown symbol. Much of the power of the approach derives from the particular set of attributes used, and our metrics for measuring similarity between graphs. One challenge addressed in the current work is how to perform the graph matching efficiently. We present five approximate matching techniques: stochastic matching, which is based on stochastic search; error-driven matching, which uses local matching errors to drive the solution to an optimal match; greedy matching, which uses greedy search; hybrid matching, which uses exhaustive search for small problems and stochastic matching for larger ones; and sort matching, which relies on geometric information to accelerate the matching. Finally, we present the results of a user study, and discuss the tradeoffs between the various matching techniques.

Qualitative rigid-body mechanics

We present a theory of qualitative rigid body mechanics and describe a program that uses this theory to compute qualitative dynamic simulations. The program works directly from a qualitative representation of geometry (qc-space). It employs a new qualitative representation for forces that reduces ambiguity in force sums and hence reduces branching.

Kirchhoff's Pen: a pen-based circuit analysis tutor

Kirchhoffs Pen is a pen-based tutoring system that teaches students to apply Kirchhoffs voltage law (KVL) and current law (KCL). To use the system, the student sketches a circuit schematic and annotates it to indicate component labels, mesh currents, and nodal voltages. The student then selects either mesh (KVL) or nodal (KCL) analysis, and writes the appropriate equations. The system interprets the equations, compares them to the correct equations (which are automatically derived from the circuit), and provides tutorial feedback about errors. Unlike traditional tutoring systems that work from input provided with a keyboard and mouse, our system works from ambiguous, hand-drawn input. The goal of our work is to create computational techniques to enable natural, pen-based tutoring systems that scaffold students in solving problems in the same way they would ordinarily solve them with paper and pencil. Kirchhoffs Pen is an important first step toward this goal.

Sim-U-Sketch: a sketch-based interface for SimuLink

Sim-U-Sketch is an experimental sketch-based interface we developed for Matlab®s Simulink® software package. With this tool, users can construct functional Simulink models simply by drawing sketches on a computer screen. To support iterative design, Sim-U-Sketch allows users to interact with their sketches in real time to modify existing objects and add new ones. The system is equipped with a domain-independent, trainable symbol recognizer that can learn new symbols from single prototype examples. This makes our system easily extensible and customizable to new domains and unique drawing styles.