Sashi Raghupathy

Modeling and Rendering for Realistic Facial Animation

Rendering realistic faces and facial expressions requires good models for the reflectance of skin and the motion of the face. We describe a system for modeling, animating, and rendering a face using measured data for geometry, motion, and reflectance, which realistically reproduces the appearance of a particular person’s face and facial expressions. Because we build a complete model that includes geometry and bidirectional reflectance, the face can be rendered under any illumination and viewing conditions. Our face modeling system creates structured face models with correspondences across different faces, which provide a foundation for a variety of facial animation operations.

Discerning structure from freeform handwritten notes

This paper presents an integrated approach to parsing textual structure in freeform handwritten notes. Text-graphics classification and text layout analysis are classical problems in printed document analysis, but the irregularity in handwriting and content in freeform notes reveals limitations in existing approaches. We advocate an integrated technique that solves the layout analysis and classification problems simultaneously: the problems are so tightly coupled that it is not possible to solve one without the other for real user notes. We tune and evaluate our approach on a large corpus of unscripted user files and reflect on the difficult recognition scenarios that we have encountered in practice.

Parsing ink annotations on heterogeneous documents

Annotation is an integral part of reading, comprehending, commenting, and authoring notes and documents. In this paper we present a system for recognizing annotations in a flexible digital notebook that may contain a variety of content ranging from text, to images, to handwritten notes. To accomplish the recognition task in real-time makes the complicated annotation parsing problem more difficult. Our approach differs from previous approaches in several ways. First, our approach handles annotations on ink notes, which are significantly more ambiguous than annotations on printed documents and hence more difficult to recognize. Second, our approach is entirely learned from data, so it is easy to adapt to other scenarios. Third, our approach is more thoroughly evaluated than previous systems. On a test set of real user notes, the system has achieved an average recall of 0.9258 on all annotation types. Finally, the implementation of the approach will be commercially available as an API in the upcoming release of Windows® Vista® and Office 12®.

Learning to Group Text Lines and Regions in Freeform Handwritten Notes

This paper proposes a machine learning approach to grouping problems in ink parsing. Starting from an initial segmentation, hypotheses are generated by perturbing local configurations and processed in a high-confidence-first fashion, where the confidence of each hypothesis is produced by a data-driven AdaBoost decision-tree classifier with a set of intuitive features. This framework has successfully applied to grouping text lines and regions in complex freeform digital ink notes from real TabletPC users. It holds great potential in solving many other grouping problems in the ink parsing and document image analysis domains.

An empirical study in pen-centric user interfaces: diagramming

We present a user study aimed at helping understand the applicability of pen-computing in desktop environments. The study applied three mouse-and-keyboard-based and three pen-based interaction techniques to six variations of a diagramming task. We ran 18 subjects from a general population and the key finding was that while the mouse and keyboard techniques generally were comparable or faster than the pen techniques, subjects ranked pen techniques higher and enjoyed them more. Our contribution is the results from a formal user study that suggests there is a broader applicability and subjective preference for pen user interfaces than the niche PDA and mobile market they currently serve.

Ink Annotations and their Anchoring in Heterogeneous Digital Documents

The analysis, especially the anchoring, of free-form ink annotations on mixed-content digital documents is more difficult than that of documents containing only text. In this paper, first, we present our work on analyzing the common user scenarios of ink annotations and their anchoring in such an environment, and we also present a unified way to define and analyze the anchoring of different types of annotations. Then we describe how our ink parsing system integrates the anchoring of and annotation with the recognition of its type. The method can be used in a real-time heterogeneous document analysis system.