Darryl Greig

Video object detection speedup using staggered sampling

This paper presents an enhancement of the standard sampling strategy for filter-based object detection and tracking in video streams. The proposed method, called staggered sampling, seeks to maximize the sampling density across video frames, thus reducing the number of patches sampled while retaining proportionally high recall rates. The method can be tailored to virtually any constraint on resources and may be used in conjunction with any filter-based object detector / tracking algorithm combination. We test our method using a modified version of the face detector in the OpenCV library and a simple tracking algorithm. The resulting detector was applied to some video sequences from the QCIF collection. Our results show that staggered sampling can achieve around 90% of the recall of full (dense) sampling while only evaluating the detector on around 10% of the image locations. At the same time the precision of the detector increases. The staggered sampling approach therefore addresses the problem of acquiring new objects in an object tracking framework by enabling a low-cost background scan of the video stream to run continuously. The simplicity and robustness of this approach make it an excellent enhancement to existing video object detection methods.

Learnable Stroke Models for Example-based Portrait Painting.

We present a novel algorithm for stylizing photographs into portrait paintings comprised of curved brush strokes. Rather than drawing upon a prescribed set of heuristics to place strokes, our system learns a flexible model of artistic style by analyzing training data from a human artist. Given a training pair — a source image and painting of that image — a non-parametric model of style is learned by observing the geometry and tone of brush strokes local to image features. A Markov Random Field (MRF) enforces spatial coherence of style parameters. Style models local to facial features are learned using a semantic segmentation of the input face image, driven by a combination of an Active Shape Model and Graph-cut. We evaluate style transfer between a variety of training and test images, demonstrating a wide gamut of learned brush and shading styles.

Comprehensive solutions for automatic removal of dust and scratches from images

Dust, scratches, or hair on originals (prints, slides, or negatives) distinctly appear as light or dark artifacts on a scan. These unsightly artifacts have become a major consumer concern. There are several scenarios for removal of dust and scratch artifacts. One scenario is during acquisition, e.g., while scanning photographic media. Another is artifact removal from a digital image in an image editor. For each scenario, a different solution is suitable, with different performance requirements and differing levels of user interaction. This work describes a comprehensive set of algorithms for automatically removing dust and scratches from images. Our algorithms solve a wide range of use scenarios. A dust and scratch removal solution has two steps: a detection step and a reconstruction step. Very good detection of dust and scratches is possible using side information, such as provided by dedicated hardware. Without hardware assistance, dust and scratch removal algorithms generally resort to blurring, thereby losing image detail. We present algorithmic alternatives for dust and scratch detection. In addition, we present reconstruction algorithms that preserve image detail better than previously available alternatives. These algorithms consistently produce visually pleasing images in extensive testing.

Web-based magazine design for self publishers

Short run printing technology and web services such as MagCloud provide new opportunities for long-tail magazine publishing. They enable self publishers to supply magazines to a wide range of communities, including groups that are too small to be viable as target communities for conventional publishers. In a Web 2.0 world where users constantly discover new services and where they may be infrequent patrons of any single service, it is unreasonable to expect users to learn the complex service behaviors. Furthermore, we want to open up publishing opportunities to novices who are unlikely to have prior experience of publishing and who lack design expertise. Magazine design automation is an ambitious goal, but recent progress with another web service, Autophotobook, proves that some level of automation of publication design is feasible. This paper describes our current research effort to extend the automation capabilities of Autophotobook to address the issues of magazine design so that we can provide a service to support professional-quality self publishing by novice users for a wide range of community types and sizes.

Biblio: automatic meta-data extraction

Biblio is an adaptive system that automatically extracts meta-data from semi-structured and structured scanned documents. Instead of using hand-coded templates or other methods manually customized for each given document format, it uses example-based machine learning to adapt to customer-defined document and meta-data types. We provide results from experiments on the recognition of document information in two document corpuses: a set of scanned journal articles and a set of scanned legal documents. The first set is semi-structured, as the different journals use a variety of flexible layouts. The second set is largely free-form text based on poor quality scans of FAX-quality legal documents. We demonstrate accuracy on the semi-structured document set roughly comparable to hand-coded systems, and much worse performance on the legal documents.

Layout hierarchies for interactive design reuse

The advent of viable long tail & self-publishing solutions ([1], [2]) has spawned new requirements for automatic layout technologies. In most cases these attempt to lay out whole pages, spreads or documents based on complete content data. In this paper we introduce a new approach to document layout based on the principle of interactive design reuse, in which a new design is created from an existing high quality design via a sequence of simple steps to establish the final content. Based on our experience building such a system we propose a method of building layout hierarchies and discuss the implementation of editing operations appropriate to this new paradigm.

Faces from the web: automatic selection and composition of media for casual screen consumption and printed artwork

Web image search engines facilitate the production of image sets in which faces appear. Many people enjoy producing and sharing media collections of this type and generating new images or video experiences. Skilled practitioners produce visually appealing artifacts from such collections but few users have the time or creative ability to do so. The problem is to automatically create an image or ambient experience which sustains interest. A full solution requires agreements with copyright holders and input from graphics designers. We address the underlying technical problems of extraction and composition. We describe an automatic system that identifies regions containing human faces in each image of an image set resulting from a web search. The face regions are composed into dynamically synthesized multilayer graphical backgrounds. The aesthetic aspects of the composition are controlled by active templates. These aspects include face size and positioning but also face identity and number of faces in a group. The output structure is multi layer supporting both the generation of static images and video consisting of transitions between the compositions.

Automatic Page Composition with Combined Image Crop and Layout Metrics

Automatic layout algorithms simplify the composition of image-rich documents, but they still require users to have sufficient artistry to supply well cropped and composed imagery. Combining an automatic cropping technology with a document layout system enables better results to be produced faster by less-skilled users. This paper reviews prior work in automatic image cropping and automatic page layout and presents a case for a combined crop and layout technology. We describe one such technology in a system for interactive publication design by amateur self-publishers and show that providing an automatic cropping system with additional information about the layout context can enable it to generate a more appropriate set of ranked crop options for a given image. Furthermore, we show that providing an automatic layout system with sets of ranked crop options for images can enable it to compose more appropriate page layouts.