Jorge Moraleda

Toward Massive Scalability in Image Matching

A method for image matching from partial blurry images is presented that leverages existing text retrieval algorithms to provide a solution that scales to hundreds of thousands of images. As an initial application, we present a document image matching system in which the user supplies a query image of a small patch of a paper document taken with a cell phone camera, and the system returns a label identifying the original electronic document if found in a previously indexed collection. Experimental results show that a retrieval rate of over 70% is achieved on a collection of nearly 500,000 document pages.

Mobile image recognition: architectures and tradeoffs

We argue that the most desirable architecture for mobile image recognition runs the complete algorithm on the mobile device. Alternative solutions that run the recognizer on a remote server will not be as desirable because of the delay between image capture and receipt of a result that can cause users to abandon the technique. We present a method for mobile recognition of paper documents and an application to newspapers that lets readers retrieve electronic data linked to articles, photos, and advertisements. We show that the index for a reasonable collection of daily newspapers can be downloaded in less than a minute and will fit in the memory of todays mid-range smart phones. Experimental results show that the recognition system has an overall error rate of less than 1%. We achieved a run time of 1.2 secs. per image with a collection of 140 newspaper pages on an HTC-8282 Windows Mobile phone.

Gregory Shakhnarovich, Trevor Darrell and Piotr Indyk: Nearest-Neighbors Methods in Learning and Vision. Theory and Practice: 0-262-19547-X, MIT Press, October 13, 2006, 260

Nearest neighbor (NN) methods, that is regression and classification methods based on similarity of the input to stored examples, have been known and used for decades. The premise of ‘‘Nearest-Neighbors Methods in Learning and Vision’’ is that traditional data structures however do not scale well to high dimensional problems and thus nearest neighbor methods are seldom employed for high dimensional data sets. The book presents a suite of recent results and techniques aimed to extending the range of problems for which nearest neighbor methods are tractable. ‘‘Nearest-Neighbors Methods in Learning and Vision’’, edited by G. Shakhnarovich, T. Darrell, and P. Indyk, is written clearly and the quality of the material exposed is generally high, with a few exceptions. Overall this book will be most useful to the computer scientist interested in machine learning applications. Each chapter is self contained and both algorithms and techniques are described in sufficient detail to be implemented without need for resorting to the references. Nonetheless, these are sufficient for readers interested in expanding beyond the material presented. The discussions in several chapters on how to choose and fine-tune the free parameters of the algorithms are welcome. It is well known that these choices can have a large impact in performance, and I expect these discussions will greatly improve the adoption of the methods presented. ‘‘Nearest-Neighbors Methods in Learning and Vision’’ is well balanced between algorithms and applications. While all applications described are in the field of vision as suggested by the title, most address general machine learning problems: classification, clustering, non-parametric regression, and it should not be difficult for practitioners in other fields to adapt them. A unifying theme throughout the book is that the dimension of the data space is crucial to the computational complexity of nearest neighbor searches. It is well known that nearest neighbor search in uniform high-dimensional spaces is a hard problem. This phenomenon is often referred to as the curse of dimensionality. The authors remark however that often the intrinsic dimensionality of the data is much lower than that of the space it is embedded in, explaining the promising empirical results obtained. The majority of the book is devoted to approximate methods not guaranteed to return the exact ‘‘nearest’’ neighbor, but a ‘‘near’’ neighbor instead. It presents results and examples of how approximate methods can be orders of magnitude faster than their exact counterparts with very small degradation of performance, which can sometimes be statistically quantified. In particular, recent algorithms for and applications of Locality Sensitive Hashing (LSH) are explored in several chapters. The idea behind LSH is to hash data points such that points that are near each other hash to the same bin with higher probability than points that are far from each other. Multiple independent hashes with this property are constructed to ensure that with high probability the nearestneighbor to any query point will be hashed in the same bin as the query in at least one of the hashes. Chapter 3, by A. Andoni, M. Datar, N. Immorlika, P. Indyk, and V. Mirrokni, presents a novel mechanism for obtaining such hashes, with statistical guarantees when the data has been drawn from a stable distribution (such as the normal distribution). The practitioner will enjoy detailed procedures for determining the parameters of the LSH scheme as J. Moraleda (&) Ricoh Innovations, 2882 Sand Hill Road Suite 115, Menlo Park, CA 94025-7022, USA e-mail: moraleda@rii.ricoh.com

Ground-Truthed Video Generation from Symbolic Information

An algorithm is presented that automatically generates ground-truthed video from a symbolic description for an object and a specification for the movement of a handheld video camera around that object. This provides a method to generate large amounts of training and test data for the development of computer vision algorithms. We describe an implementation of this technique for an imaging application in which a cell phone video camera is moved over a paper document. Experimental results demonstrate the similarity of images captured by the real camera to images generated by the proposed technique.

Large scalability in document image matching using text retrieval

Highlights? We cast image matching as text retrieval. ? We have indexed more than 500,000 images. ? We provide recognition in approximately 500ms. We present a method that addresses image matching from partial blurry images by casting it as a problem of text retrieval. This allows us to leverage existing text document retrieval techniques and achieve efficiency and scalability similar to text search applications.As an initial application, we present a document image matching system in which the user supplies a query image of a small patch of a paper document taken with a cell phone camera, and the system returns a label identifying the original electronic document if found in a previously indexed collection. We have implemented our method in a client server architecture. Feature computation on a mobile client is done in under 100ms, while end-to-end document recognition on a collection of more than 4300 pages requires approximately 500ms per image. Approximately 170ms is connection time and thus subject to network speed variations. We conclude presenting scalability results on a collection of nearly 500,000 documents.

Visual search applications for connecting published works to digital material

Visual search connects physical (offline) objects with (online) digital media. Using objects from the environment, like newspapers, magazines, books and posters, we can retrieve supplemental information from the online world. In this demonstration, we show a framework for delivering visual search services to users of mobile devices. We show how users can point a mobile device at any location in a document, magazine or book to view related, online material on the device. We describe client applications now being deployed for the iPhone and the server architecture used for recognition of scanned images.

Lake Wobegon Dice.

Summary We introduce Lake Wobegon dice, where each die is “better than the set average.” Specifically, these dice have the paradoxical property that on every roll, each die is more likely to roll greater than the set average on the roll, than less than this set average. We also show how to construct minimal optimal Lake Wobegon sets for all n ≥ 3.