Yiqun Li

Scene Recognition with Camera Phones for Tourist Information Access

Camera phones present new opportunities and challenges for mobile information association and retrieval. The visual input in the real environment is a new and rich interaction modality between a mobile user and vast information base connected to a users device via rapidly advancing communication infrastructure. We have developed a system for tourist information access to provide scene description based on an image taken of the scene. In this paper, we describe the working system, the STOIC 101 database, and a new pattern discovery algorithm to learn image patches that are recurrent within a scene class and discriminative across others. We report preliminary scene recognition results on 90 scenes, trained on 5 images per scene, with an accuracy of 92% and 88% on a test set of 110 images, with and without location priming.

Outdoor place recognition using compact local descriptors and multiple queries with user verification

In this paper, we propose a novel method to model outdoor places with compact local descriptors extracted from images taken around geographical places. Region-based and clustering-based methods are used to reduce the number of feature vectors to represent the natural scene images. A Multiple Queries with User Verification (MQUV) scheme is proposed to improve the recognition accuracy and the system reliability. In our application, a mobile phone camera is used to take images around a place and send them back to the server to get relevant information about the place. The MQUV scheme calculates the maximum confidence level of all top 5 matching places and returns the best matching result to the user together with a typical sample image of the recognized place for the users visual verification. User is suggested to take more images if the system is not confident enough to provide a result. The user can also make ones own decision by visually matching the returned image with the scenery of the place. Experimental results show that the number of feature vectors is significantly reduced with the compact place modeling and the recognition accuracy is improved with the MQUV scheme.

Cascaded classification with optimal candidate selection for effective place recognition

A two-stage cascaded classification approach with an optimal candidate selection scheme is proposed to recognize places using images taken by camera phones. An optimal acceptance threshold is chosen to maximize the probability of accepting more positives and rejecting more negatives at the first stage so that an optimal number of candidates are selected. The first classifier is trained using simple color and texture features. The second classifier is trained by scale invariant feature transform (SIFT). For a query image, a number of matching candidates are selected using k nearest neighbor at the first stage and passed on to the second stage for a refining classification to select the best matching result. The searching range is narrowed down dynamically at the second stage depending on the output of the first stage. Experimental results show that this method is promising by improving the recognition accuracy and reducing the computation time.

Advertisement image recognition for a location-based reminder system

In this paper, we propose a location-based reminder system on mobile phones using image recognition technology. With this system, mobile phone users can actively capture images from their favorite product or event promotional materials. Upon recognizing the image sent to a computer server, location-based reminders will be downloaded to the phone. The mobile phone will alert the user when he/she is close to the place where the product is being sold or the event is happening. Nearduplicate image recognition is employed to identify the advertisement. Using scale-invariant features followed by Kd-tree image matching and geometric validation, near-duplicates of trained images in the database are recognized. The image recognition provides accurate and efficient retrieval of the corresponding reminders. A mobile client application is developed to capture images, conduct GPS location tracking and to pop up reminders.

Showroom introduction using mobile phone based on scene image recognition

In this paper, image recognition is used to link a picture to relevant information. As mobile camera phone becoming more and more popular, applications for these phones will bring value to the phone and convenience to the users. We develop a system to perform audio introduction for our technology showroom using images taken by the built-in camera of mobile phone. Visitors to the showroom can take a picture of a poster, a physical prototype or any materials using their mobile phone. An audio introduction in his favorite language will be delivered to the visitor. A few schemes are used to improve the accuracy and system reliability, which include discriminative feature selection based on feature distance; repeating queries if result is not correct; users verification on delivered content; employment of multiple classifiers. Experimental results show that the system performance is promising.

A platform for mobile augmented reality app creation without programming

There are many application areas on using smartphone to access relevant information. By taking a picture from a physical object using a smartphone app, we use image recognition technology to provide a quick link between the physical object and its relevant information. However, developing smartphone apps is expensive and time consuming. We developed a platform called MIMAS AR Creator, which is a web based software platform for automatic creation of smartphone apps for multimedia access using pictures captured from the smartphone camera and its GPS location. This platform allows people without programming skill to create a smartphone app in a few minutes with existing multimedia contents, shorten more than 90% of the app development time. The digital contents can be web pages, videos, audios, images, or 3D graphics with or without animation etc. The platform can be used for mobile advertising and retail marketing, mobile learning and tour guide etc. For example, with the created app running, people can point their phone camera to a picture on newspaper, product brochure, or physical product to obtain more relevant information provided by the advertisers or vendors. They can also point the phone camera to a building or monument to retrieve relevant historical information.

An image-based outdoor place recognition and information retrieval system

In image-based place recognition, an image is used to deduce the location of the viewer during image acquisition. The identified place can subsequently be used to provide additional information to the user. This demonstration paper briefly describes a system that performs image-based place recognition on outdoor images, made possible by viewer-centric data sampling and local feature-based scene identification. It also explains our proposed demonstration that displays the recognized place on a map and provides information of amenities in its vicinity.