Mark D. Wood

A software system for automatic albuming of consumer pictures

The wide-spread use of image capturing and scanning devices such as digital cameras and low-cost scanners is rapidly resulting in the digital equivalent of the overstuffed shoebox full of snapshots. This paper describes a software system for automatic albuming of pictures to facilitate the creation of consumer albums and album pages. This software system provides consumers an efficient way of organizing and albuming their pictures digitized born different sources. The system includes advanced features such as image event clustering, dud detection, and duplicate detection necessary for automated albuming. The software architecture is based on a simple database structure that can efficiently handle metadata information. The system includes a substantial user interface component that enables the user to preview and edit the generated album.

Multidimensional image value assessment and rating for automated albuming and retrieval

The ability to automatically assess image characteristics is an important function for content management, building digital image albums, storytelling with images, and retrieval of specific visual content. This capability is needed to organize and sort large numbers of image and video assets. This paper proposes a novel approach to assess and rate images based on multidimensional characteristics including image quality, social relationships, aesthetic quality, important events, and usage. This new approach provides additional flexibility for end user applications that utilize different aspects of image characteristics. Specifically, we describe a method for assessing image quality based upon technical characteristics of the image, and for predicting the significance of an image based upon the people portrayed in the image. Experimental results indicate that the proposed multidimensional approach provides a promising framework for automated image value assessment and rating.

CPXe: web services for Internet imaging

The Common Picture eXchange environment leverages the Web services paradigm to serve the electronic photographic services market, combining open standards for exchanging digital images, orders, and other information with an online directory of service providers.

Exploiting Semantics for Personalized Story Creation

The task of creating albums or multimedia output from consumer content is becoming increasingly difficult as the amount of content grows. This work presents a system for using semantic information to automate the process of selecting and combining digital assets into summary presentations or storylines, as well as determining triggers for when to generate such content. The system obtains semantic information from a variety of sources, including the capture metadata, image and video understanding algorithms, user profiles and third party ontologies; all such semantic information is stored in a triple store. Prolog-based rules leverage the triple store to provide a knowledgebase for determining when to create particular types of output and how to select assets for such output. This knowledgebase greatly simplifies the task of creating consumer-grade multimedia content.

Matching Songs to Events in Image Collections

This work identifies relevant songs from a user’s personal music collection to accompany pictures of an event. The event’s pictures are analyzed to extract aggregated semantic concepts in a variety of dimensions, including scene type, geospatial information, and event type, along with user-provided keywords. These semantic concepts are then used to form a search query against a song database based primarily on the song lyrics. Songs are scored using probabilistic techniques to come up with a rank ordered list of candidate songs that could then be used as, e.g., the audio track in a multimedia slideshow.

Searching consumer image collections using web-based concept expansion

As consumers accumulate more and more personal imagery, searching for specific images has become increasingly difficult. Consumers typically provide little or no annotations, and automated classifiers and concept tagging tools are limited in their scope and vocabulary. This work addresses this sparsity of semantic information by leveraging domain-specific information provided by online photo-sharing communities. Such information enables improved search by allowing user-provided search terms to be expanded into a set of semantically related concepts, using relevant semantic relationships provided by millions of users. Our system first extracts metadata using a modest number of image and event-based semantic classifiers, as well as any meaningful file or folder names. When users pose text-based queries, our system retrieves images from their personal image collections by leveraging Flickrs tag dataset for concept expansion. This approach enables users to search their collections without having to manually annotate their pictures. We compare the retrieval performance of using a Flickr-based concept expander with the performance obtained without concept expansion and with using a WordNet-based concept expander. The results demonstrate that common sense knowledge gleaned from online photo sharing communities can enable meaningful image search on consumer image collections, searches that would be impossible using only the available image metadata.

Exploring Photos in Facebook

Facebook has rapidly become for many the dominant means for sharing images, and the number of shared images accessible to any given Facebook user is easily in the tens of thousands. The sheer volume of pictures relegates most to obscurity, yet some of those pictures would be of great interest -- if a person could only find them. This research explores ways to harness latent semantic information associated with pictures and interpersonal relationships to enable a person to browse for potentially interesting and germane images shared by people in their social network. The possibilities for semantic analysis are endless, this work illustrates two possible approaches while also highlighting future potential applications of semantic understanding.

Event-centric View of Consumer Image Collections

Benchmark and ground truth databases are critical for evaluating imaging- and event-based algorithms, but such databases can be time consuming to create, cannot be representative of all real-life consumer image collections, and do not necessarily provide feedback on the general utility of an imperfect algorithm applied to real-life data. An alternative solution is needed to complement evaluation by benchmark and ground truth databases. We designed and developed Event Analyzer as a tool to meet this need. Event Analyzer combines data visualization of metadata with event and image retrieval results in a way that enables researchers to easily review data trends, quickly filter and select data, see relationships between metadata values, and visually scan retrieved results to determine correctness. In this paper, we describe the tool and discuss how it can and has been used to analyze real-life consumer collections in an event-centric manner.

Semantics Meets UX: Mediating Intelligent Indexing of Consumers' Multimedia Collections for Multifaceted Visualization and Media Creation

Unorganized media collections hinder consumers from fully experiencing and enjoying their visual media. User interfaces can mediate the results of automated indexing by presenting data and interactions that leverage the strengths of individual and combined algorithm results, supporting multifaceted browsing, and enabling user correction in a way that is not disruptive to the userspsila activities. We describe the semantic system demonstration framework (SSDF), a flexible and extensible framework for combining multiple semantic indexing algorithms for consumer photo and video clip collections into one integrated system. We also describe key features of Koi, an SSDF desktop client application with a user interface designed to mediate and leverage the intelligent indexing incorporated in the SSDF server. Together, Koi and SSDF empower users to experience their personal multimedia in novel and sophisticated ways.