Christoph Kofler

A classification scheme for user intentions in image search

Searching for images on the web is still an open problem. While multiple approaches have been presented, there has been surprisingly little work on the actual goals and intentions of users. In this poster we present our classification scheme for user goals in image search and describe our ongoing work focusing on identification and classification of user intentions during image search tasks.

Dynamic presentation adaptation based on user intent classification

Results of internet searches are typically presented as lists. When searching for digital photos different search result presentations however offer different benefits. If users are primarily interested in the visual content of images a thumbnail grid may be more appropriate than a list. For people searching photos taken at a specific place image metadata in the result presentation is of interest too. In this paper we present an application which monitors a users behavior while searching for digital photos and classifies the users intention. Based on the intention, the result is adapted to support the user in an optimal way.

User Intent in Multimedia Search: A Survey of the State of the Art and Future Challenges

Todays multimedia search engines are expected to respond to queries reflecting a wide variety of information needs from users with different goals. The topical dimension (“what” the user is searching for) of these information needs is well studied; however, the intent dimension (“why” the user is searching) has received relatively less attention. Specifically, intent is the “immediate reason, purpose, or goal” that motivates a user to query a search engine. We present a thorough survey of multimedia information retrieval research directed at the problem of enabling search engines to respond to user intent. The survey begins by defining intent, including a differentiation from related, often-confused concepts. It then presents the key conceptual models of search intent. The core is an overview of intent-aware approaches that operate at each stage of the multimedia search engine pipeline (i.e., indexing, query processing, ranking). We discuss intent in conventional text-based search wherever it provides insight into multimedia search intent or intent-aware approaches. Finally, we identify and discuss the most important future challenges for intent-aware multimedia search engines. Facing these challenges will allow multimedia information retrieval to recognize and respond to user intent and, as a result, fully satisfy the information needs of users.

Intent-Aware Video Search Result Optimization

Video search engines are relatively successful at returning search results that users find to be on topic. These results do not, however, completely satisfy the users information need unless they also fulfill the users intent, i.e., the immediate goal a user seeks to accomplish with video search. Satisfying a users information need to its full extent poses a particular challenge to video search engines because user intent is often not explicitly reflected in the query. In this paper, we propose a multimodal approach that addresses this challenge by refining the results lists returned by a mainstream video search engine in order to optimally capture user intent. Our approach is based on the insight that the results lists returned by video search engines do contain videos that satisfy users intent, but that videos with the highest potential for satisfaction are often buried within or scattered over the results list. The proposed approach consists of three steps. In the first step, it analyzes the initial results list to determine the intent distribution pattern. On the basis of this pattern, in the second step, it refines the video search results list such that the top of the list better reveals intent. The third step further improves this refinement by visual reranking, exploiting intent-sensitive lightweight visual features extracted from thumbnails. Extensive evaluation of the approach includes a user study carried out on a crowdsourcing platform and a system-oriented evaluation. Evaluation results demonstrate that our approach leads to a substantial improvement of the information need satisfaction at users.

Uploader Intent for Online Video: Typology, Inference, and Applications

We investigate automatic inference of uploader intent for online video, i.e., prediction of the reason for which a user has uploaded a particular video to the Internet. Users upload video for specific reasons, but rarely state these reasons explicitly in the video metadata. Information about the reasons motivating uploaders has the potential ultimately to benefit a wide range of application areas, including video production, video-based advertising , and video search. In this paper, we apply a combination of social-Web mining and crowdsourcing to arrive at a typology that characterizes the uploader intent of a broad range of videos. We then use a set of multimodal features, including visual semantic features, found to be indicative of uploader intent in order to classify videos automatically into uploader intent classes. We evaluate our approach on a dataset containing ca. 3K crowdsourcing-annotated videos and demonstrate its usefulness in prediction tasks relevant to common application areas.

SocialZap: Catch-up on Interesting Television Fragments Discovered from Social Media

In this paper we present SocialZap, a multimedia search engine that finds the most interesting fragments, zap points, in a television broadcast based on microblog posts and socially tagged photos. The main novelty of SocialZap is the fully-automatic transfer of the learned viewers interest from textual posts to the visual channel, without the need for any manual effort in the process. Once SocialZap finds the zap points, users can easily browse through a television broadcast and directly watch the interesting fragments. Thus, SocialZap adds social experience to watching television.

Predicting Failing Queries in Video Search

The ability to predict when a video search query is not likely to deliver satisfying search results is expected to enable more effective search results optimizations and improved search experience for users. In this paper, we propose a novel context-aware query failure prediction approach that predicts whether a particular query submitted in a users search session is likely to fail. The approach builds on the well-known concept of query performance prediction introduced in conventional text-based Web search to estimate the querys retrieval performance, but extends this concept with two novel characteristics, user indicators and engine indicators. User indicators are derived from transaction logs, capture the patterns of user interactions with the video search engine, and exploit the context in which a particular query was submitted. Engine indicators are derived from the search results list and measure the consistency of visual search results at the level of visual concepts and textual metadata associated with videos. Extensive evaluation of the approach on a test set containing over one million video search queries shows its effectiveness and demonstrates a significant improvement over traditional and state-of-the-art baseline approaches.

A peer's-eye view: network term clouds in a peer-to-peer system

We investigate term clouds that represent the content available in a peer-to-peer (P2P) network. Such network term clouds are non-trivial to generate in distributed settings. Our term cloud generator was implemented and released in Tribler--a widely-used, server-free P2P system--to support users in understanding the sorts of content available. Our evaluation and analysis focuses on three aspects of the clouds: coverage, usefulness and accumulation speed. A live experiment demonstrates that individual peers accumulate substantial network-level information, indicating good coverage of the overall content of the system. The results of a user study carried out on a crowdsourcing platform confirm the usefulness of clouds, showing that they succeed in conveying to users information on the type of content available in the network. An analysis of five example peers reveals that accumulation speeds of terms at new peers can support the development of a semantically diverse term set quickly after a cold start. This work represents the first investigation of term clouds in a live, 100% server-free P2P setting.