Michele Trevisiol

Scalable triangulation-based logo recognition

We propose a scalable logo recognition approach that extends the common bag-of-words model and incorporates local geometry in the indexing process. Given a query image and a large logo database, the goal is to recognize the logo contained in the query, if any. We locally group features in triples using multi-scale Delaunay triangulation and represent triangles by signatures capturing both visual appearance and local geometry. Each class is represented by the union of such signatures over all instances in the class. We see large scale recognition as a sub-linear search problem where signatures of the query image are looked up in an inverted index structure of the class models. We evaluate our approach on a large-scale logo recognition dataset with more than four thousand classes.

6 Seconds of Sound and Vision: Creativity in Micro-videos

The notion of creativity, as opposed to related concepts such as beauty or interestingness, has not been studied from the perspective of automatic analysis of multimedia content. Meanwhile, short online videos shared on social media platforms, or micro-videos, have arisen as a new medium for creative expression. In this paper we study creative micro-videos in an effort to understand the features that make a video creative, and to address the problem of automatic detection of creative content. Defining creative videos as those that are novel and have aesthetic value, we conduct a crowdsourcing experiment to create a dataset of over 3, 800 micro-videos labelled as creative and non-creative. We propose a set of computational features that we map to the components of our definition of creativity, and conduct an analysis to determine which of these features correlate most with creative video. Finally, we evaluate a supervised approach to automatically detect creative video, with promising results, showing that it is necessary to model both aesthetic value and novelty to achieve optimal classification accuracy.

Retrieving geo-location of videos with a divide & conquer hierarchical multimodal approach

This paper presents a strategy to identify the geographic location of videos. First, it relies on a multi-modal cascade pipeline that exploits the available sources of information, namely the users upload history, his social network and a visual-based matching technique. Second, we present a novel divide & conquer strategy to better exploit the tags associated with the input video. It pre-selects one or several geographic area of interest of higher expected relevance and performs a deeper analysis inside the selected area(s) to return the coordinates most likely to be related to the input tags. The experiments were conducted as part of the MediaEval 2012 Placing Task. Our approach, which differs significantly from the other submitted techniques, achieves the best results on this benchmark when considering the same amount of external information, i.e. when not using any gazetteers nor any other kind of external information.

Cold-start news recommendation with domain-dependent browse graph

Online social networks and mash-up services create opportunities to connect different web services otherwise isolated. Specifically in the case of news, users are very much exposed to news articles while performing other activities, such as social networking or web searching. Browsing behavior aimed at the consumption of news, especially in relation to the visits coming from other domains, has been mainly overlooked in previous work. To address that, we build a BrowseGraph out of the collective browsing traces extracted from a large viewlog of Yahoo News (0.5B entries), and we define the ReferrerGraph as its subgraph induced by the sessions with the same referrer domain. The structural and temporal properties of the graph show that browsing behavior in news is highly dependent on the referrer URL of the session, in terms of type of content consumed and time of consumption. We build on this observation and propose a news recommender that addresses the cold-start problem: given a user landing on a page of the site for the first time, we aim to predict the page she will visit next. We compare 24 flavors of recommenders belonging to the families of content-based, popularity-based, and browsing-based models. We show that the browsing-based recommender that takes into account the referrer URL is the best performing, achieving a prediction accuracy of 48% in conditions of heavy data sparsity.

Buon appetito: recommending personalized menus

This paper deals with the problem of menu recommendation, namely recommending menus that a person is likely to consume at a particular restaurant. We mine restaurant reviews to extract food words, we use sentiment analysis applied to each sentence in order to compute the individual food preferences. Then we extract frequent combination of dishes using a variation of the Apriori algorithm. Finally, we propose several recommender systems to provide suggestions of food items or entire menus, i.e. sets of dishes.

Analyzing Favorite Behavior in Flickr

Liking or marking an object, event, or resource as a favorite is one of the most pervasive actions in social media. This particular action plays an important role in platforms in which a lot of content is shared. In this paper we take a large sample of users in Flickr and analyze logs of their favorite actions considering factors such as time period, type of connection with the owner of the photo, and other aspects. The objective of our work is, on one hand to gain insights into the “liking” behavior in social media, and on the other hand, to inform strategies for recommending items users may like. We place particular focus on analyzing the relationship between recent photos uploaded by user’s connections and the favorite action, noting that a direct application of our work would lead to algorithms for recommending users a subset of these “recently uploaded” photos that they might favorite. We compare several features derived from our analysis, in terms of how effective they might be in retrieving favorite photographs.

The Benchmark as a Research Catalyst: Charting the Progress of Geo-prediction for Social Multimedia

Benchmarks have the power to bring research communities together to focus on specific research challenges. They drive research forward by making it easier to systematically compare and contrast new solutions, and evaluate their performance with respect to the existing state of the art. In this chapter, we present a retrospective on the Placing Task, a yearly challenge offered by the MediaEval Multimedia Benchmark. The Placing Task, launched in 2010, is a benchmarking task that requires participants to develop algorithms that automatically predict the geolocation of social multimedia (videos and images). This chapter covers the editions of the Placing Task offered in 2010–2013, and also presents an outlook onto 2014. We present the formulation of the task and the task dataset for each year, tracing the design decisions that were made by the organizers, and how each year built on the previous year. Finally, we provide a summary of future directions and challenges for multimodal geolocation, and concluding remarks on how benchmarking has catalyzed research progress in the research area of geolocation prediction for social multimedia.

Exploiting Implicit User Activity for Media Recommendation

This thesis explores in depth how to exploit the user browsing behavior, and in particular the referrer URL, to understand the interest of the users. The aim is, first, to understand the preferences of the users from their navigation patterns, i.e., from the implicit actions of the users. Then, to exploit this information to personalize the content offered by the service provider. The key findings from our studies allowed us to propose innovative solutions to perform recommendation and ranking of media content. We show how the browsing logs are extremely meaningful also for cold-start problem -- estimating the preferences of newcomers.

Georeferencing Flickr Resources Based on Multimodal Features

The popularity of social media, and location-based services in particular, has led to a vast increase in the number of georeferenced resources on the web.