José Luis Redondo García

What fresh media are you looking for?: retrieving media items from multiple social networks

Social networks play an increasingly important role for sharing media items related to daily life moments or for the live coverage of events. One of the problems is that media are spread over multiple social networks. In this paper, we propose a social network-agnostic approach for collecting recent images and videos which can be potentially attached to an event. These media items can be used for the automatic generation of visual summaries in the form of media galleries. Our approach includes the alignment of the varying search result formats of different social networks, while putting media items in correspondence with the status updates and stories they are related to. More precisely we leverage on: (i) visual features from media items, (ii) textual features from status updates, and (iii) social features from social networks to interpret, deduplicate, cluster, and visualize media items. We address the technical details of media item extraction and media item processing, discuss criteria for media item filtering and envision several visualization options for media presentation. Our evaluation is divided into two parts: first we assess the performances of the image process deduplication and then we propose a human evaluation of the summary creation compared with Teleportd and Twitter media galleries. A demo of our approach is publicly available at http://eventmedia.eurecom.fr/media-finder.

Enriching media fragments with named entities for video classification

With the steady increase of videos published on media sharing platforms such as Dailymotion and YouTube, more and more efforts are spent to automatically annotate and organize these videos. In this paper, we propose a framework for classifying video items using both textual features such as named entities extracted from subtitles, and temporal features such as the duration of the media fragments where particular entities are spotted. We implement four automatic machine learning algorithms for multiclass classification problems, namely Logistic Regression (LG), K-Nearest Neighbour (KNN), Naive Bayes (NB) and Support Vector Machine (SVM). We study the temporal distribution patterns of named entities extracted from 805 Dailymotion videos. The results show that the best performance using the entity distribution is obtained with KNN (overall accuracy of 46.58%) while the best performance using the temporal distribution of named entities for each type is obtained with SVM (overall accuracy of 43.60%). We conclude that this approach is promising for automatically classifying online videos.

Automatic fine-grained hyperlinking of videos within a closed collection using scene segmentation

This paper introduces a framework for establishing links between related media fragments within a collection of videos. A set of analysis techniques is applied for extracting information from different types of data. Visual-based shot and scene segmentation is performed for defining media fragments at different granularity levels, while visual cues are detected from keyframes of the video via concept detection and optical character recognition (OCR). Keyword extraction is applied on textual data such as the output of OCR, subtitles and metadata. This set of results is used for the automatic identification and linking of related media fragments. The proposed framework exhibited competitive performance in the Video Hyperlinking sub-task of MediaEval 2013, indicating that video scene segmentation can provide more meaningful segments, compared to other decomposition methods, for hyperlinking purposes.

Live topic generation from event streams

Social platforms constantly record streams of heterogeneous data about humans activities, feelings, emotions and conversations opening a window to the world in real-time. Trends can be computed but making sense out of them is an extremely challenging task due to the heterogeneity of the data and its dynamics making often short-lived phenomena. We develop a framework which collects microposts shared on social platforms that contain media items as a result of a query, for example a trending event. It automatically creates different visual storyboards that reflect what users have shared about this particular event. More precisely it leverages on: (i) visual features from media items for near-deduplication, and (ii) textual features from status updates to interpret, cluster, and visualize media items. A screencast showing an example of these functionalities is published at: http://youtu.be/8iRiwz7cDYY while the prototype is publicly available at http://mediafinder.eurecom.fr.

Tracking and Analyzing The 2013 Italian Election

Social platforms open a window to what is happening in the world in near real-time: (micro-)posts and media items are shared by people to report their feelings and their activities related to any type of events. Such an information can be collected and analyzed in order to get the big picture of an event from the crowd point of view. In this paper, we present a general framework to capture and analyze micro-posts containing media items relevant to a search term. We describe the results of an experiment that consists in collecting fresh social media posts (posts containing media items) from numerous social platforms in order to generate the story of the “2013 Italian Election”. Items are grouped in meaningful time intervals that are further analyzed through deduplication, clusterization, and visual representation. The final output is a storyboard that provides a satirical summary of the elections as perceived by the crowd. A screencast showing an example of these functionalities is published at http://youtu.be/jIMdnwMoWnk while the system is publicly available at http://mediafinder.eurecom.fr/story/elezioni2013 .

MediaFinder: collect, enrich and visualize media memes shared by the crowd

Social networks play an increasingly important role for sharing media items related to humans activities, feelings, emotions and conversations opening a window to the world in real-time. However, these images and videos are spread over multiple social networks. In this paper, we first describe a so-called media server that collect recent images and videos which can be potentially attached to an event. These media items can then be used for the automatic generation of visual summaries. However, making sense out of the resulting media galleries is an extremely challenging task. We present a framework that leverages on: (i) visual features from media items for near-deduplication and (ii) textual features from status updates to enrich, cluster and generate storyboards. A prototype is publicly available at http://mediafinder.eurecom.fr.

Capturing News Stories Once, Retelling a Thousand Ways

We live in a constantly evolving world where news stories and relevant facts are happening every moment. For each of those stories, numerous news articles, posts, and social media reactions are created, offering a multitude of viewpoints about what is happening around us. Many applications have tried to deal with this complexity from very different angles, targeting particular needs, reconstructing certain parts of the story, and exploiting certain visualization paradigms. In this paper, we identify those challenges and study how an adequate news story representation can effectively support the different phases of the news consumption process. We propose an innovative model called News Semantic Snapshot (NSS) that is designed to capture the entire context of a news item. This model can feed very different applications assisting the users before, during, and after the news story consumption. It formalizes a duality in the news annotations that distinguishes between representative entities and relevant entities, and considers different relevancy dimensions that are incorporated into the model in the form of concentric layers. Finally, we analyze the impact of this NSS on existing prototypes and how it can support future ones.

The Concentric Nature of News Semantic Snapshots: Knowledge Extraction for Semantic Annotation of News Items

The Web enables to have access to silo-ed information describing news articles, often offering a multitude of viewpoints that, once combined, can provide a broader picture of the story being reported on the news. In this paper, we propose an approach that automatically extracts representative features of a news item, namely named entities, from textual content attached to a video item (subtitles) and from a set of documents from the Web collected using entity expansion techniques. Approaches relying on entity expansion generally try to collect and process the important facts behinds a particular news item, but they are often too dependent on frequency-based functions and information retrieval techniques thus neglecting the multi-dimensional relationships that are established among the entities. We propose a concentric-based approach that enables to represent the context of a news item, by harmonizing into a single model the representative entities, which can be extracted using information retrieval and natural language processing techniques (Core), and other entities that get prominent according to different dimensions such as informativeness, semantic connectivity, or popularity (Crust). We compare our approach with a baseline by analyzing the compactness of the generated summary on an existing gold standard available on the Web. Results of the experiments show that our approach converges faster to the ideal compact news snapshot with an improvement of 36.9% over the baseline.