Markus Brenner

Social event detection and retrieval in collaborative photo collections

In this paper, we present an approach to detect social events and retrieve associated photos in collaboratively annotated photo collections. We combine data of various modalities such as time, location, and textual and visual features within a framework that has a classification model at its core. Compared to traditional approaches that mainly consider the photos only as a source of information, we also incorporate external information from datasets and online web services to further improve the performance. Experiments based on the MediaEval Social Event Detection Dataset demonstrate the effectiveness of our approach.

Graph-based recognition in photo collections using social semantics

In this paper, we show how to recognize people in Consumer Photo Collections by employing a graphical model together with a distance-based face description method. We devise a graph design and explore ways to further improve the recognition performance by incorporating context in the form of social semantics. Experiments on a public dataset demonstrate the effectiveness of our probabilistic approach compared to traditional nearest-neighbor matching.

Joint People Recognition across Photo Collections Using Sparse Markov Random Fields

We show how to jointly recognize people across an entire photo collection while considering the specifies of personal photos that often depict multiple people. We devise and explore a sparse but efficient graph design based on a second-order Markov Random Field, and that utilizes a distance-based face description method. Experiments on two datasets demonstrate and validate the effectiveness of our probabilistic approach compared to traditional methods.

People recognition using gamified ambiguous feedback

We present a semi-supervised approach to recognize faces or people while incorporating crowd-sourced and gamified feedback to iteratively improve recognition accuracy. Unlike traditional approaches which are often limited to explicit feedback, we model ambiguous feedback information that we implicitly gather through a crowd that plays a game. We devise a graph-based recognition approach that incorporates such ambiguous feedback to jointly recognize people across an entire dataset. Multiple experiments demonstrate the effectiveness of our gamified feedback approach.

Recognizing people by face and body in Photo Collections

We show how to detect and recognize people based on their faces and bodies in Consumer Photo Collections. We devise a graphical model that incorporates multiple contextual cues to discriminate faces, upper and lower bodies, and ultimately, individuals without relying on faces. For efficiency, we only consider body features when faces are not discriminative enough. Experiments on two datasets demonstrate the effectiveness of our probabilistic approach.

Gender-aided people recognition in photo collections

We show how to recognize people based on their faces in Consumer Photo Collections while also incorporating context in the form of gender information. We devise and explore a unified framework that has a graphical model along a distance-based face description method at its core. We jointly recognize people across an entire photo collection to also consider the specifics of photos that depict multiple people. Experiments on two datasets demonstrate and validate the effectiveness of our probabilistic approach compared to traditional methods that do not consider gender information.

People recognition in ambiguously labeled Photo Collections

We show how to recognize people based on their faces in Consumer Photo Collections while also incorporating context in the form of ambiguous labels. Such labels can be assigned to single photos (depicting multiple people) as well as to entire sets of photos (e.g. relating to events). To achieve this, we devise a unified framework that has a graphical model along a distance-based face description method at its core. We evaluate our probabilistic approach by performing experiments on two datasets, one of which includes around 5000 face appearances spanning nearly ten years.

Event-driven retrieval in collaborative photo collections

We present an approach to retrieve photos relating to social events in collaborative photo collections. Compared to traditional approaches that typically consider only the visual features of photos as a source of information, we incorporate multiple additional contextual cues like date and time, location and usernames to improve retrieval performance. Experiments based on the MediaEval Social Event Detection Dataset demonstrate the effectiveness of our approach.

Temporal face embedding and propagation in photo collections

We present a two-step approach for modeling facial variations and class likelihoods over time. Unlike traditional approaches, we explicitly model the temporal domain that is often available, for example, in consumer photos or surveillance systems. Our combined approach draws upon the concepts of manifold transformation and semi-supervised graph-based propagation to simultaneously recognize faces across entire photo collections. Experiments on two datasets demonstrate improved face recognition accuracy.

Mining People’s Appearances to Improve Recognition in Photo Collections

We show how to recognize people in Consumer Photo Collections by employing a graphical model together with a distance-based face description method. To further improve recognition performance, we incorporate context in the form of social semantics. We devise an approach that has a data mining technique at its core to discover and incorporate patterns of groups of people frequently appearing together in photos. We demonstrate the effect of our probabilistic approach through experiments on a dataset that spans nearly ten years.