Sergio Benini

XKin: an open source framework for hand pose and gesture recognition using kinect

This work targets real-time recognition of both static hand-poses and dynamic hand-gestures in a unified open-source framework. The developed solution enables natural and intuitive hand-pose recognition of American Sign Language (ASL), extending the recognition to ambiguous letters not challenged by previous work. While hand-pose recognition exploits techniques working on depth information using texture-based descriptors, gesture recognition evaluates hand trajectories in the depth stream using angular features and hidden Markov models (HMM). Although classifiers come already trained on ASL alphabet and 16 uni-stroke dynamic gestures, users are able to extend these default sets by adding their personalized poses and gestures. The accuracy and robustness of the recognition system have been evaluated using a publicly available database and across many users. The XKin open project is available online (Pedersoli, XKin libraries. https://github.com/fpeder/XKin, 2013) under FreeBSD License for researchers in human–machine interaction.

Extraction of Significant Video Summaries by Dendrogram Analysis

In the current video analysis scenario, effective clustering of shots facilitates the access to the content and helps in understanding the associated semantics. This paper introduces a cluster analysis on shots which employs dendrogram representation to produce hierarchical summaries of the video document. Vector quantization codebooks are used to represent the visual content and to group the shots with similar chromatic consistency. The evaluation of the cluster codebook distortions, and the exploitation of the dependency relationships on the dendrograms, allow to obtain only a few significant summaries of the whole video. Finally the user can navigate through summaries and decide which one best suites his/her needs for eventual post-processing. The effectiveness of the proposed method is demonstrated by testing it on a collection of video-data from different kinds of programmes. Results are evaluated in terms of metrics that measure the content representational value of the summarization technique.

Hidden Markov Models for Video Skim Generation

In this paper we present a statistical framework based on hidden Markov models (HMMs) for video skimming. A chain of HMMs is used to model subsequent story units: HMM states represent different visual-concepts, transitions model the temporal dependencies in each story unit, and stochastic observations are given by single shots. The skim is generated as an observation sequence, where, in order to privilege more informative segments for entering the skim, dynamic shots are assigned higher probability of observation. The effectiveness of the method is demonstrated on a video set from different kinds of programmes, and results are evaluated in terms of metrics that assess the content representational value of the obtained video skims.

Emotional identity of movies

In the field of multimedia analysis, attempts that lead to an emotional characterization of content have been proposed. In this work we aim at defining the emotional identity of a feature movie by positioning it into an emotional space, as if it was a piece of art. The multimedia content is mapped into a trajectory whose coordinates are connected to filming and cinematographic techniques used by directors to convey emotions. The trajectory evolution over time provides a strong characterization of the movie, by locating different movies into different regions of the emotional space. The ability of this tool in characterizing content has been tested by retrieving emotionally similar movies from a large database, using IMDb genre classification for the evaluation of results.

Interactive storytelling via video content recombination

In the paper we present a prototype of video-based storytelling that is able to generate multiple story variants from a baseline video. The video content for the system is generated by an adaptation of forefront video summarisation techniques that decompose the video into a number of Logical Story Units (LSU) representing sequences of contiguous and interconnected shots sharing a common semantic thread. Alternative storylines are generated using AI Planning techniques and these are used to direct the combination of elementary LSU for output. We report early results from experiments with the prototype in which the reordering of video shots on the basis of their high-level semantics produces trailers giving the illusion of different storylines.

XKin -: eXtendable hand pose and gesture recognition library for kinect

In this work we provide an open-source framework for Kinect enabling more natural and intuitive hand-gesture communication between human and computer devices. The software package is endowed with useful tools for training the system to work with user-defined postures and gestures. The XKin project is fully implemented in C and freely available at https://github.com/fpeder/XKin under FreeBSD License. Our goal is to encourage contributions from other researchers and developers in building an open and effective system for empowering a natural modality for human-machine interaction.

Markers of unsustainable gambling for early detection of at-risk online gamblers

In this work we propose novel markers for identifying at-risk gamblers based on the concept of sustainability. The first hypothesis here verified is that problematic gamblers oscillate between intervals of increasing wager size followed by rapid drops, probably because they exceed their economic sustainability limits. Due to the non-periodic nature of these fluctuations, the proposed marker detects a certain occurring feature, such as a rapid drop in wager size, over a wide range of fluctuation periods, drop sizes and shapes. The second marker, counting the number of games the gambler is involved in, aims at predicting possible consequences of an exceeding amount of time dedicated to gambling, that ultimately causes social and relational breakdowns. In the experimental phase we demonstrate how the adoption of these markers allows for identifying larger segments of high- and medium-risk gamblers with respect to previous research on actual betting behaviours.

Users' response to affective film content: A narrative perspective

In this paper, we take a human-centred view to the definition of the affective content of films. We investigate the relationship between users physiological response and multimedia features extracted from the movies, from the perspective of narrative evolution rather than by measuring average values. We found a certain dynamic correlation between arousal, derived from measures of Galvanic Skin Resistance during film viewing, and specific multimedia features in both sound and video domains. Dynamic physiological measurements were also consistent with post-experiment self-assessment by the subjects. These findings suggest that narrative aspects (including staging) are central to the understanding of video affective content, and that direct mapping of video features to emotional models taken from psychology may not capture these phenomena in a straightforward manner.

A Statistical Framework for Video Skimming Based on Logical Story Units and Motion Activity

In this work we present a method for video skimming based on hidden Markov Models (HMMs) and motion activity. Specifically, a set of HMMs is used to model subsequent logical story units, where the HMM states represent different visual-concepts, the transitions model the temporal dependencies in each story unit, and stochastic observations are given by single shots. The video skim is generated as an observation sequence, where, in order to privilege more informative segments for entering the skim, dynamic shots are assigned higher probability of observation. The effectiveness of the method is demonstrated on a video set from different kinds of programmes, and results are evaluated in terms of metrics that measure the content representational value of the obtained video skims.

Statistical Skimming of Feature Films

We present a statistical framework based on Hidden Markov Models (HMMs) for skimming feature films. A chain of HMMs is used to model subsequent story units: HMM states represent different visual-concepts, transitions model the temporal dependencies in each story unit, and stochastic observations are given by single shots. The skim is generated as an observation sequence, where, in order to privilege more informative segments for entering the skim, shots are assigned higher probability of observation if endowed with salient features related to specific film genres. The effectiveness of the method is demonstrated by skimming the first thirty minutes of a wide set of action and dramatic movies, in order to create previews for users useful for assessing whether they would like to see that movie or not, but without revealing the movie central part and plot details. Results are evaluated and compared through extensive user tests in terms of metrics that estimate the content representational value of the obtained video skims and their utility for assessing the users interest in the observed movie.