Vito Gentile

Continuous Hand Openness Detection Using a Kinect-Like Device

This paper presents a novel method to reproduce in real time the opening and closing gestures of a human hand, animating a three-dimensional model of it. In other works, this result can be achieved by mapping a set of significant points of a real hand on the corresponding points of the model to animate. We propose an alternative way to produce the same effect without mapping points, but using a level-based estimation of the degree of opening of the hand. The experiments have been executed using Microsoft KinectTM, but the method would work on any other Kinect-like devices (as defined herein). The results obtained are particularly encouraging and demonstrate real-time performance of the system.

Real-Time Hand Pose Recognition Based on a Neural Network Using Microsoft Kinect

The Microsoft Kinect sensor is largely used to detect and recognize body gestures and layout with enough reliability, accuracy and precision in a quite simple way. However, the pretty low resolution of the optical sensors does not allow the device to detect gestures of body parts, such as the fingers of a hand, with the same straightforwardness. Given the clear application of this technology to the field of the user interaction within immersive multimedia environments, there is the actual need to have a reliable and effective method to detect the pose of some body parts. In this paper we propose a method based on a neural network to detect in real time the hand pose, to recognize whether it is closed or not. The neural network is used to process information of color, depth and skeleton coming from the Kinect device. This information is preprocessed to extract some significant feature. The output of the neural network is then filtered with a time average, to reduce the noise due to the fluctuation of the input data. We analyze and discuss three possible implementations of the proposed method, obtaining an accuracy of 90% under good conditions of lighting and background, and even reaching the 95% in best cases, in real time.

Touchless Interfaces For Public Displays: Can We Deliver Interface Designers From Introducing Artificial Push Button Gestures?

Public displays have lately become ubiquitous thanks to the decreasing cost of such technology and public policies supporting the development of smart cities. Depending on form factor, those displays might use touchless gestural interfaces that therefore are becoming more often the subject of public and private research. In this paper, we focus on touchless interactions with situated public displays, and introduce a pilot study on comparing two interfaces: an interface based on the Microsoft Human Interface Guidelines (HIG), a de facto standard in the field, and a novel interface, designed by us. Differently from the HIG-based one, our interface displays an avatar, which does not require any activation gestures to trigger actions. Our aim is to study how the two interfaces address the so-called interaction blindness --- the inability of the users to recognize the interactive capabilities of those displays. According to our pilot study, although providing a different approach, both interfaces has proven effective in the proposed scenario: a public display in a hall inside a University campus building.

Designing Touchless Gestural Interactions for Public Displays In-the-Wild

Public displays, typically equipped with touchscreens, are used for interactions in public spaces, such as streets or fairs. Currently low-cost visual sensing technologies, such as Kinect-like devices and high quality cameras, allow to easily implement touchless interfaces. Nevertheless, the arising interactions have not yet been fully investigated for public displays in-the-wild (i.e. in appropriate social contexts where public displays are typically deployed). Different audiences, cultures and social settings strongly affect users and their interactions. Besides gestures for public displays must be guessable to be easy to use for a wide audience. Issues like these could be solved with user-centered design: gestures must be chosen by users in different social settings, and then selected to be resilient to cultural bias and provide a good level of guessability. Therefore the main challenge is to define touchless gestures in-the-wild by using novel UCD methods applied out of controlled environments, and evaluating their effectiveness.

They are looking at me!: understanding how audience presence impacts on public display users

It is well known from prior work, that people interacting as well as attending to a public display attract further people to interact. This behavior is commonly referred to as the honeypot effect. At the same time, there are often situations where an audience is present in the vicinity of a public display that does not actively engage or pay attention to the display or an approaching user. However, it is largely unknown how such a passive audience impacts on users or people who intend to interact. In this paper, we investigate the influence of a passive audience on the engagement of people with a public display. In more detail, we report on the deployment of a display in a public space. We collected and analyzed video logs to understand how people react to passive audience in the vicinity of public displays. We found an influence on where interacting users position themselves relative to both display and passive audience as well as on their behavior. Our findings are valuable for display providers and space owners who want to maximize the displays benefits.

Body Gestures and Spoken Sentences: A Novel Approach for Revealing User’s Emotions

In the last decade, there has been a growing interest in emotion analysis research, which has been applied in several areas of computer science. Many authors have contributed to the development of emotion recognition algorithms, considering textual or non verbal data as input, such as facial expressions, gestures or, in the case of multi-modal emotion recognition, a combination of them. In this paper, we describe a method to detect emotions from gestures using the skeletal data obtained from Kinect-like devices as input, as well as a textual description of their meaning. The experimental results show that the correlation existing between body movements and spoken user sentence(s) can be used to reveal users emotions from gestures.

Modular Middleware for Gestural Data and Devices Management

In the last few years, the use of gestural data has become a key enabler for human-computer interaction (HCI) applications. The growing diffusion of low-cost acquisition devices has thus led to the development of a class of middleware aimed at ensuring a fast and easy integration of such devices within the actual HCI applications. The purpose of this paper is to present a modular middleware for gestural data and devices management. First, we describe a brief review of the state of the art of similar middleware. Then, we discuss the proposed architecture and the motivation behind its design choices. Finally, we present a use case aimed at demonstrating the potential uses as well as the limitations of our middleware.

Palmprint principal lines extraction

The palmprint recognition has become a focus in biological recognition and image processing fields. In this process, the features extraction (with particular attention to palmprint principal line extraction) is especially important. Although a lot of work has been reported, the representation of palmprint is still an open issue. In this paper we propose a simple, efficient, and accurate palmprint principal lines extraction method. Our approach consists of six simple steps: normalization, median filtering, average filters along four prefixed directions, grayscale bottom-hat filtering, combination of bottom-hat filtering, binarization and post processing. The contribution of our work is a new method for palmprint principal lines detection and a new dataset of hand labeled principal lines images (that we use as ground truth in the experiments). Preliminary experimental results showed good performance in terms of accuracy with respect to three methods of the state of the art.

Touchless gestural interfaces for networked public displays: overcoming interaction blindness and performing evaluations in-the-wild

In the near future, we can easily imagine a significant increment in diffusion of networked public displays, as well as novel interaction modalities used in their applications. In the following, we present two of the main challenges related to networked displays we are dealing with, with a particular focus on touchless gestural interfaces: overcoming interaction blindness (i.e. enable users to immediately guess the interactivity of the display, and the gestural nature of it) and performing evaluations in-the-wild (i.e. outside any controlled environment).

Conveying Audience Emotions through Humanoid Robot Gestures to an Orchestra during a Live Musical Exhibition

In the last twenty years, robotics have been applied in many heterogeneous contexts. Among them, the use of humanoid robots during musical concerts have been proposed and investigated by many authors. In this paper, we propose a contribution in the area of robotics application in music, consisting of a system for conveying audience emotions during a live musical exhibition, by means of a humanoid robot. In particular, we provide all spectators with a mobile app, by means of which they can select a specific color while listening to a piece of music (act). Each color is mapped to an emotion, and the audience preferences are then processed in order to select the next act to be played. This decision, based on the overall emotion felt by the audience, is then communicated by the robot through body gestures to the orchestra. Our first results show that spectators enjoy such kind of interactive musical performance, and are encouraging for further investigations.