Salvatore Sorce

An agent-based service network for personal mobile devices

We propose the Agent Network for Bluetooth Devices, a system that uses personal mobile devices as adaptive human-environment interfaces to supply people with ad hoc information and high-level services. The ANBD system operates with a hierarchical framework of service-providing nodes, dynamically composed and managed by mobile agents.

Human-to-human interfaces: emerging trends and challenges

We present a new research domain, human-to-human interaction (HHI) that describes how todays human interaction is largely indirect and mediated by a wide variety of technologies and devices. We show how this new and exciting field of design originates from the convergence of a few well-established research areas, such as traditional graphical user interfaces (GUIs), tangible user interfaces (TUIs), touchless gesture user interfaces (TGUIs), voice user interfaces (VUIs), and brain computer interfaces (BCIs). We analyse and describe current research in those areas and offer a first-hand view and presentation of its salient aspects for the human-to human interaction domain.

Continuous Hand Openness Detection Using a Kinect-Like Device

This paper presents a novel method to reproduce in real time the opening and closing gestures of a human hand, animating a three-dimensional model of it. In other works, this result can be achieved by mapping a set of significant points of a real hand on the corresponding points of the model to animate. We propose an alternative way to produce the same effect without mapping points, but using a level-based estimation of the degree of opening of the hand. The experiments have been executed using Microsoft KinectTM, but the method would work on any other Kinect-like devices (as defined herein). The results obtained are particularly encouraging and demonstrate real-time performance of the system.

An augmented campus design for context-aware service provision

This paper deals with the design of a multi-modal system for pervasive context-aware service provision and human-environment interaction in augmented environments by the use of Personal Digital Assistants (PDA) or SmartPhones. The system enables mobile devices and remote displays to perform as interaction devices with pervasive applications which run on a dynamically composed server network. Visual interaction for service setup and provision are driven by appropriate graphical interfaces and XML-based protocols, which are dynamically composed according to the type of service and to the user current position by means of a mobile agent-based framework. The paper discusses both protocols, hardware and software system components. The first part of the document gives a general description of the system, which is managed by an entity-driven organization in augmented reality.The mobile and reference devices of the system framework are then discussed, along with the mobile agent software which is used to manage connections among them and with system entities. The paper also gives some details about the ad-hoc protocols for entity interaction. Next, a case study is discussed dealing with service provision in a campus augmented environment which has been arranged according to service requirements. Finally the paper discusses some user experiences while using trial services.

Interacting with Augmented Environments

Pervasive systems augment environments by integrating information processing into everyday objects and activities. They consist of two parts: a visible part populated by animate (visitors, operators) or inanimate (AI) entities interacting with the environment through digital devices, and an invisible part composed of software objects performing specific tasks in an underlying framework. This paper shows an ongoing work from the University of Palermos Department of Computer Science and Engineering that addresses two issues related to simplifying and broadening augmented environment access.

Real-Time Hand Pose Recognition Based on a Neural Network Using Microsoft Kinect

The Microsoft Kinect sensor is largely used to detect and recognize body gestures and layout with enough reliability, accuracy and precision in a quite simple way. However, the pretty low resolution of the optical sensors does not allow the device to detect gestures of body parts, such as the fingers of a hand, with the same straightforwardness. Given the clear application of this technology to the field of the user interaction within immersive multimedia environments, there is the actual need to have a reliable and effective method to detect the pose of some body parts. In this paper we propose a method based on a neural network to detect in real time the hand pose, to recognize whether it is closed or not. The neural network is used to process information of color, depth and skeleton coming from the Kinect device. This information is preprocessed to extract some significant feature. The output of the neural network is then filtered with a time average, to reduce the noise due to the fluctuation of the input data. We analyze and discuss three possible implementations of the proposed method, obtaining an accuracy of 90% under good conditions of lighting and background, and even reaching the 95% in best cases, in real time.

Exploitation of Mobile Access to Context-Based Information in Cultural Heritage Fruition

More than one billion smartphone users are estimated by 2014. With this in mind, visiting cultural heritage sites and exhibits may offer new level of engagement and entertainment just reaching down in our pockets. With orders of magnitude more computational horsepower than a five years-old desktop machine, stuffed with all sorts of sensors, these modern gizmos have a largely untapped potential to gain us access to personalized and on-demand information wherever it is needed. This paper is exactly about this, exploring with several case studies how these devices may become part of a memorable experience during a visit that one may want to share with friends and relatives. Specifically, the paper will focus on the definition of the user-experience (UX), on integration issues, and on context detection within augmented environments in cultural heritage sites, along with a discussion on the lessons learnt.

Novel Human-to-Human Interactions from the Evolution of HCI

The interaction ways made available by the evolution of the human-computer interfaces, led to novel Human-to-Human Interaction (HHI) modes, enabling people to cooperate for almost any task any time and any where. HHI nowadays is largely indirect and mediated by a wide variety of technologies and devices. This new and exciting field of design originates from the convergence of a few well-established research fields within the HCI area, such as traditional Graphical User Interfaces (GUI), Tangible User Interfaces (TUI), Touchless Gesture User Interface (TGUI), Voice User Interfaces (VUI), and Brain Computer Interfaces (BCI). We analyze and describe the evolution of the HCI in those fields, and how it paved the way for novel current and upcoming human-to human interaction ways.

A virtual shopper customer assistant in pervasive environments

In this work we propose a smart, human-like PDA-based personal shopper assistant. The system is able to understand the user needs through a spoken natural language interaction and then stores the preferences of the potential customer. Subsequently the personal shopper suggests the most suitable items and shops that match the user profile. The interaction is given by automatic speech recognition and text-to-speech technologies; localization is allowed by the use of Wireless technologies, while the interaction is performed by an Alice-based chat-bot endowed with reasoning capabilities. Besides, being implemented on a PDA, the personal shopper satisfies the user needs of mobility and it is also usable on different mobile devices.

Touchless Interfaces For Public Displays: Can We Deliver Interface Designers From Introducing Artificial Push Button Gestures?

Public displays have lately become ubiquitous thanks to the decreasing cost of such technology and public policies supporting the development of smart cities. Depending on form factor, those displays might use touchless gestural interfaces that therefore are becoming more often the subject of public and private research. In this paper, we focus on touchless interactions with situated public displays, and introduce a pilot study on comparing two interfaces: an interface based on the Microsoft Human Interface Guidelines (HIG), a de facto standard in the field, and a novel interface, designed by us. Differently from the HIG-based one, our interface displays an avatar, which does not require any activation gestures to trigger actions. Our aim is to study how the two interfaces address the so-called interaction blindness --- the inability of the users to recognize the interactive capabilities of those displays. According to our pilot study, although providing a different approach, both interfaces has proven effective in the proposed scenario: a public display in a hall inside a University campus building.