Floriana Renna

A haptic/acoustic application to allow blind the access to spatial information

This paper describes a multi-modal application, based on haptic/acoustic interaction, to allow visually impaired people to access media with high spatial information content, extending the visual and auditory-verbal data with the tactile sensory channel. Haptic interfaces are exploited to enable the use of touch on 3D virtual objects which have a larger and greater flexibility with respect to physical artefacts. The system allows to interact with haptic/acoustic active objects and to select the information that must be shown on the basis of the user requirements. Two different tests, involving the exploration of indoor environment and of complex geographical areas, are presented. Several experiments, with users with different stories and different level of visual disability, have been done. They showed that the haptic/acoustic interaction and the modularity in the information representation help blind people to cope with the serious and challenging task of managing spatial data

Non-destructive evaluation of quality and ammonia content in whole and fresh-cut lettuce by computer vision system

The paper describes the developed hardware and software components of a computer vision system that extracts colour parameters from calibrated colour images and identifies non-destructively the different quality levels exhibited by lettuce (either whole or fresh-cut) during storage. Several colour parameters extracted by computer vision system have been evaluated to characterize the product quality levels. Among these, brown on total and brown on white proved to achieve a good identification of the different quality levels on whole and fresh-cut lettuce (P-value<0.0001). In particular, these two parameters were able to discriminate three levels: very good or good products (quality levels from 5 to 4), samples at the limit of marketability (quality level of 3) and waste items (quality levels from 2 to 1). Quality levels were also chemically and physically characterized. Among the parameters analysed, ammonia content proved to discriminate the marketable samples from the waste in both products typologies (either fresh-cut or whole); even the two classes of waste were well discriminated by ammonia content (P-value<0.0001). A function that infers quality levels from the extracted colour parameters has been identified using a multi-regression model (R2=0.77). Multi-regression also identified a function that predicts the level of ammonia (an indicator of senescence) in the iceberg lettuce from a colour parameter provided by the computer vision system (R2=0.73), allowing a non-destructive evaluation of a chemical parameter that is particularly useful for the objective assessment of lettuce quality. The developed computer vision system offers flexible and simple non-destructive tool that can be employed in the food processing industry to monitor the quality and shelf life of whole and fresh-cut lettuce in a reliable, objective and quantitative way.

Haptic fruition of 3D virtual scene by blind people

Haptic interfaces may allow blind people to interact naturally and realistically with 3D virtual models of objects that are unsuitable for direct tactile exploration. The haptic interaction can be offered at different scales, by changing the relative size of probe and objects and by organizing different levels of details into the model. In addition, haptic interfaces can actively drive the user along the most effective exploration path around the scene. All these features can significantly help the synthesis and the understanding of the huge amount of tactile sensations (that blinds must collect serially) beyond the limits of the exploration in the real world. The paper describes an architecture (and its already realized modules for visualization, collision detection and force simulation) intended to generate a reliable simulation of the geometrical and physical interactions between the users hand and a virtual 3D scene.

Color correction for the virtual recomposition of fragmented frescos

The paper describes the solution to the color correction problem used in the digital system we have designed for the reconstruction of the S. Matthew fresco, painted by Cimabue for the Upper Church of S. Francis in Assisi. The characteristics of the problem make it difficult to evaluate correspondences between colors and require specific corrections to be applied to different parts of the fresco. The obtained improvement in terms of color similarity and retrieval results is shown.

Hapto-Acoustic Interaction Metaphors in 3D Virtual Environments for Non-Visual Settings

Three-dimensional Virtual Environments (VEs) enable the acquisition of knowledge on a given domain through the interaction with virtual entities. The flexibility of a VE allows to represent only the part of the world that is considered relevant for the final user: the proper choice of the information, representation and rendering included in the virtual world can strongly simplify the perception and interpretation efforts required to the users. Moreover, VE can provide data that would be difficult or impossible to appreciate in the real world in an easily and simply perceivable way: domain experts can communicate specific views and interpretations of the reality in a way accessible to final users. A properly designed virtual experience can significantly improve and simplify several learning tasks. Organizing information in three dimensions and designing techniques to interact with them require a complex effort: interaction metaphors have been introduced to facilitate the access and interaction with VEs (Bowmann, 2001). A metaphor is the process of mapping a set of correspondences from a source domain to a target domain (Lakoff & Johnson, 1980). Metaphors help designers to map features of the interaction techniques to concepts more immediately accessible to final users. Interaction can be made more immersive and engaging by multi-modality. A multimodal system coordinates the processing of multiple natural input modalities—such as speech, touch, hand gestures, eye gaze and head and body movements—with multimedia system output (Oviat, 1999). The interaction is carried out with advanced input/output devices involving different sensorial channels (sight, hear, touch, etc.) in an integrated way. Spatial input devices (such as trackers, 3D pointing devices, gesture and vocal devices) and multisensory output technologies (head mounted displays, spatial audio and haptic devices) are increasingly being used as common components of Virtual Reality applications. Each device addresses a particular sense and exhibits a different interface: (Bowmann et al., 2004) offers a broad review of multimodal interaction while (Salisbury, 2004) is a good introduction to haptics. A multimodal interaction requires data to be redundant and polymorphous to address different sensorial modalities at the same time (Jacobson, 2002). Metaphors effectiveness strongly depends on the sensory channels they refer to and on the users characteristics. Therefore this presentation will correlate and compare hapto-acoustic

A geographical virtual laboratory for the recomposition of fragments

The paper describes a digital system for the virtual aided recomposition of fragmented frescos whose approach allows knowledge and experience of restorers to cooperate with computational power and flexibility of digital tools for image analysis and retrieval. The physical laboratory for the traditional recomposition is replaced by a geographically distributed client-server architecture implementing a virtual laboratory of fragments. Image processing and analysis techniques support the whole recomposition task. A properly designed engine for image indexing and retrieval enables the retrieval of fragments similar to suitably chosen sample images.

Perceptually grounded color evaluation in virtual aided recomposition of fragmented frescos

The paper deals with a problem arisen in developing a system for the aided virtual recomposition of fragmented frescos (in particular the S. Mathews fresco of the S. Francis Upper Church in Assisi). The goal is to expand the capabilities of the operators which remains responsible of the whole process. A core functionality is the automatic evaluation of similarity between images of fragments in a consistent way with evaluations made by humans using their visual perception: a critical property for working in tight cooperation with the operators. This requires a color representation close to human color matching. S-CIELAB, a spatial extension of the CIELAB color representation, is a space whose metrics closely reproduces, through the Euclidean norm, the color distances perceived by a human observer and accounts for the effects of the spatial distribution of colors. S-CIELAB extends CIELAB by incorporating factors related to the pattern-color sensitivity of the human eye. The system ascribes to the fragment pattern-color characteristics according to the visual perception the human operator has of the fragment; the use of automatic tools for color evaluation avoids the inconsistent results due to different operators and to fatigue of the same person over time.