Fabio Bruno

Augmented touch without visual obtrusion

Visuo-haptic mixed reality consists of adding to a real scene the ability to see and touch virtual objects. It requires the use of see-through display technology for visually mixing real and virtual objects, and haptic devices for adding haptic interaction with the virtual objects. However, haptic devices tend to be bulky items that appear in the field of view of the user. In this work, we propose a novel mixed reality paradigm where it is possible to touch and see virtual objects in combination with a real scene, but without visual obtrusion produced by the haptic device. This mixed reality paradigm relies on the following three technical steps: tracking of the haptic device, visual deletion of the device from the real scene, and background completion using image-based models. We have developed a successful proof-of-concept implementation, where a user can touch virtual objects in the context of a real scene.

A comparative analysis between active and passive techniques for underwater 3D reconstruction of close-range objects.

In some application fields, such as underwater archaeology or marine biology, there is the need to collect three-dimensional, close-range data from objects that cannot be removed from their site. In particular, 3D imaging techniques are widely employed for close-range acquisitions in underwater environment. In this work we have compared in water two 3D imaging techniques based on active and passive approaches, respectively, and whole-field acquisition. The comparison is performed under poor visibility conditions, produced in the laboratory by suspending different quantities of clay in a water tank. For a fair comparison, a stereo configuration has been adopted for both the techniques, using the same setup, working distance, calibration, and objects. At the moment, the proposed setup is not suitable for real world applications, but it allowed us to conduct a preliminary analysis on the performances of the two techniques and to understand their capability to acquire 3D points in presence of turbidity. The performances have been evaluated in terms of accuracy and density of the acquired 3D points. Our results can be used as a reference for further comparisons in the analysis of other 3D techniques and algorithms.

Visuo-Haptic Mixed Reality with Unobstructed Tool-Hand Integration

Visuo-haptic mixed reality consists of adding to a real scene the ability to see and touch virtual objects. It requires the use of see-through display technology for visually mixing real and virtual objects, and haptic devices for adding haptic interaction with the virtual objects. Unfortunately, the use of commodity haptic devices poses obstruction and misalignment issues that complicate the correct integration of a virtual tool and the users real hand in the mixed reality scene. In this work, we propose a novel mixed reality paradigm where it is possible to touch and see virtual objects in combination with a real scene, using commodity haptic devices, and with a visually consistent integration of the users hand and the virtual tool. We discuss the visual obstruction and misalignment issues introduced by commodity haptic devices, and then propose a solution that relies on four simple technical steps: color-based segmentation of the hand, tracking-based segmentation of the haptic device, background repainting using image-based models, and misalignment-free compositing of the users hand. We have developed a successful proof-of-concept implementation, where a user can touch virtual objects and interact with them in the context of a real scene, and we have evaluated the impact on user performance of obstruction and misalignment correction.

Mixed prototyping with configurable physical archetype for usability evaluation of product interfaces

Mixed prototyping (MP) is an emerging approach for usability testing, thanks to its multimodal environment, which is able to involve sight, hearing and touch thus improving the ability to analyze the inter-relationships between the physical form and the behavior of the industrial products. This paper presents a method to perform usability tests in a mixed reality (MR) environment for analyzing human performance in target acquisition tasks while interacting with household appliances. The proposed method is based on the use of different kinds of digital and physical prototypes and, moreover, it introduces an experimental physical archetype for mixed prototyping that contributes to increase the efficiency of the usability evaluation process. Through this archetype the design of a user interface can be easily changed by the adoption of plug-and-play moving components (knobs and buttons) that allow to model in a few seconds any kind of control panel for washing machines, thus reducing the prototyping costs and enlarging the variety of MR interfaces that can be evaluated. The paper proposes also a validation of the use of the physical archetype through a case study in which three different control panel alternatives have been evaluated in a competitive usability study. The competitive testing allows to gather user behaviors with a broad range of design options before the development of a new control panel refined through iterative design. Experimental results show that the proposed method based on the physical archetype can be an effective support to improve the usability of the product interface.

3D documentation and monitoring of the experimental cleaning operations in the underwater archaeological site of Baia (Italy)

The CoMAS project aims to develop new methodologies and tools for the restoration and conservation of submerged archaeological artefacts. One of the project goals is to study the cleaning operations that are intended to remove the living organisms (algae, sponges, molluscs, etc.). These organisms cause severe deteriorations of the artefacts and their removal is crucial for the subsequent phases of consolidation and protection. In particular, the Unit of Underwater Archaeology of the Istituto Superiore per la Conservazione ed il Restauro is experimenting and comparing different cleaning techniques and tools developed in the context of the CoMAS project. This comparison requires also a precise documentation of the test cases and a quantitative measurement of the effectiveness of the cleaning procedures. This paper describes the process that has been defined in order to document and monitor the results obtained through different cleaning experiments, conducted with different tools and utensils over various types of surfaces (marble, bricks, mortar, etc.) affected by several types of biological colonization. The process includes: 1) the 3D mapping of the experimental site; 2) the planning of the tests and the choice of the areas to be cleaned; 3) the 3D reconstruction of the selected areas; 4) the execution of cleaning operations; 5) the 3D acquisition of the cleaned areas; 6) the comparison of the 3D model of the areas before and after the cleaning; 7) the analysis of the data about the cleaning effectiveness. This process has been implemented during an experimentation carried out in the “Villa dei Pisoni”, located within the underwater archaeological site of Baia near Naples.

Visualization of Industrial Engineering Data Visualization of Industrial Engineering Data in Augmented Reality

This paper presents an innovative application of Augmented Reality (AR) techniques in the field of industrial engineering in which the user explores data from numerical simulations or the results of measurements and experiments, superimposed to the real object that they refer to. The user observes the object through a tablet PC, used as a video see-through handheld display. Data are visualized superimposed to the real object that represents a spatial reference relative to which the user can refer to, so the exploration is more natural compared to a traditional visualization software. Moreover, we have developed a new framework, called VTK4AR, that provides a set of useful software classes for the rapid development of AR applications for scientific visualization. VTK4AR is built on top of VTK (an open source API for scientific visualization), so it will be possible to employ a wide range of visualization techniques in many application fields, and moreover, it is possible to interactively manipulate data-sets in order to achieve a more effective way of visualization.

An Alignment Method for the Integration of Underwater 3D Data Captured by a Stereovision System and an Acoustic Camera.

The integration of underwater 3D data captured by acoustic and optical systems is a promising technique in various applications such as mapping or vehicle navigation. It allows for compensating the drawbacks of the low resolution of acoustic sensors and the limitations of optical sensors in bad visibility conditions. Aligning these data is a challenging problem, as it is hard to make a point-to-point correspondence. This paper presents a multi-sensor registration for the automatic integration of 3D data acquired from a stereovision system and a 3D acoustic camera in close-range acquisition. An appropriate rig has been used in the laboratory tests to determine the relative position between the two sensor frames. The experimental results show that our alignment approach, based on the acquisition of a rig in several poses, can be adopted to estimate the rigid transformation between the two heterogeneous sensors. A first estimation of the unknown geometric transformation is obtained by a registration of the two 3D point clouds, but it ends up to be strongly affected by noise and data dispersion. A robust and optimal estimation is obtained by a statistical processing of the transformations computed for each pose. The effectiveness of the method has been demonstrated in this first experimentation of the proposed 3D opto-acoustic camera.

Mixed prototyping for products usability evaluation

Mixed prototyping is an industrial practice that combines virtual and real components in order to realize a prototype of a product used to evaluate and assess the design choices. Recently, mixed prototypes have been also used to assess the usability of products interface. This particular application arises several problems related to the devices and the interaction techniques that, better than others, allow a natural interaction with the mixed prototype. This paper presents a mixed reality environment for usability evaluation that deals with two specific problems of this kind of application: the occlusion between real and virtual objects and the interpretation of the user’s gestures while he/she is interacting with the elements of the product interface. In particular we propose a technique able to manage both the problems by using only commodity hardware and video processing algorithms, thus avoiding the use of expensive data-gloves and tracking devices. The proposed approach has been validated through a user study addressed to establish whether and to what extent the augmented reality devices and the techniques proposed may distort the usability assessment of the product. Moreover, the user study compares the mixed reality environment adopted in this study with a classical virtual reality set-up.Copyright

Methodology and tools to support knowledge management in topology optimization

Topological optimization (TO) tools are today widely employed in several engineering fields (e.g., construction, aeronautics, aerospace, and automotive). The diffusion of these tools is due to their capacity to improve mechanical properties of products through a global optimization of the product in terms of weight, stiffness, strength, and cost. On the other hand, the adoption of TO tools still requires a sizeable organizational effort because, at present, these tools are mostly stand-alone and are not well integrated into the product development process (PDP). This paper presents an innovative methodology that supports designers and analysts in formalizing and transmitting design choices taken during project activities and in making the integration of TO tools in the PDP more efficient. The methodology clearly defines the roles, the activities, the data to exchange, and the software tools to be used in the process. Some custom computer-aided design automation tools have been implemented to improve the efficiency of the methodology. Moreover, this paper defines an original procedure to support the interpretation of the TO results.

Virtual and Augmented Reality Tools to Improve the Exploitation of Underwater Archaeological Sites by Diver and Non-diver Tourists

The underwater cultural heritage is an immeasurable archaeological and historical resource with huge, but yet largely unexploited, potentials for the maritime and coastal tourism.