Fabrizio Cutolo

Augmented reality as an aid in maxillofacial surgery: validation of a wearable system allowing maxillary repositioning.

AIM We present a newly designed, localiser-free, head-mounted system featuring augmented reality as an aid to maxillofacial bone surgery, and assess the potential utility of the device by conducting a feasibility study and validation. METHODS Our head-mounted wearable system facilitating augmented surgery was developed as a stand-alone, video-based, see-through device in which the visual features were adapted to facilitate maxillofacial bone surgery. We implement a strategy designed to present augmented reality information to the operating surgeon. LeFort1 osteotomy was chosen as the test procedure. The system is designed to exhibit virtual planning overlaying the details of a real patient. We implemented a method allowing performance of waferless, augmented-reality assisted bone repositioning. In vitro testing was conducted on a physical replica of a human skull, and the augmented reality system was used to perform LeFort1 maxillary repositioning. Surgical accuracy was measured with the aid of an optical navigation system that recorded the coordinates of three reference points (located in anterior, posterior right, and posterior left positions) on the repositioned maxilla. The outcomes were compared with those expected to be achievable in a three-dimensional environment. Data were derived using three levels of surgical planning, of increasing complexity, and for nine different operators with varying levels of surgical skill. RESULTS The mean error was 1.70 ± 0.51 mm. The axial errors were 0.89 ± 0.54 mm on the sagittal axis, 0.60 ± 0.20 mm on the frontal axis, and 1.06 ± 0.40 mm on the craniocaudal axis. The simplest plan was associated with a slightly lower mean error (1.58 ± 0.37 mm) compared with the more complex plans (medium: 1.82 ± 0.71 mm; difficult: 1.70 ± 0.45 mm). The mean error for the anterior reference point was lower (1.33 ± 0.58 mm) than those for both the posterior right (1.72 ± 0.24 mm) and posterior left points (2.05 ± 0.47 mm). No significant difference in terms of error was noticed among operators, despite variations in surgical experience. Feedback from surgeons was acceptable; all tests were completed within 15 min and the tool was considered to be both comfortable and usable in practice. CONCLUSION We used a new localiser-free, head-mounted, wearable, stereoscopic, video see-through display to develop a useful strategy affording surgeons access to augmented reality information. Our device appears to be accurate when used to assist in waferless maxillary repositioning. Our results suggest that the method can potentially be extended for use with many surgical procedures on the facial skeleton. Further, our positive results suggest that it would be appropriate to proceed to in vivo testing to assess surgical accuracy under real clinical conditions.

Development of a novel algorithm for human fall detection using wearable sensors

A novel algorithm for human fall detection by means of a tri-axial accelerometer, is described. A module constituted by the accelerometer and an on board processing unit was designed and realized. The system is conceived to be used in a multi-sensor network context for the remote monitoring of personnel working in very severe conditions (firefighters and civil protection operators). In the real application the module is thought to be integrated in the operator uniform collar. The algorithm is based on the detection of a critical trunk inclination in correspondence of an high rotational velocity. A Kalman filter was designed in order to separate the signal component due to gravity (i.e useful to extract the subject orientation) from the one due to the system acceleration. In comparison with the existing solutions the realized algorithm presents many advantages: no training is needed, low computational costs, fast time response and good performances also during critical activities (e.g jumping, running).

Augmented reality visualization of deformable tubular structures for surgical simulation

Surgical simulation based on augmented reality (AR), mixing the benefits of physical and virtual simulation, represents a step forward in surgical training. However, available systems are unable to update the virtual anatomy following deformations impressed on actual anatomy.

Human-PnP: Ergonomic AR Interaction Paradigm for Manual Placement of Rigid Bodies

The human perception of the three-dimensional world is influenced by the mutual integration of physiological and psychological depth cues, whose complexity is still an unresolved issue per se. Even more so if we wish to mimic the perceptive efficiency of the human visual system within augmented reality (AR) based surgical navigation systems. In this work we present a novel and ergonomic AR interaction paradigm that aids the manual placement of a non-tracked rigid body in space by manually minimizing the reprojection residuals between a set of corresponding virtual and real feature points. Our paradigm draws its inspiration from the general problem of estimating camera pose from a set of n-correspondences, i.e. perspective-n-point problem. In a recent work, positive results were achieved in terms of geometric error by applying the proposed strategy on the validation of a wearable AR system to aid manual maxillary repositioning.

[Poster] HMD Video see though AR with unfixed cameras vergence

Stereoscopic video see though AR systems permit accurate marker video based registration. To guarantee accurate registration, cameras are normally rigidly blocked while the user could require changing their vergence. We propose a solution working with lightweight hardware that, without the need for a new calibration of the cameras relative pose after each vergence adjustment, guarantees registration accuracy using pre-determined calibration data.

A new head-mounted display-based augmented reality system in neurosurgical oncology: a study on phantom

Abstract Purpose: Benefits of minimally invasive neurosurgery mandate the development of ergonomic paradigms for neuronavigation. Augmented Reality (AR) systems can overcome the shortcomings of commercial neuronavigators. The aim of this work is to apply a novel AR system, based on a head-mounted stereoscopic video see-through display, as an aid in complex neurological lesion targeting. Effectiveness was investigated on a newly designed patient-specific head mannequin featuring an anatomically realistic brain phantom with embedded synthetically created tumors and eloquent areas. Materials and methods: A two-phase evaluation process was adopted in a simulated small tumor resection adjacent to Broca’s area. Phase I involved nine subjects without neurosurgical training in performing spatial judgment tasks. In Phase II, three surgeons were involved in assessing the effectiveness of the AR-neuronavigator in performing brain tumor targeting on a patient-specific head phantom. Results: Phase I revealed the ability of the AR scene to evoke depth perception under different visualization modalities. Phase II confirmed the potentialities of the AR-neuronavigator in aiding the determination of the optimal surgical access to the surgical target. Conclusions: The AR-neuronavigator is intuitive, easy-to-use, and provides three-dimensional augmented information in a perceptually-correct way. The system proved to be effective in guiding skin incision, craniotomy, and lesion targeting. The preliminary results encourage a structured study to prove clinical effectiveness. Moreover, our testing platform might be used to facilitate training in brain tumour resection procedures.

Augmented Reality System for Freehand Guide of Magnetic Endovascular Devices

Magnetic guide of endovascular devices or magnetized therapeutic microparticles to the specific target in the arterial tree is increasingly studied, since it could improve treatment efficacy and reduce side effects. Most proposed systems use external permanent magnets attached to robotic manipulators or magnetic resonance imaging (MRI) systems to guide internal carriers to the region of treatment. We aim to simplify this type of procedures, avoiding or reducing the need of robotic arms and MRI systems in the surgical scenario. On account of this we investigated the use of a wearable stereoscopic video see-through augmented reality system to show the hidden vessel to the surgeon; in this way, the surgeon is able to freely move the external magnet, following the showed path, to lead the endovascular magnetic device towards the desired position. In this preliminary study, we investigated the feasibility of such an approach trying to guide a magnetic capsule inside a vascular mannequin. The high rate of success and the positive evaluation provided by the operators represent a good starting point for further developments of the system.

A sensorized glove for hand rehabilitation

Sensorized garments represent a breakthrough in wearable sensor technology. Sensorized garments have potential for the development of exciting new applications in rehabilitation. In this paper, we present our work toward the development of a system based on a sensorized glove that tracks hand movements during hand grasp/release tasks. The project aims at developing technology to facilitate the implementation of rehabilitation protocols focused on improving hand function. Preliminary results presented in this paper show that the sensorized glove can reliably track hand aperture during static and dynamic tasks.

Application of a New Wearable Augmented Reality Video See-Through Display to Aid Percutaneous Procedures in Spine Surgery

In mini-invasive surgery, the surgeon operates without a direct visualization of the patient’s anatomy. In image-guided surgery, solutions based on augmented reality (AR) represent the most promising ones. The aim of this study was to evaluate the efficacy of a new wearable AR system as aid in the performance of percutaneous procedures in spine surgery. Our solution is based on a video see-through head mounted display (HMD) and it allows the augmentation of video frames acquired by two external cameras with the rendering of patient-specific 3D models obtained elaborating radiological images. We tested the system on an in vitro setup intended to simulate the reaching of a lumbar pedicle. An experienced surgeon performed the percutaneous task wearing the HMD. System accuracy was evaluated through post-operative CT scan, measuring the maximum distance between the planned and obtained trajectories inside the pedicle canal. The mean insertion error was of 1.18 ±0.16 mm.

Neural correlates of human-robot handshaking

Handshaking represents a complex motor and cognitive task that poses several challenges from both engineering and neuroscientific viewpoints. In particular, it is an intriguing application which can be profitably studied in the field of Human Robot Interaction (HRI). In this work an experimental paradigm is proposed to investigate the neural correlates of handshaking between humans and between humans and robots using functional Magnetic Resonance Imaging. More specifically the role of visual and haptic components during handshaking interaction will be studied. A wearable sensing glove will be used to monitor hand finger position and movement. Preliminary results will be reported and discussed.