Vijay Pawar

Interaction with co-located haptic feedback in virtual reality

This paper outlines a study into the effects of co-location (the term ‘co-location’ is used throughout to refer to the co-location of haptic and visual sensory modes, except where otherwise specified) of haptic and visual sensory modes in VR simulations. The study hypothesis is that co-location of these sensory modes will lead to improved task performance within a VR environment. Technical challenges and technological limitations are outlined prior to a description of the implementation adopted for this study. Experiments were conducted to evaluate the effect on user performance of co-located haptics (force feedback) in a 3D virtual environment. Results show that co-location is an important factor, and when coupled with haptic feedback the performance of the user is greatly improved.

In-vivo high resolution AFM topographic imaging of Caenorhabditis elegans reveals previously unreported surface structures of cuticle mutants

Atomic force microscopy (AFM) is a powerful method for topographic imaging of surfaces with nanometer resolution. AFM offers significant advantages over scanning electron microscopy (SEM) including the acquisition of quantitative 3D-images and biomechanical information. More importantly, for in-vivo biological imaging, AFM does not require sample dehydration/labeling. We show for the first time high-resolution topographical images of the cuticle of the model organism C. elegans under physiological conditions using AFM. C. elegans is used extensively for drug screening and to study pathogen adherence in innate immunity; both applications highly depend on the integrity of the nematodes cuticle. Mutations affecting both drug adsorption and pathogen clearance have been proposed to relate to changes in the cuticle structure, but never visually examined in high resolution. In this study we use AFM to visualize the topography of wild-type adult C. elegans as well as several cuticle collagen mutants and describe previously unseen anatomical differences.

Hand-eye calibration for robotic assisted minimally invasive surgery without a calibration object

In a robot mounted camera arrangement, hand-eye calibration estimates the rigid relationship between the robot and camera coordinate frames. Most hand-eye calibration techniques use a calibration object to estimate the relative transformation of the camera in several views of the calibration object and link these to the forward kinematics of the robot to compute the hand-eye transformation. Such approaches achieve good accuracy for general use but for applications such as robotic assisted minimally invasive surgery, acquiring a calibration sequence multiple times during a procedure is not practical. In this paper, we present a new approach to tackle the problem by using the robotic surgical instruments as the calibration object with well known geometry from CAD models used for manufacturing. Our approach removes the requirement of a custom sterile calibration object to be used in the operating room and it simplifies the process of acquiring calibration data when the laparoscope is constrained to move around a remote centre of motion. This is the first demonstration of the feasibility to perform hand-eye calibration using components of the robotic system itself and we show promising validation results on synthetic data as well as data acquired with the da Vinci Research Kit.

Evaluating the Influence of Haptic Force-Feedback on 3D Selection Tasks using Natural Egocentric Gestures

Immersive Virtual Environments (IVEs) allow participants to interact with their 3D surroundings using natural hand gestures. Previous work shows that the addition of haptic feedback cues improves performance on certain 3D tasks. However, we believe this is not true for all situations. Depending on the difficulty of the task, we suggest that we should expect differences in the ballistic movement of our hands when presented with different types of haptic force-feedback conditions. We investigated how hard, soft and no haptic force-feedback responses, experienced when in contact with the surface of an object, affected user performance on a task involving selection of multiple targets. To do this, we implemented a natural egocentric selection interaction technique by integrating a two-handed large-scale force-feedback device in to a CAVETM-like IVE system. With this, we performed a user study where we show that participants perform selection tasks best when interacting with targets that exert soft haptic force-feedback cues. For targets that have hard and no force-feedback properties, we highlight certain associated hand movement that participants make under these conditions, that we hypothesise reduce their performance.

Saliency detection as a reactive process: unexpected sensory events evoke cortico-muscular coupling

Survival in a fast-changing environment requires animals not only to detect unexpected sensory events, but also to react. In humans, these salient sensory events generate large electrocortical responses, which have been traditionally interpreted within the sensory domain. Here we describe a basic physiological mechanism coupling saliency-related cortical responses with motor output. In four experiments conducted on 70 healthy participants, we show that salient substartle sensory stimuli modulate isometric force exertion by human participants, and that this modulation is tightly coupled with electrocortical activity elicited by the same stimuli. We obtained four main results. First, the force modulation follows a complex triphasic pattern consisting of alternating decreases and increases of force, time-locked to stimulus onset. Second, this modulation occurs regardless of the sensory modality of the eliciting stimulus. Third, the magnitude of the force modulation is predicted by the amplitude of the electrocortical activity elicited by the same stimuli. Fourth, both neural and motor effects are not reflexive but depend on contextual factors. Together, these results indicate that sudden environmental stimuli have an immediate effect on motor processing, through a tight corticomuscular coupling. These observations suggest that saliency detection is not merely perceptive but reactive, preparing the animal for subsequent appropriate actions. SIGNIFICANCE STATEMENT Salient events occurring in the environment, regardless of their modalities, elicit large electrical brain responses, dominated by a widespread “vertex” negative-positive potential. This response is the largest synchronization of neural activity that can be recorded from a healthy human being. Current interpretations assume that this vertex potential reflects sensory processes. Contrary to this general assumption, we show that the vertex potential is strongly coupled with a modulation of muscular activity that follows the same pattern. Both the vertex potential and its motor effects are not reflexive but strongly depend on contextual factors. These results reconceptualize the significance of these evoked electrocortical responses, suggesting that saliency detection is not merely perceptive but reactive, preparing the animal for subsequent appropriate actions.

A Continuum Robot and Control Interface for Surgical Assist in Fetoscopic Interventions

Twin–twin transfusion syndrome requires interventional treatment using a fetoscopically introduced laser to sever the shared blood supply between the fetuses. This is a delicate procedure relying on small instrumentation with limited articulation to guide the laser tip and a narrow field of view to visualize all relevant vascular connections. In this letter, we report on a mechatronic design for a comanipulated instrument that combines concentric tube actuation to a larger manipulator constrained by a remote centre of motion. A stereoscopic camera is mounted at the distal tip and used for imaging. Our mechanism provides enhanced dexterity and stability of the imaging device. We demonstrate that the imaging system can be used for computing geometry and enhancing the view at the operating site. Results using electromagnetic sensors for verification and comparison to visual odometry from the distal sensor show that our system is promising and can be developed further for multiple clinical needs in fetoscopic procedures.

Chromaticity based smoke removal in endoscopic images

In minimally invasive surgery, image quality is a critical pre-requisite to ensure a surgeons ability to perform a procedure. In endoscopic procedures, image quality can deteriorate for a number of reasons such as fogging due to the temperature gradient after intra-corporeal insertion, lack of focus and due to smoke generated when using electro-cautery to dissect tissues without bleeding. In this paper we investigate the use of vision processing techniques to remove surgical smoke and improve the clarity of the image. We model the image formation process by introducing a haze medium to account for the degradation of visibility. For simplicity and computational efficiency we use an adapted dark-channel prior method combined with histogram equalization to remove smoke artifacts to recover the radiance image and enhance the contrast and brightness of the final result. Our initial results on images from robotic assisted procedures are promising and show that the proposed approach may be used to enhance image quality during surgery without additional suction devices. In addition, the processing pipeline may be used as an important part of a robust surgical vision pipeline that can continue working in the presence of smoke.

Investigation of mechanosensation in C. elegans using light field calcium imaging

We describe a new experimental approach to investigate touch sensation in the model organism C. elegans using light field deconvolution microscopy. By combining fast volumetric image acquisition with controlled indentation of the organism using a high sensitivity force transducer, we are able to simultaneously measure activity in multiple touch receptor neurons expressing the calcium ion indicator GCaMP6s. By varying the applied mechanical stimulus we show how this method can be used to quantify touch sensitivity in C. elegans. We describe some of the challenges of performing light field calcium imaging in moving samples and demonstrate that they can be overcome by simple data processing.

Three-dimensional behavioural phenotyping of freely moving C. elegans using quantitative light field microscopy

Behavioural phenotyping of model organisms is widely used to investigate fundamental aspects of organism biology, from the functioning of the nervous system to the effects of genetic mutations, as well as for screening new drug compounds. However, our capacity to observe and quantify the full range and complexity of behavioural responses is limited by the inability of conventional microscopy techniques to capture volumetric image information at sufficient speed. In this article we describe how combining light field microscopy with computational depth estimation provides a new method for fast, quantitative assessment of 3D posture and movement of the model organism Caenorhabditis elegans (C. elegans). We apply this technique to compare the behaviour of cuticle collagen mutants, finding significant differences in 3D posture and locomotion. We demonstrate the ability of quantitative light field microscopy to provide new fundamental insights into C. elegans locomotion by analysing the 3D postural modes of a freely swimming worm. Finally, we consider relative merits of the method and its broader application for phenotypic imaging of other organisms and for other volumetric bioimaging applications.

Atomic force stiffness imaging: capturing differences in mechanical properties to identify and localize areas of prostate cancer tissue.

Prostate cancer is now the most commonly diagnosed cancer in men in western countries. Due to the difficulty for early detection, there are an estimated 10000 deaths a year in the UK from prostate cancer alone; whereby the only curative option is interventional treatment that aims to excise all diseased cells while preserving the neurovascular bundle. To date, several studies have shown that the mechanical properties of cancer cells and tissues i.e. adhesion, stiffness, roughness and viscoelasticity are significantly different from benign cells and regions of tissue that are healthy. Building upon these results, we believe novel methods of imaging the mechanical properties of prostate cancer samples can provide new surgical intervention opportunities beyond what is possible through vision alone. In this paper, we used an Atomic Force Microscope (AFM) to measure the stiffness and topography variations correlating to regions of prostate cancer at the surface of an excised sample at a cellular level. Preliminary results show that by using an AFM we can detect structural differences in non-homogeneous tissue samples, confirming previous results that cancerous tissues appear stiffer than benign areas. Through these results, we aim to develop a stiffness imaging protocol to aid the early detection of prostate cancer, in addition to force sensing surgical tools.