Kanchan Bahirat

Augmented reality-based exergames for rehabilitation

Rehabilitation for stroke afflicted patients, through exercises tailored for individual needs, aims at relearning basic motor skills, especially in the extremities. Rehabilitation through Augmented Reality (AR) based games engage and motivate patients to perform exercises which, otherwise, maybe boring and monotonic. Also, mirror therapy allows users to observe ones own movements in the game providing them with good visual feedback. This paper presents an augmented reality based system for rehabilitation by playing four interactive, cognitive and fun Exergames (exercise and gaming). The system uses low-cost RGB-D cameras such as Microsoft Kinect V2 to capture and generate 3D model of the person by extracting him/her from the entire captured data and immersing it in different interactive virtual environments. Animation based limb movement enhancement along with cognitive aspects incorporated in the game can help in positive reinforcement, progressive challenges and motion improvement. Recording module of the toolkit allows future reference and facilitates feedback from the physician. 10 able-bodied users, 2 psychological experts and 2 Physical Medicine and Rehabilitation physicians evaluated the user experience and usability aspects of the exergames. Results obtained shows the games to be fun and realistic, and at the same time engaging and motivating for performing exercises.

Evaluating the efficacy of RGB-D cameras for surveillance

RGB-D cameras have enabled real-time 3D video processing for numerous computer vision applications, especially for surveillance type applications. In this paper, we first present a real-time anti-forensic 3D object stream manipulation framework to capture and manipulate live RBG-D data streams to create realistic images/videos showing individuals performing activities they did not actually do. The framework uses computer vision and graphics methods to render photorealistic animations of live mesh models captured using the camera. Next, we conducted a visual inspection of the manipulated RGB-D streams (just like security personnel would do) by users who are computer vision and graphics scientists. The study shows that it was significantly difficult to distinguish between the real or reconstructed rendering of such 3D video sequences, thus clearly showing the potential security risk involved. Finally, we investigate the efficacy of forensic approaches for detecting such manipulations.

Network Adaptive Textured Mesh Generation for Collaborative 3D Tele-Immersion

3D Tele-Immersion (3DTI) has emerged as an efficient environment for virtual interactions and collaborations in a variety of fields like rehabilitation, education, gaming, etc. In 3DTI, geographically distributed users are captured using multiple cameras and immersed in a single virtual environment. The quality of experience depends on the available network bandwidth, quality of the 3D model generated and the time taken for rendering. In a collaborative environment, achieving high quality, high frame rate rendering by transmitting data to multiple sites having different bandwidth is challenging. In this paper we introduce a network adaptive textured mesh generation scheme to transmit varying quality data based on the available bandwidth. To reduce the volume of information transmitted, a visual quality based vertex selection approach is used to generate a sparse representation of the user. This sparse representation is then transmitted to the receiver side where a sweep-line based technique is used to generate a 3D mesh of the user. High visual quality is maintained by transmitting a high resolution texture image compressed using a lossy compression algorithm. In our studies users were unable to notice visual quality variations of the rendered 3D model even at 90% compression.

Learning-based objective evaluation of 3D human open meshes

Current state-of-the-art mesh quality measures evaluate closed and complete meshes obtained after mesh postprocessing applications, such as mesh simplification or watermarking, and compare them against the corresponding reference mesh. Emerging 3D immersive VR/AR applications use noisy 3D point cloud, typically from single RGB-D camera (such as Microsofts Kinect) to generate standalone (no reference) 3D human open mesh (with boundaries) in real time, that needs evaluation. A learning-based objective measure is proposed to rate the visual quality by emulating human perception of 3D human open mesh quality. 2-pronged objective evaluation is performed: (a) Global holistic score captures the efficacy of the mesh to represent the human model as a whole, by considering mesh completeness and mesh noise. (b) Local part-based score caters to the need of varying roughness in different parts of the human body, by finding the deviation in the face normals for all the adjacent triangles in that part (segment). Learning technique aligns the objective scores with the subjective user evaluation, in turn combining the concepts of white-box and black-box evaluation for 3D meshes. Experimental results for a database, specifically generated for the purpose proves the efficacy of the proposed method.

A study on lidar data forensics

3D LiDAR (Light Imaging Detection and Ranging) data has recently been used in a wide range of applications such as vehicle automation and crime scene reconstruction. Decision making in such applications is highly dependent on LiDAR data. Thus, it becomes crucial to authenticate the data before using it. Though authentication of 2D digital images and video has been widely studied, the area of 3D data forensic is relatively unexplored. In this paper, we investigate and identify three possible attacks on the LiDAR data. We also propose two novel forensic approaches as a countermeasure for such attacks and study their effectiveness. The first forensic approach utilises the density consistency check while the second method leverages the occlusion effect for revealing the forgery. Experimental results demonstrate the effectiveness of the proposed forgery attacks and raise the awareness against unauthenticated use of LiDAR data. The performance analyses of the proposed forensic approaches indicate that the proposed methods are very efficient and provide the detection accuracy of more than 95% for certain kinds of forgery attacks. While the forensic approach is unable to handle all forgery attacks, the study motivates to explore more sophisticated forensic methods for LiDAR data.

A Boundary and Texture Preserving Mesh Simplification Algorithm for Virtual Reality

With the increasing accessibility of the mobile head-mounted displays (HMDs), mobile virtual reality (VR) systems are finding applications in various areas. However, mobile HMDs are highly constrained with limited graphics processing units (GPUs), low processing power and onboard memory. Hence, VR developers must be cognizant of the number of polygons contained within their virtual environments to avoid rendering at low frame rates and inducing simulator sickness. The most robust and rapid approach to keeping the overall number of polygons low is to use mesh simplification algorithms to create low-poly versions of preexisting, high-poly models. Unfortunately, most existing mesh simplification algorithms cannot adequately handle meshes with lots of boundaries or non-manifold meshes, which are common attributes of 3D models made with computer-aided design tools.; AB@In this paper, we present a high-fidelity mesh simplification algorithm specifically designed for VR. This new algorithm, QEM4VR, addresses the deficiencies of prior quadric error metric (QEM) approaches by leveraging the insight that the most relevant boundary edges lie along curvatures while linear boundary edges can be collapsed. Additionally, our QEM4VR algorithm preserves key surface properties, such as normals, texture coordinates, colors, and materials. It pre-processes the 3D models and generate their low-poly approximations offline. We used six publicly available, high-poly models, with and without textures to compare the accuracy and fidelity of our QEM4VR algorithm to previous QEM variations. We also performed a frame rate analysis with original high-poly models and low-poly models obtained using QEM4VR and previous QEM variations. Our results indicate that QEM4VR creates low-poly, high-fidelity virtual environments for VR applications on devices that are constrained by the low number of polygons they can work with.

Mr.MAPP: Mixed Reality for MAnaging Phantom Pain

Phantom Limb Pain or simply, Phantom Pain is a severe chronic pain that is experienced as a vivid sensation of the pain in missing body part. Epidemiological studies obtained from a large samples indicate that the short-term incidence rate of the phantom limb pain is 72% [13], while long-term incidence rate (6 months after amputation) is 67%, [5, 13]. A wide spectrum of treatments developed for alleviating phantom limb pain includes the traditional mirror box therapy as well as recently developed virtual reality-based methods. Most of the virtual reality-based methods rely on 3D CAD models of the virtual limb, animating them using the motion data acquired either from patients existing anatomical limb or myoelectric activity at patients stump (of the amputated limb). Since motion activity is typically captured using body sensors (Electromyography, EMG, or inertial sensors), these methods are considered as invasive approaches. Further, in the case of virtual reality-based methods, the dependency on the pre-built 3D models degrades the immersive experience due to a mismatch in the skin color, clothes, artificial and rigid look and misalignment of the phantom limb. In this work, we propose a novel Mixed Reality based system for MAnaging Phantom Pain (Mr.MAPP), utilizing off-the-shelf RGB-D cameras such as Microsoft Kinect V2 to capture and generate a 3D model of the patient in real-time. An illusion of the virtual limb is crafted in real-time by mirroring the patients symmetric anatomical limb in the captured data with the help of various computer vision and graphics techniques. Along with that, a phantom limb skeleton is also generated in real-time to enable interaction with virtual objects. We conducted a multi-pronged user Quality of Experience (QoE) study of Mr.MAPP employing various rendering displays such as 3D Television, and Head mounted displays (Oculus Rift, Samsung Gear VR). The user study involved two classes of users: (a) a big pool of Subject-Matter Experts (SMEs) that includes Physical Medicine and Rehab (PM&R) experts, Amputee Occupational Therapist and Doctors of Chiropractic; (b) healthy non-expert users. SMEs, as well as the healthy non-expert users provided a very positive feedback of Mr. MAPP indicating the potential value of Mr.MAPP for the phantom pain management.

ALERT: Adding a Secure Layer in Decision Support for Advanced Driver Assistance System (ADAS)

With the ever-increasing popularity of LiDAR (Light Image Detection and Ranging) sensors, a wide range of applications such as vehicle automation and robot navigation are developed utilizing the 3D LiDAR data. Many of these applications involve remote guidance - either for safety or for the task performance - of these vehicles and robots. Research studies have exposed vulnerabilities of using LiDAR data by considering different security attack scenarios. Considering the security risks associated with the improper behavior of these applications, it has become crucial to authenticate the 3D LiDAR data that highly influence the decision making in such applications. In this paper, we propose a framework, ALERT (Authentication, Localization, and Estimation of Risks and Threats), as a secure layer in the decision support system used in the navigation control of vehicles and robots. To start with, ALERT tamper-proofs 3D LiDAR data by employing an innovative mechanism for creating and extracting a dynamic watermark. Next, when tampering is detected (because of the inability to verify the dynamic watermark), ALERT then carries out cross-modal authentication for localizing the tampered region. Finally, ALERT estimates the level of risk and threat based on the temporal and spatial nature of the attacks on LiDAR data. This estimation of risk and threats can then be incorporated into the decision support system used by ADAS (Advanced Driver Assistance System). We carried out several experiments to evaluate the efficacy of the proposed ALERT for ADAS and the experimental results demonstrate the effectiveness of the proposed approach.

Real-Time, Curvature-Sensitive Surface Simplification Using Depth Images

With the rising popularity of handheld virtual reality (VR) devices and depth sensing RGB-D cameras, a variety of VR applications merging these two technologies has been suggested. However, immersive quality of experience in such VR applications is constrained mainly by the large data size and the hardware limitations to handle it. The depth data captured by RGB-D cameras provide a dense sampling of the surface, resulting in a high-poly mesh, which is difficult to be rendered on handheld VR devices due to their limited processing power. To improve the immersive VR experience, a sparse approximation of the depth data is needed. Traditional mesh and point cloud simplification methods are iterative and so are unsuitable for real-time applications. In this paper, we introduce a depth-image-based approach that is capable of generating a good quality sparse mesh for visualization in real time. We propose a curvature-sensitive surface simplification—

Designing and Evaluating a Mesh Simplification Algorithm for Virtual Reality

\text{CS}^3