Computer Science

This paper presents a fast projection mapping method for moving image content projected onto a markerless planar surface using a low-latency Digital Micromirror Device (DMD) projector. By adopting a closed-loop alignment approach, in which not only the surface texture but also the projected image is tracked by a camera, the proposed method is free from a calibration or position adjustment between the camera and projector. We designed fiducial patterns to be inserted into a fast flapping sequence of binary frames of the DMD projector, which allows the simultaneous tracking of the surface texture and a fiducial geometry separate from a single image captured by the camera. The proposed method implemented on a CPU runs at 400 fps and enables arbitrary video contents to be "stuck" onto a variety of textured surfaces.

3D animation of humans in action is quite challenging as it involves using a huge setup with several motion trackers all over the person's body to track the movements of every limb. This is time-consuming and may cause the person discomfort in wearing exoskeleton body suits with motion sensors. In this work, we present a trivial yet effective solution to generate 3D animation of multiple persons from a 2D video using deep learning. Although significant improvement has been achieved recently in 3D human pose estimation, most of the prior works work well in case of single person pose estimation and multi-person pose estimation is still a challenging problem. In this work, we firstly propose a supervised multi-person 3D pose estimation and animation framework namely AnimePose for a given input RGB video sequence. The pipeline of the proposed system consists of various modules: i) Person detection and segmentation, ii) Depth Map estimation, iii) Lifting 2D to 3D information for person localization iv) Person trajectory prediction and human pose tracking. Our proposed system produces comparable results on previous state-of-the-art 3D multi-person pose estimation methods on publicly available datasets MuCo-3DHP and MuPoTS-3D datasets and it also outperforms previous state-of-the-art human pose tracking methods by a significant margin of 11.7% performance gain on MOTA score on Posetrack 2018 dataset.

The majority of methods for line clipping make a rather large number of comparisons and involve a lot of calculations compared to modern ones. Most of the times, they are not so efficient as well as not so simple and applicable to the majority of cases. Besides the most popular ones, namely, Cohen-Sutherland, Liang-Barsky, Cyrus-Beck and Nicholl-Lee-Nicholl, other line-clipping methods have been presented over the years, each one having its own advantages and disadvantages. In this paper a new computation method for 2D line clipping against a rectangular window is introduced. The proposed method has been compared with the afore-mentioned ones as well as with two others; namely, Skala and Kodituwakku-Wijeweera-Chamikara, with respect to the number of operations performed and the computation time. The performance of the proposed method has been found to be better than all of the above-mentioned methods and it is found to be very fast, simple and can be implemented easily in any programming language or integrated development environment.

This paper describes a pipeline built with open source tools for interpolating 3D facial expressions taken from images. The presented approach allows anyone to create 3D face animations from 2 input photos: one from the start face expression, and the other from the final face expression. Given the input photos, corresponding 3D face models are constructed and texture-mapped using the photos as textures aligned with facial features. Animations are then generated by morphing the models by interpolation of the geometries and textures of the models. This work was performed as a MS project at the University of California, Merced.

This paper presents novel and efficient strategies to spatially adapt the amount of computational effort applied based on the local dynamics of a free surface flow, for both classic weakly compressible SPH (WCSPH) and predictive-corrective incompressible SPH (PCISPH). Using a convenient and readily parallelizable block-based approach, different regions of the fluid are assigned differing time steps and solved at different rates to minimize computational cost. Our approach for WCSPH scheme extends an asynchronous SPH technique from compressible flow of astrophysical phenomena to the incompressible free surface setting, and further accelerates it by entirely decoupling the time steps of widely spaced particles. Similarly, our approach to PCISPH adjusts the the number of iterations of density correction applied to different regions, and asynchronously updates the neighborhood regions used to perform these corrections; this sharply reduces the computational cost of slowly deforming regions while preserving the standard density invariant. We demonstrate our approaches on a number of highly dynamic scenarios, demonstrating that they can typically double the speed of a simulation compared to standard methods while achieving visually consistent results.

Virtual Environments (VEs) provide the opportunity to simulate a wide range of applications, from training to entertainment, in a safe and controlled manner. For applications which require realistic representations of real world environments, the VEs need to provide multiple, physically accurate sensory stimuli. However, simulating all the senses that comprise the human sensory system (HSS) is a task that requires significant computational resources. Since it is intractable to deliver all senses at the highest quality, we propose a resource distribution scheme in order to achieve an optimal perceptual experience within the given computational budgets. This paper investigates resource balancing for multi-modal scenarios composed of aural, visual and olfactory stimuli. Three experimental studies were conducted. The first experiment identified perceptual boundaries for olfactory computation. In the second experiment, participants (N=25) were asked, across a fixed number of budgets (M=5), to identify what they perceived to be the best visual, acoustic and olfactory stimulus quality for a given computational budget. Results demonstrate that participants tend to prioritise visual quality compared to other sensory stimuli. However, as the budget size is increased, users prefer a balanced distribution of resources with an increased preference for having smell impulses in the VE. Based on the collected data, a quality prediction model is proposed and its accuracy is validated against previously unused budgets and an untested scenario in a third and final experiment.

We propose an image warping-based remote rendering technique for volumes that decouples the rendering and display phases. Our work builds on prior work that samples the volume on the client using ray casting and reconstructs a z-value based on some heuristic. The color and depth buffer are then sent to the client that reuses this depth image as a stand-in for subsequent frames by warping it according to the current camera position until new data was received from the server. We augment that method by implementing the client renderer using ray tracing. By representing the pixel contributions as spheres, this allows us to effectively vary their footprint based on the distance to the viewer, which we find to give better results than point-based rasterization when applied to volumetric data sets.

Digital mosaics have usually used regular tiles, simulating the historical "tessellated" mosaics. In this paper, we present a method for synthesizing pebble mosaics, a historical mosaic style in which the tiles are rounded pebbles. We address both the tiling problem, where pebbles are distributed over the image plane so as to approximate the input image content, and the problem of geometry, creating a smooth rounded shape for each pebble. We adapt SLIC, simple linear iterative clustering, to obtain elongated tiles conforming to image content, and smooth the resulting irregular shapes into shapes resembling pebble cross-sections. Then, we create an interior and exterior contour for each pebble and solve a Laplace equation over the region between them to obtain height-field geometry. The resulting pebble set approximates the input image while presenting full geometry that can be rendered and textured for a highly detailed representation of a pebble mosaic.

Accurate segmentation of connective soft tissues is still a challenging task, which hinders the generation of corresponding geometric models for biomechanical computations. Alternatively, one could predict ligament insertion sites and then approximate the shapes, based on anatomical knowledge and morphological studies. Here, we describe a corresponding integrated framework for the automatic modelling of human musculoskeletal ligaments. We combine statistical shape modelling with geometric algorithms to automatically identify insertion sites, based on which geometric surface and volume meshes are created. For demonstrating a clinical use case, the framework has been applied to generate models of the interosseous membrane in the forearm. For the adoption to the forearm anatomy, ligament insertion sites in the statistical model were defined according to anatomical predictions following an approach proposed in prior work. For evaluation we compared the generated sites, as well as the ligament shapes, to data obtained from a cadaveric study, involving five forearms with a total of 15 ligaments. Our framework permitted the creation of 3D models approximating ligaments' shapes with good fidelity. However, we found that the statistical model trained with the state-of-the-art prediction of the insertion sites was not always reliable. Using that model, average mean square errors as well as Hausdorff distances of the meshes increased by more than one order of magnitude, as compared to employing the known insertion locations of the cadaveric study. Using the latter an average mean square error of 0.59 mm and an average Hausdorff distance of less than 7 mm resulted, for the complete set of ligaments. In conclusion, the presented approach for generating ligament shapes from insertion points appears to be feasible but the detection of the insertion sites with a SSM is too inaccurate.

We present a user-friendly image editing system that supports a drag-and-drop object insertion (where the user merely drags objects into the image, and the system automatically places them in 3D and relights them appropriately), post-process illumination editing, and depth-of-field manipulation. Underlying our system is a fully automatic technique for recovering a comprehensive 3D scene model (geometry, illumination, diffuse albedo and camera parameters) from a single, low dynamic range photograph. This is made possible by two novel contributions: an illumination inference algorithm that recovers a full lighting model of the scene (including light sources that are not directly visible in the photograph), and a depth estimation algorithm that combines data-driven depth transfer with geometric reasoning about the scene layout. A user study shows that our system produces perceptually convincing results, and achieves the same level of realism as techniques that require significant user interaction.