Computer Science

Point cloud upsampling is vital for the quality of the mesh in three-dimensional reconstruction. Recent research on point cloud upsampling has achieved great success due to the development of deep learning. However, the existing methods regard point cloud upsampling of different scale factors as independent tasks. Thus, the methods need to train a specific model for each scale factor, which is both inefficient and impractical for storage and computation in real applications. To address this limitation, in this work, we propose a novel method called ``Meta-PU" to firstly support point cloud upsampling of arbitrary scale factors with a single model. In the Meta-PU method, besides the backbone network consisting of residual graph convolution (RGC) blocks, a meta-subnetwork is learned to adjust the weights of the RGC blocks dynamically, and a farthest sampling block is adopted to sample different numbers of points. Together, these two blocks enable our Meta-PU to continuously upsample the point cloud with arbitrary scale factors by using only a single model. In addition, the experiments reveal that training on multiple scales simultaneously is beneficial to each other. Thus, Meta-PU even outperforms the existing methods trained for a specific scale factor only.

Augmented reality (AR) has the potential to revolutionize the way in which information is presented by overlaying virtual information onto a person's direct view of their real-time surroundings. By placing the display on the surface of the eye, a contact lens display (CLD) provides a versatile solution for compact AR. However, an unaided human eye cannot visualize patterns on the CLD simply because of the limited accommodation of the eye. Here, we introduce a holographic display technology that casts virtual information directly to the retina so that the eye sees it while maintaining the visualization of the real-world intact. The key to our design is to introduce metasurfaces to create a phase distribution that projects virtual information in a pixel-by-pixel manner. Unlike conventional holographic techniques, our metasurface-based technique is able to display arbitrary patterns using a single passive hologram. With a small form-factor, the designed metasurface empowers near-eye AR excluding the need of extra optical elements, such as a spatial light modulator, for dynamic image control.

We propose MetroSets, a new, flexible online tool for visualizing set systems using the metro map metaphor. We model a given set system as a hypergraph H=(V,S) , consisting of a set V of vertices and a set S , which contains subsets of V called hyperedges. Our system then computes a metro map representation of H , where each hyperedge E in S corresponds to a metro line and each vertex corresponds to a metro station. Vertices that appear in two or more hyperedges are drawn as interchanges in the metro map, connecting the different sets. MetroSets is based on a modular 4-step pipeline which constructs and optimizes a path-based hypergraph support, which is then drawn and schematized using metro map layout algorithms. We propose and implement multiple algorithms for each step of the MetroSet pipeline and provide a functional prototype with easy-to-use preset configurations. Furthermore, using several real-world datasets, we perform an extensive quantitative evaluation of the impact of different pipeline stages on desirable properties of the generated maps, such as octolinearity, monotonicity, and edge uniformity.

Complex 3D curves can be created by directly drawing mid-air in immersive environments (Augmented and Virtual Realities). Drawing mid-air strokes precisely on the surface of a 3D virtual object, however, is difficult; necessitating a projection of the mid-air stroke onto the user "intended" surface curve. We present the first detailed investigation of the fundamental problem of 3D stroke projection in VR. An assessment of the design requirements of real-time drawing of curves on 3D objects in VR is followed by the definition and classification of multiple techniques for 3D stroke projection. We analyze the advantages and shortcomings of these approaches both theoretically and via practical pilot testing. We then formally evaluate the two most promising techniques spraycan and mimicry with 20 users in VR. The study shows a strong qualitative and quantitative user preference for our novel stroke mimicry projection algorithm. We further illustrate the effectiveness and utility of stroke mimicry, to draw complex 3D curves on surfaces for various artistic and functional design applications.

Mode surfaces are the generalization of degenerate curves and neutral surfaces, which constitute 3D symmetric tensor field topology. Efficient analysis and visualization of mode surfaces can provide additional insight into not only degenerate curves and neutral surfaces, but also how these features transition into each other. Moreover, the geometry and topology of mode surfaces can help domain scientists better understand the tensor fields in their applications. Existing mode surface extraction methods can miss features in the surfaces. Moreover, the mode surfaces extracted from neighboring cells have gaps, which make their subsequent analysis difficult. In this paper, we provide novel analysis on the topological structures of mode surfaces, including a common parameterization of all mode surfaces of a tensor field using 2D asymmetric tensors. This allows us to not only better understand the structures in mode surfaces and their interactions with degenerate curves and neutral surfaces, but also develop an efficient algorithm to seamlessly extract mode surfaces, including neutral surfaces. The seamless mode surfaces enable efficient analysis of their geometric structures, such as the principal curvature directions. We apply our analysis and visualization to a number of solid mechanics data sets.

We present a data-driven algorithm for generating gaits of virtual characters with varying dominance traits. Our formulation utilizes a user study to establish a data-driven dominance mapping between gaits and dominance labels. We use our dominance mapping to generate walking gaits for virtual characters that exhibit a variety of dominance traits while interacting with the user. Furthermore, we extract gait features based on known criteria in visual perception and psychology literature that can be used to identify the dominance levels of any walking gait. We validate our mapping and the perceived dominance traits by a second user study in an immersive virtual environment. Our gait dominance classification algorithm can classify the dominance traits of gaits with ~73% accuracy. We also present an application of our approach that simulates interpersonal relationships between virtual characters. To the best of our knowledge, ours is the first practical approach to classifying gait dominance and generate dominance traits in virtual characters.

We give a method for modeling solid objects undergoing large spatially varying and/or anisotropic strains, and use it to reconstruct human anatomy from medical images. Our novel shape deformation method uses plastic strains and the Finite Element Method to successfully model shapes undergoing large and/or anisotropic strains, specified by sparse point constraints on the boundary of the object. We extensively compare our method to standard second-order shape deformation methods, variational methods and surface-based methods and demonstrate that our method avoids the spikiness, wiggliness and other artefacts of previous methods. We demonstrate how to perform such shape deformation both for attached and un-attached ("free flying") objects, using a novel method to solve linear systems with singular matrices with a known nullspace. While our method is applicable to general large-strain shape deformation modeling, we use it to create personalized 3D triangle and volumetric meshes of human organs, based on MRI or CT scans. Given a medically accurate anatomy template of a generic individual, we optimize the geometry of the organ to match the MRI or CT scan of a specific individual. Our examples include human hand muscles, a liver, a hip bone, and a gluteus medius muscle ("hip abductor").

We present a modular differentiable renderer design that yields performance superior to previous methods by leveraging existing, highly optimized hardware graphics pipelines. Our design supports all crucial operations in a modern graphics pipeline: rasterizing large numbers of triangles, attribute interpolation, filtered texture lookups, as well as user-programmable shading and geometry processing, all in high resolutions. Our modular primitives allow custom, high-performance graphics pipelines to be built directly within automatic differentiation frameworks such as PyTorch or TensorFlow. As a motivating application, we formulate facial performance capture as an inverse rendering problem and show that it can be solved efficiently using our tools. Our results indicate that this simple and straightforward approach achieves excellent geometric correspondence between rendered results and reference imagery.

The brachial plexus is a complex network of peripheral nerves that enables sensing from and control of the movements of the arms and hand. Nowadays, the coordination between the muscles to generate simple movements is still not well understood, hindering the knowledge of how to best treat patients with this type of peripheral nerve injury. To acquire enough information for medical data analysis, physicians conduct motion analysis assessments with patients to produce a rich dataset of electromyographic signals from multiple muscles recorded with joint movements during real-world tasks. However, tools for the analysis and visualization of the data in a succinct and interpretable manner are currently not available. Without the ability to integrate, compare, and compute multiple data sources in one platform, physicians can only compute simple statistical values to describe patient's behavior vaguely, which limits the possibility to answer clinical questions and generate hypotheses for research. To address this challenge, we have developed \systemname, an interactive visual analytics system which provides an efficient framework to extract and compare muscle activity patterns from the patient's limbs and coordinated views to help users analyze muscle signals, motion data, and video information to address different tasks. The system was developed as a result of a collaborative endeavor between computer scientists and orthopedic surgery and rehabilitation physicians. We present case studies showing physicians can utilize the information displayed to understand how individuals coordinate their muscles to initiate appropriate treatment and generate new hypotheses for future research.

The analysis of human motion opens up a wide range of possibilities, such as realistic training simulations or authentic motions in robotics or animation. One of the problems underlying motion analysis is the meaningful comparison of actions based on similarity measures. Since the motion analysis is application-dependent, it is essential to find the appropriate motion similarity method for the particular use case. This state of the art report provides an overview of human motion analysis and different similarity modeling methods, while mainly focusing on approaches that work with 3D motion data. The survey summarizes various similarity aspects and features of motion and describes approaches to measuring the similarity between two actions.