Huagen Wan

MIVAS: A Multi-Modal Immersive Virtual Assembly System

Evaluation and planning of assembly processes in virtual environments have become an active research area in engineering community. However, planning of complex assemblies in virtual environments, especially large-scale virtual environments, is still hindered by limitations like unnatural user interaction, insufficient frame rates, and deficiencies in processing of assembly constraints. In this paper, we present MIVAS, a Multi-modal Immersive Virtual Assembly System. By viewing the virtual assembly system as a finite state machine, we incorporate tracked devices, force feedback dataglove, voice commands, human sounds, fully immersive 4-sided CAVE, together with optimization techniques for both complex assembly models and assembly operations to provide for engineers an intuitive and natural way of assembly evaluation and planning. Testing scenarios on disassembling different components of an intelligent hydraulic excavator are described. Special attention is paid upon such technical issues as interface between CAD packages and the CAVE virtual environment, natural and intuitive user interaction including realistic virtual hand interaction and force feedback, intelligent navigation for assembly operations, and real-time display of complex assemblies.Copyright

Realistic virtual hand modeling with applications for virtual grasping

In virtual environments, virtual hand interactions play key roles in the human-computer interface. Specifically, the virtual grasping of 3D objects provides an intuitive way for users to interact with virtual objects. This paper demonstrates the creation of a sophisticated virtual hand model simulating natural anatomy in its appearance and motion. To achieve good visual realism, the virtual hand is modeled with metaball modeling, and visually enhanced by applying texture mapping. For realistic kinematics modeling, a three-layer model (skeleton, muscle and skin layers) is adopted to handle the motion as well as the deformation of the virtual hand. We also present an approach for virtual grasping of 3D objects with the realistic virtual hand driven by a CyberGlove dataglove. Grasping heuristics are proposed based on the classification with the shapes of objects, and simplified proxies for the virtual hand are used for the purpose of real-time collision detection between the virtual hand and 3D objects.

An approach to solid modeling in a semi-immersive virtual environment

Abstract Researches on using virtual reality (VR) to support computer-aided design are rapidly increasing in recent years. This is because VR techniques are very helpful in improving the interactivity and 3D visualization function of CAD systems. In this paper, we present an approach to solid modeling in a semi-immersive virtual environment. The approach allows users to create, manipulate and visualize a precise solid model by direct 3D manipulations and voice commands in a virtual environment with a 3D input device. In the approach, to make the solid model created and modified by direct 3D manipulations precise, real-time constraint recognition, propagation and solution algorithms are developed and utilized. An extended solid model is also adopted to effectively support the solid modeling by direct 3D manipulations. The experiment shows that the approach enables users to conduct solid modeling intuitively, effectively and accurately, thus can be used for both early stage design and detailed design.

Grasp Identification and Multi-Finger Haptic Feedback for Virtual Assembly

The sense of touch is an important way for humans to feel the world. It is very important to provide realistic haptic feedback in virtual assembly applications as to enhancing immersion experience and improving efficiency. This paper presents a novel approach for grasp identification and multi-finger haptic feedback for virtual assembly. Firstly, the Voxmap-PointShell (VPS) algorithm is adapted and utilized to detect collisions between a dexterous virtual hand and a mechanical component or between two mechanical components, and collision detection results are used to guide the motion of a virtual hand. Then collision forces at collision points are calculated (using Hooke Law), classified and converted. Finally, forces received at fingertips of a virtual hand are used to identify whether or not a virtual hand can grasp a mechanical component, and are mapped to exert forces at user’s fingertips with a CyberGrasp force feedback system. Our approach has been incorporated and verified in a CAVE virtual environment.Copyright

Vision-based hand interaction and its application in pervasive games

Pervasive games have become a popular field of investigation in recent years, in which natural human computer interaction (HCI) plays a key role. In this paper, a vision-based approach for human hand motion gesture recognition is proposed for natural HCI in pervasive games. A LED light pen is used to indicate the users hand position, while a web-camera is used to capture the hand motion. A rule-based approach is used to design a set of hand gestures which are classified into two categories: linear gestures and arc-shaped gestures. A determinate finite state automaton is developed to segment the captured hand motion trajectories. The proposed interaction method has been applied to the traditional game Tetris on PC with the hand-held LED light pen being used to drive the game instead of traditional key strokes. Experimental results show that the vision-based interactions are natural and effective.

IBCD: a fast collision detection algorithm based on image space using OBB

Image-based collision detection algorithms make efficient use of the graphics rendering hardware and reduce the computational cost of CPU. In this article, a fast collision detection algorithm based on image space is presented, which combines graphics hardware capabilities with a simplified geometric representation (oriented bounding box) in order to rapidly detect collisions between complex objects. The method can deal with arbitrary polyhedra, while preserving the merits of image-based collision detection algorithms. This is achieved by decomposing the surfaces of an object into a list of convex pieces. High efficiency of the algorithm is obtained by organizing the convex pieces into a binary tree with each node representing a convex piece, and by adopting triangle strip compression. The algorithm has been implemented and compared with related algorithms. Copyright # 2003 John Wiley & Sons, Ltd.

A realistic force rendering algorithm for CyberGrasp

Haptic rendering is a highly device-dependent process. Accordingly, haptic rendering algorithms should be designed regarding the specific physical features of a haptic device. The Immersion CyberGrasp is a force feedback device which can provide an individual force roughly perpendicular to the fingertip of each finger. In this paper, we propose a novel force rendering algorithm for the Immersion CyberGrasp considering the feature of area contacts between the virtual hand avatar and target geometric models. Line segment clusters are used as haptic proxies for the real-time collision detection between the fingertips and target geometric models. Force computations are conducted directly using the collision detection results. Spatial subdivision is performed on target geometric models to facilitate real-time collision detection. Experimental results show that our algorithm can provide stable and realistic force feedback.

Hardware-accelerated collision detection for 3D virtual reality gaming

Collision detection in simulation can easily become a bottleneck due to its computationally intensive nature. Recent developments in graphics hardware, however, offer a viable solution for rapid and efficient collision detection. The authors propose a new two-phase technique using the latest graphics hardware. In the broad phase, a scene graph is created to partition objects in a 3D environment for initial collision checking. In the narrow phase, a multiple-viewing volumes method is used to detect interferences between a convex model and a model of arbitrary geometry. First, the convex model is used to define six viewing volumes. The convex and arbitrary models are then rendered respectively within the defined viewing volumes. Finally, results of collision detection can be easily achieved by querying the occlusions between these rendered models in the image space. Compared with other collision detection algorithms, this method produces promising results and is successfully applied in our 3D virtual reality games.

OPTIMIZATION TECHNIQUES FOR ASSEMBLY PLANNING OF COMPLEX MODELS IN LARGE-SCALE VIRTUAL ENVIRONMENTS

Evaluation and planning of assembly processes in virtual environments, especially large-scale virtual environments, have attracted much attention in the engineering community. However, the planning of complex assemblies in virtual environments is still hindered by limitations like insufficient frame rates, unnatural user interaction, as well as deficiencies in the processing of assembly constraints. This paper discusses some optimization techniques for the assembly planning of complex assemblies in a large-scale virtual environment such as CAVE. Specifically, the real-time display of complex assemblies, the natural and intuitive assembly operations, and a real-time collision detection algorithm for the assembly applications are presented.

Constraint-Based Virtual Solid Modeling

Constraint-based solid modeling is the kernel part of current CAD systems. It has been widely used in supporting detailed design and variational design. However, it cannot support early stage design and is not easy-to-use because it demands fully detailed input description of a design. To solve these problems, researchers attempt to incorporate virtual reality techniques into geometric modeling systems. This paper presents a novel approach for interactive constraint-based solid modeling in a virtual reality environment. The approach allows the designer to construct and edit a constraint-based solid model by direct 3D manipulations, and ensures the created solid model to be precise by recognizing and solving geometric constraints. To effectively support 3D manipulations and change propagation, a new constraint-based solid model is adopted. In the model, besides the normal constituents of the typical constraint-based solid model, some new attributes like shape control points, location pattern and explicit shape constraints of a primitive are defined. Guided by the location pattern, our algorithms for recognizing and solving location constraints are real-time.