Suyang Dong

Collaborative visualization of engineering processes using tabletop augmented reality

3D computer visualization has emerged as an advanced problem-solving tool for engineering education and practice. For example in civil engineering, the integration of 3D/4D CAD models in the construction process helps to minimize the misinterpretation of the spatial, temporal, and logical aspects of construction planning information. Yet despite the advances made in visualization, the lack of collaborative problem-solving abilities leaves outstanding challenges that need to be addressed before 3D visualization can become widely accepted in the classroom and in professional practice. The ability to smoothly and naturally interact in a shared workspace characterizes a collaborative learning process. This paper introduces tabletop Augmented Reality to accommodate the need to collaboratively visualize computer-generated models. A new software program named ARVita is developed to validate this idea, where multiple users wearing Head-Mounted Displays and sitting around a table can all observe and interact with dynamic visual simulations of engineering processes. The applications of collaborative visualization using Augmented Reality are reviewed, the technical implementation is covered, and the programs underlying tracking libraries are presented.

SMART: scalable and modular augmented reality template for rapid development of engineering visualization applications

BackgroundThe visualization of civil infrastructure systems and processes is critical for the validation and communication of computer generated models to decision-makers. Augmented Reality (AR) visualization blends real-world information with graphical 3D models to create informative composite views that are difficult to create or replicate on the computer alone.MethodsThis paper presents a scalable and extensible mobile computing framework that allows users to readily create complex outdoor AR visual applications. The technical challenges of building this reusable framework from the software and hardware perspectives are described.ResultsSMART is a generic and loosely-coupled software computing framework for creating AR visual applications with accurate registration algorithms and visually credible occlusion effects. While SMART is independent of any specific hardware realization, ARMOR is built as a hardware implementation of SMART to test its algorithm correctness and its adaption to engineering applications. In particular, ARMOR is a modular mobile hardware platform designed for user position and orientation tracking, as well as augmented view display.ConclusionThe framework has been validated in several case studies, including the visualization of underground utilities for excavator control and collision avoidance to demonstrate SMART’s rapid creation of complex AR visual applications.

Integration of infrastructure based positioning systems and inertial navigation for ubiquitous context-aware engineering applications

This paper presents research that investigated and implemented a hybrid integrated location tracking framework that was developed by integrating infrastructure based positioning systems and inertial navigation. The authors implemented this research by using the Personal Dead Reckoning positioning system to serve as the inertial navigation based positioning system. The primary contribution of the presented work is the development and implementation of the Integrated Tracking System algorithm which was implemented in two levels. At the first level, the Integrated Tracking System (ITS) was developed by integrating Global Positioning System (GPS) and Personal Dead Reckoning (PDR) system to ubiquitously track a mobile user, in dynamic environments where GPS coverage may be uncertain. At the second level, the PDR system was integrated with a database of pre-determined known indoor location points in order to correct the accumulated drift error during a mobile users navigation in an indoor environment. Finally, a hybrid tracking system was developed and implemented by integrating the PDR system with the mobile users intervention and discernment of the environment. The implementation and the results obtained from validation experiments performed on the aforementioned hybrid tracking systems demonstrate the potential of using hybrid tracking for determining the spatial context of mobile users in ubiquitous context-aware engineering applications.

Geospatial Databases and Augmented Reality visualization for improving safety in urban excavation operations

The U.S. has more than 14 million miles of buried pipelines and utilities, many of which are in congested urban environments where several lines share the underground space. Errors in locating excavations for new installation or for repair/rehabilitation of existing utilities can result in significant costs, delays, loss of life, and damage to property (Sterling 2000). There is thus a clear need for new solutions to accurately locate buried infrastructure and improve excavation safety. This paper presents ongoing research being collaboratively conducted by the University of Michigan and DTE Energy (Michigans largest electric and gas utility company) that is investigating the use of Real-Time Kinematic global positioning system (GPS), combined with Geospatial Databases of subsurface utilities to design a new visual excavator-utility collision avoidance technology. 3D models of buried utilities are created from available geospatial data, and then superimposed over an excavators work space using geo-referenced Augmented Reality (AR) to provide the operator and the spotter(s) with visual information on the location and type of utilities that exist in the excavators vicinity. This paper describes the overall methodology and the first results of the research.

Robust mobile computing framework for visualization of simulated processes in augmented reality

Visualization of engineering processes can be critical for validation and communication of simulation models to decision-makers. Augmented Reality (AR) visualization blends real-world information with graphical 3D models to create informative composite views that are difficult to replicate on the computer alone. This paper presents a robust and general-purpose mobile computing framework that allows users to readily create complex AR visual simulations. The technical challenges of building this framework from the software and hardware perspective are described. SMART is a generic and loosely-coupled software application framework for creating AR visual simulations with accurate registration and projection algorithms. ARMOR is a modular mobile hardware platform designed for user position and orientation tracking and augmented view display. Together, SMART and ARMOR allow the creation of complex AR visual simulations. The framework has been validated in several case studies, including the visualization of underground infrastructure for applications in excavation planning and control.

Real-Time Occlusion Handling for Dynamic Augmented Reality Using Geometric Sensing and Graphical Shading

AbstractThe primary challenge in generating convincing augmented reality (AR) graphics is to project three-dimensional (3D) models onto a user’s view of the real world and create a temporal and spatial sustained illusion that the virtual and real objects coexist. Regardless of the spatial relationship between the real and virtual objects, traditional AR graphical engines break the illusion of coexistence by displaying the real world merely as a background and superimposing virtual objects on the foreground. This research proposes a robust depth-sensing and frame buffer algorithm for handling occlusion problems in ubiquitous AR applications. A high-accuracy time-of-flight (TOF) camera is used to capture the depth map of the real world in real time. The distance information is processed in parallel using the OpenGL shading language (GLSL) and render to texture (RTT) techniques. The final processing results are written to the graphics frame buffers, allowing accurate depth resolution and hidden surface remov...

Collaborative visualization of simulated processes using tabletop fiducial augmented reality

In typical scenarios of construction planning, engineers communicate ideas primarily using paper based media (e.g. drawings) spread across table surfaces. Even though the traditional communication approach offers convenient interaction among participants, the media used are cumbersome to handle. Moreover, they present static information that cannot reflect the dynamic nature of a jobsite. These limitations can be somewhat overcome by computer based virtual environments. However, the convenience of interactive collaboration among participants is lost. This paper introduces tabletop fiducial Augmented Reality to bridge the gap between paper based static information and computer based graphical models. A software named ARVita is developed to validate this idea, where multiple users wearing Head-Mounted Displays and sitting across a table can observe and interact with visual simulations of planned processes. The applications of collaborative visualization using Augmented Reality are reviewed, and the technical implementation of ARVita is presented.

Vision-based articulated machine pose estimation for excavation monitoring and guidance

The pose of an articulated machine includes the position and orientation of not only the machine base (e.g., tracks or wheels), but also each of its major articulated components (e.g., stick and bucket). The ability to automatically estimate this pose is a crucial component of technical innovations aimed at improving both safety and productivity in many construction tasks. A computer vision based solution using a network of cameras and markers is proposed in this research to enable such a capability for articulated machines. Firstly, a planar marker is magnetically mounted on the end-effector of interest. Another marker is fixed on the jobsite whose 3D pose is pre-surveyed in a project coordinate frame. Then a cluster of at least two cameras respectively observing and tracking the two markers simultaneously forms a camera-marker network and transfers the end-effectors pose into the desired project frame, based on a pre-calibration of the relative poses between each pair of cameras. Through extensive sets of uncertainty analyses and field experiments, this approach is shown to be able to achieve centimeter level depth tracking accuracy within up to 15 meters with only two ordinary cameras (1.1 megapixel each) and a few markers, providing a flexible and cost-efficient alternative to other commercial products that use infrastructure dependent sensors like GPS. A working prototype has been tested on several active construction sites with positive feedback from excavator operators confirming the solutions effectiveness.

Relative Displacement Sensing Techniques for Postevent Structural Damage Assessment: Review

AbstractRelative displacement, which is displacement of a point on a structure with respect to its original location or an adjacent point on the structure that has also undergone movement, can be an effective indicator of postevent structural damage. Although quantifying relative deformations for postevent damage quantification is currently technically feasible, and of great potential benefit, the field remains undeveloped because of lack of knowledge among structural engineers of (1) the enabling technologies and (2) the performance limits of these technologies. The primary objectives of this paper are to review available techniques for measuring relative deformations, identify their limitations, and propose areas where further research is needed. Current methodologies are divided into remote and in situ methods. The latter techniques are deemed most practically feasible and are surveyed in detail and critiqued. Suggestions for current challenges and research opportunities are proposed with emphasis on a...

Occlusion handling method for ubiquitous augmented reality using reality capture technology and GLSL

The primary challenge in generating convincing Augmented Reality (AR) graphics is to project 3D models onto a user’s view of the real world and create a temporal and spatial sustained illusion that the virtual and real objects co-exist. Regardless of the spatial relationship between the real and virtual objects, traditional AR graphical engines break the illusion of co-existence by displaying the real world merely as a background, and superimposing virtual objects on the foreground. This research proposes a robust depth sensing and frame buffer algorithm for handling occlusion problems in ubiquitous AR applications. A high-accuracy Time-of-flight (TOF) camera is used to capture the depth map of the real-world in real time. The distance information is processed using the OpenGL Shading Language (GLSL) and rendered into the graphics depth buffer, allowing accurate depth resolution and hidden surface removal in composite AR scenes. The designed algorithm is validated in several indoor and outdoor experiments using the SMART AR framework.