SangUk Han

Computer vision techniques for construction safety and health monitoring

For construction safety and health, continuous monitoring of unsafe conditions and action is essential in order to eliminate potential hazards in a timely manner. As a robust and automated means of field observation, computer vision techniques have been applied for the extraction of safety related information from site images and videos, and regarded as effective solutions complementary to current time-consuming and unreliable manual observational practices. Although some research efforts have been directed toward computer vision-based safety and health monitoring, its application in real practice remains premature due to a number of technical issues and research challenges in terms of reliability, accuracy, and applicability. This paper thus reviews previous attempts in construction applications from both technical and practical perspectives in order to understand the current status of computer vision techniques, which in turn suggests the direction of future research in the field of computer vision-based safety and health monitoring. Specifically, this paper categorizes previous studies into three groups-object detection, object tracking, and action recognition-based on types of information required to evaluate unsafe conditions and acts. The results demonstrate that major research challenges include comprehensive scene understanding, varying tracking accuracy by camera position, and action recognition of multiple equipment and workers. In addition, we identified several practical issues including a lack of task-specific and quantifiable metrics to evaluate the extracted information in safety context, technical obstacles due to dynamic conditions at construction sites and privacy issues. These challenges indicate a need for further research in these areas. Accordingly, this paper provides researchers insights into advancing knowledge and techniques for computer vision-based safety and health monitoring, and offers fresh opportunities and considerations to practitioners in understanding and adopting the techniques.

Empirical assessment of a RGB-D sensor on motion capture and action recognition for construction worker monitoring

BackgroundFor construction management, data collection is a critical process for gathering and measuring information for the evaluation and control of ongoing project performances. Taking into account that construction involves a significant amount of manual work, worker monitoring can play a key role in analyzing operations and improving productivity and safety. However, time-consuming tasks involved in field observation have brought up the issue of implementing worker observation in daily management practice.MethodsIn an effort to address the issue, this paper investigates the performances of a cost-effective and portable RGB-D sensor, based on recent research efforts extended from our previous study. The performance of an RGB-D sensor is evaluated in terms of (1) the 3D positions of the body parts tracked by the sensor, (2) the 3D rotation angles at joints, and (3) the impact of the RGB-D sensor’s accuracy on motion analysis. For the assessment, experimental studies were undertaken to collect motion capture datasets using an RGB-D sensor and a marker-based motion capture system, VICON, and to analyze errors as compared with the VICON used as the ground truth. As a test case, 25 trials of ascending and descending during ladder climbing were recorded simultaneously with both systems, and the resulting motion capture datasets (i.e., 3D skeleton models) were temporally and spatially synchronized for their comparison.ResultsThrough the comparative assessment, we found a discrepancy of 10.7 cm in the tracked locations of body parts, and a difference of 16.2 degrees in rotation angles. However, motion detection results show that the inaccuracy of an RGB-D sensor does not have a considerable effect on action recognition in the experiment.ConclusionsThis paper thus provides insight into the accuracy of an RGB-D sensor on motion capture in various measures and directions of further research for the improvement of accuracy.

Motion Data-Driven Biomechanical Analysis during Construction Tasks on Sites

AbstractWork-related musculoskeletal disorders (WMSDs) are one of the major health issues that workers frequently experience due to awkward postures or forceful exertions during construction tasks. Among available job analysis methods, biomechanical models have been widely applied to assess musculoskeletal risks that may contribute to the development of WMSDs based on motion data during occupational tasks. Recently, with the advent of vision-based motion capture approaches, it has become possible to collect the motion data required for biomechanical analysis under real conditions. However, vision-based motion capture approaches have not been applied to biomechanical analysis because of compatibility issues in body models of the motion data and computerized biomechanical analysis tools. To address this issue, automated data processing is focused on to convert motion data into available data in existing biomechanical analysis tools, given the BVH motion data from vision-based approaches. To examine the feas...

An Automated Biomechanical Simulation Approach to Ergonomic Job Analysis for Workplace Design

AbstractWork-related musculoskeletal disorders (WMSDs) are reported to be the most common category of nonfatal occupational injuries that result in days away from work and are also a leading cause of temporary and permanent disability. One of the most effective approaches to preventing WMSDs is to evaluate ergonomics considerations early in the design and construction planning stage before the worker encounters the unsafe conditions. However, a lack of tools for identifying potential ergonomic risks in a proposed workplace design has led to difficulties in integrating safety and health into workplace design practice. In an effort to address this issue, this study explores a motion data-driven framework for ergonomic analysis that automates and visualizes the evaluation process in a virtual workplace. This is accomplished by coupling the ergonomic analysis with three-dimensional (3D) virtual visualization of the work environment. The proposed approach uses motion data from the 3D model of the jobsite to ev...

Vision-based Motion Detection for Safety Behavior Analysis in Construction

Workers’ unsafe actions are likely to result in injuries and are regarded as precursors of incidents. Behavior measurement can thus be used to evaluate safety performance and prevent serious accidents. However, the time-consuming task involved in field observation has made it difficult to monitor workers’ behavior on a jobsite. To address this limitation, this paper introduces vision-based motion capture techniques to detect unsafe actions in site videos. First, prior models representing unsafe actions are collected through experiments and used to identify similar actions in site videos. Then, 3D human skeleton models are extracted from these videos, and both these skeleton models and prior models are transformed onto the same space for motion detection. As a case study, motion data for actions during ladder-climbing are collected and tested. The result reveals that the proposed approach may work well for detecting particular actions with motion data.

A Stereo Vision-Based Approach to Marker-Less Motion Capture for On-Site Kinematic Modeling of Construction Worker Tasks

Marker-less motion capture has been extensively studied in recent years as a means of evaluating productivity, safety, and workplace design for manual operations on-site. These technologies are ideal for circumstances in which traditional motion capture systems are ineffective due to the need for a laboratory setting and movement-inhibiting markers or sensors. However, many marker-less motion capture systems rely on RGB-D sensors that have limited range and susceptibility to interference from sunlight and ferromagnetic radiation, making them unsuitable for modeling worker actions on construction sites. To address this issue, we propose a marker-less motion capture approach utilizing optical images and depth data obtained from stereo vision cameras. Multiple camera lenses and triangulation algorithms generate depths maps similar to those produced by RGB-D sensors, while still utilizing an optical recording process unhindered by potentially harsh construction site conditions. These data are adapted for existing kinematic modeling systems (i.e. iPi Mocap Studio) for 3-D pose estimation. The experiments show that the proposed approach can provide data precision comparable to that of RGB-D-based systems with fewer operational constraints; thus, motion data can be collected where previously developed methods fail due to environmental or maneuverability restrictions. With the proposed approach, kinematic modeling of human movements can be carried out on construction sites without inhibiting the mobility of the recorded subject.

Application of a Visualization Technique for Safety Management

Safety training and management are among the constant tasks of project management on any construction site. A review of literature on the causation model and improvement factors for safety confirms that construction accidents can be preventable with consistent safety management and effective communication between managers and workers. One limitation of traditional safety management, however, is that workers may not be efficiently informed of hazardous locations and safetyrelated issues. In that respect, any technology that facilitates the analysis of safety and communication would have potential benefits in improving safety management. In this paper, application of the D 4 AR model (4-dimensional augmented reality) model, wherein a 4D as-planned model is superimposed on safety site photographs, and a visualization framework for site safety management are introduced. Visualization of safety acts as an effective communication tool to facilitate interactions between workers and managers and brings beneficial spatial information on and cognition of safety education and training. The proposed model is applied to an ongoing building project and the preliminary results are presented.

Tracking-based 3D human skeleton extraction from stereo video camera toward an on-site safety and ergonomic analysis

Purpose As a means of data acquisition for the situation awareness, computer vision-based motion capture technologies have increased the potential to observe and assess manual activities for the prevention of accidents and injuries in construction. This study thus aims to present a computationally efficient and robust method of human motion data capture for the on-site motion sensing and analysis. Design/methodology/approach This study investigated a tracking approach to three-dimensional (3D) human skeleton extraction from stereo video streams. Instead of detecting body joints on each image, the proposed method tracks locations of the body joints over all the successive frames by learning from the initialized body posture. The corresponding body joints to the ones tracked are then identified and matched on the image sequences from the other lens and reconstructed in a 3D space through triangulation to build 3D skeleton models. For validation, a lab test is conducted to evaluate the accuracy and working ranges of the proposed method, respectively. Findings Results of the test reveal that the tracking approach produces accurate outcomes at a distance, with nearly real-time computational processing, and can be potentially used for site data collection. Thus, the proposed approach has a potential for various field analyses for construction workers’ safety and ergonomics. Originality/value Recently, motion capture technologies have rapidly been developed and studied in construction. However, existing sensing technologies are not yet readily applicable to construction environments. This study explores two smartphones as stereo cameras as a potentially suitable means of data collection in construction for the less operational constrains (e.g. no on-body sensor required, less sensitivity to sunlight, and flexible ranges of operations).

Application of dimension reduction techniques for motion recognition: Construction worker behavior monitoring

In the construction industry, the unsafe actions and behavior of workers are the most significant causes of accidents. Measurement of worker behavior thus can be used as a positive indicator in assessing safety management and preventing accidents. The monitoring of worker behavior, however, has not been applied to safety management in practice due to the time-consuming and painstaking nature of this type of monitoring. To address this problem, this paper utilizes a computer vision-based approach that automatically monitors workers with video cameras installed on-site and focuses on motion recognition methods. Templates predefined through experiments are used to determine safe and unsafe poses. Using a dimension reduction technique on a set of spatio-temporal motion segments, the human motion data obtained from experiments are clustered and generalized to recognize motions. In this manner, the unsafe behavior of workers is detected and analyzed through the shape of the human skeleton and joints. The use of video cameras allows worker behavior to be monitored automatically and constantly. The measured information then can be used to reduce the frequency of unsafe behavior and potentially reduce the number of accidents.

Dynamic biomechanical simulation for identifying risk factors for work-related musculoskeletal disorders during construction tasks

We propose a dynamic biomechanical simulation method that uses motion capture to evaluate the risk of Work-related Musculoskeletal Disorders (WMSDs). Statistics show that WMSDs accounted for 33% of all non-fatal occupational injuries and illness in construction in 2009, and were a leading cause of temporary and permanent disability. Present methods rely largely on self-reports from workers, observational techniques, and direct measurements of motion and muscle activity to assess awkward postures, physical loads, repetitiveness, and the duration of exposure. While these methods have helped to prevent WMSDs in construction work, they may not be suitable for estimating the internal tissue loads associated with WMSDs. We propose a dynamic biomechanical simulation method to estimate internal forces and moments at each body joint of construction workers with motion capture data. Particularly, we explore the biomechanical loads by simulating active 3D musculoskeletal models based on measured postures and movements. To demonstrate the feasibility of this approach, we studied a ladder climbing task using a portable ladder under controlled laboratory conditions. Postures and motions were determined with a commercial motion capture system (e.g., VICON). The results were analyzed to investigate the feasibility of identifying risk factors based on biomechanical simulation. The results show that the proposed approach allows us to determine the biomechanical basis for WMSDs, and to identify postures and movements associated with excessive physical demands on each body joint. When combined with marker-less motion capture which is our ongoing work, the proposed approach has the potential to assess an individual’s motions and to provide personalized feedback for the purpose of reducing biomechanical loads and WMSD risk in real workplaces.