Arsalan Heydarian

Vision-based action recognition of earthmoving equipment using spatio-temporal features and support vector machine classifiers

We present a computer vision based method for equipment action recognition.Our vision-based method is based on a multiple binary SVM classifier and spatio-temporal features.A comprehensive real-world video dataset of excavator and truck actions is presented.We achieve accuracies of 86.33% and 98.33% for excavator and truck action classes.The presented method can be used for construction activity analysis using long sequences of videos. Video recordings of earthmoving construction operations provide understandable data that can be used for benchmarking and analyzing their performance. These recordings further support project managers to take corrective actions on performance deviations and in turn improve operational efficiency. Despite these benefits, manual stopwatch studies of previously recorded videos can be labor-intensive, may suffer from biases of the observers, and are impractical after substantial period of observations. This paper presents a new computer vision based algorithm for recognizing single actions of earthmoving construction equipment. This is particularly a challenging task as equipment can be partially occluded in site video streams and usually come in wide variety of sizes and appearances. The scale and pose of the equipment actions can also significantly vary based on the camera configurations. In the proposed method, a video is initially represented as a collection of spatio-temporal visual features by extracting space-time interest points and describing each feature with a Histogram of Oriented Gradients (HOG). The algorithm automatically learns the distributions of the spatio-temporal features and action categories using a multi-class Support Vector Machine (SVM) classifier. This strategy handles noisy feature points arisen from typical dynamic backgrounds. Given a video sequence captured from a fixed camera, the multi-class SVM classifier recognizes and localizes equipment actions. For the purpose of evaluation, a new video dataset is introduced which contains 859 sequences from excavator and truck actions. This dataset contains large variations of equipment pose and scale, and has varied backgrounds and levels of occlusion. The experimental results with average accuracies of 86.33% and 98.33% show that our supervised method outperforms previous algorithms for excavator and truck action recognition. The results hold the promise for applicability of the proposed method for construction activity analysis.

A Visual Monitoring Framework for Integrated Productivity and Carbon Footprint Control of Construction Operations

As buildings and infrastructure are becoming more energy efficient, reducing and mitigating construction-phase carbon footprint and embodied carbon is getting more attention. Government agencies are forming incentive-based regulations on controlling these impacts and expressing control of carbon footprints as principle dynamic goals in projects. These regulations are placing requirements upon construction firms to find control techniques to minimize carbon footprint without affecting productivity of operations. Nevertheless, there is limited research on integrated real-time techniques to monitor operations productivity and carbon footprint. This paper proposes a new framework and presents preliminary data in which (1) construction operations are visually sensed through construction site imagery and video-streams; subsequently (2) equipment’s location and action are semantically analyzed through an integrated 3D image-based reconstruction and appearance-based recognition algorithm; (3) productivity and carbon footprint of construction operations are measured through a new machine learning approach; and finally (4) for each construction schedule activity, measured productivity and carbon footprint are visualized.

Use of immersive virtual environments for occupant behaviour monitoring and data collection

With the recent advancements in the field of virtual reality, occupants can now be immersed into virtual built environments, where they have the ability to visualize and interact with the environment. Researchers also have the ability to collect and observe various behavioural choices impacting performance of built environments (e.g. shading control, lighting preferences, use of appliances, etc.) in such environments. They can further create realistic occupant-related models for more accurate building performance simulation (BPS). This paper identifies the benefits and limitations of using immersive virtual environments (IVEs) for collecting and modelling occupant behaviours while highlighting how such information could improve BPS results. Furthermore, a number of lessons learned and best practices for designing IVE-based experiments, current shortcomings of such environments, and future research opportunities for adopting such technologies towards improving building design and operations, specifically for BPS purposes, are discussed in detail.

Towards Understanding End-user Lighting Preferences in Office Spaces by Using Immersive Virtual Environments

Buildings and their systems are primarily designed based on several assumptions about end-users’ requirements and needs, which in many cases are incomplete and result in inefficiencies during operation phases of buildings. With advancements in fields of augmented and virtual reality, designers and engineers have now the opportunity to collect information about end-users’ requirements, preferences, and behaviors for more informed decision-making during the design phase. These approaches allow for buildings to be designed around the users, with the goal that the design will result in reduction of energy consumption and improved building operations. The authors examine the effect of design features on occupants’ preferences and performance within immersive virtual environments (IVEs). Specifically, this paper presents an approach to understand end-users’ lighting preferences and collect end-user performance data through the use of IVEs.

Use of Immersive Virtual Environments to Understand Human-Building Interactions and Improve Building Design

Previous research has shown occupants’ behavior and interactions with building systems and components have a significant impact on the total energy consumption of buildings. Incorporating occupant requirements to the design process could result in better operations, and therefore, improve the total energy consumption of buildings. Currently, buildings are primarily designed based on several common assumptions about occupant requirements, which in many cases are incorrect and result in inefficiencies during the buildings’ operation phase. With the recent improvements in the fields of virtual and augmented reality, designers now have the opportunity to accurately collect and analyze occupants’ behavioral information. In this research, through the use of immersive virtual environments, the influence of different design features on end-user behavior (preferences and patterns) and performances are examined. A case study is presented, in which the authors measure the end-users’ lighting preferences to better understand the impact of preferences on end-users’ performances and lighting-related energy consumption.

Towards Measuring the Impact of Personal Control on Energy Use through the Use of Immersive Virtual Environments

Recent studies have focused on increasing energy efficiency in commercial buildings through technological means (e.g., efficient HVAC systems, sensors and sensing systems). However, most studies underestimate the impact of occupants’ behavioural choices. Lighting systems account for approximately a fifth of the total electricity consumption in the US; commercial buildings account for 71 percent of such consumption. This paper focuses on human behaviour related energy consumption by investigating the impact of personal control on lighting use in office environments. To effectively examine human energy consumption behaviour, alternative 3D design models of an office are created using an immersive virtual environment to visualize different lighting control features. Participants are brought into these immersive virtual environments by wearing Head-Mounted Displays and are asked to interact within these environments and perform a defined task. Participants were then allowed to control and change the room’s lighting settings based on their preferences in order to perform their assigned task. Unique to our experimental design is the use of immersive virtual environments, enabling measurement and control of a series of design feature isolations and combinations. The work presents the impact of decisions made both during design and operation of buildings on occupants’ energy related behaviour. The experiment demonstrated that when

Exploration of Building-Occupant Communication Methods for Reducing Energy Consumption in Buildings

Buildings consume enormous amounts of our nation’s total energy use (38 %). Previous work showed that occupant actions and behaviors have significant impacts (more than 40 %) on building energy demand. Our main goal is to transform buildings into interactive living spaces that communicate with their occupants via agents and influence the way the occupants interact with their building to enable energy efficiency. As a first step towards this goal, we investigated effective communication methods aimed at influencing building occupants’ energy-related behaviors. We hypothesized that human-building communication would be more persuasive if the interaction is seen as more social. To investigate the influence of social influence methods (e.g., foot in the door, rule of reciprocity, and direct request) on occupants’ energy consumption behavior, experiments were conducted in which immersive virtual environments (IVEs) were used to model real-life office settings.