Rikke Gade

Long-Term Occupancy Analysis Using Graph-Based Optimisation in Thermal Imagery

This paper presents a robust occupancy analysis system for thermal imaging. Reliable detection of people is very hard in crowded scenes, due to occlusions and segmentation problems. We therefore propose a framework that optimises the occupancy analysis over long periods by including information on the transition in occupancy, when people enter or leave the monitored area. In stable periods, with no activity close to the borders, people are detected and counted which contributes to a weighted histogram. When activity close to the border is detected, local tracking is applied in order to identify a crossing. After a full sequence, the number of people during all periods are estimated using a probabilistic graph search optimisation. The system is tested on a total of 51,000 frames, captured in sports arenas. The mean error for a 30-minute period containing 3-13 people is 4.44 %, which is a half of the error percentage optained by detection only, and better than the results of comparable work. The framework is also tested on a public available dataset from an outdoor scene, which proves the generality of the method.

Thermal cameras and applications: a survey

Thermal cameras are passive sensors that capture the infrared radiation emitted by all objects with a temperature above absolute zero. This type of camera was originally developed as a surveillance and night vision tool for the military, but recently the price has dropped, significantly opening up a broader field of applications. Deploying this type of sensor in vision systems eliminates the illumination problems of normal greyscale and RGB cameras. This survey provides an overview of the current applications of thermal cameras. Applications include animals, agriculture, buildings, gas detection, industrial, and military applications, as well as detection, tracking, and recognition of humans. Moreover, this survey describes the nature of thermal radiation and the technology of thermal cameras.

Thermal tracking of sports players.

We present here a real-time tracking algorithm for thermal video from a sports game. Robust detection of people includes routines for handling occlusions and noise before tracking each detected person with a Kalman filter. This online tracking algorithm is compared with a state-of-the-art offline multi-target tracking algorithm. Experiments are performed on a manually annotated 2-minutes video sequence of a real soccer game. The Kalman filter shows a very promising result on this rather challenging sequence with a tracking accuracy above 70% and is superior compared with the offline tracking approach. Furthermore, the combined detection and tracking algorithm runs in real time at 33 fps, even with large image sizes of 1920 × 480 pixels.

Controlling urban lighting by human motion patterns results from a full scale experiment

This paper presents a full-scale experiment investigating the use of human motion intensities as input for interactive illumination of a town square in the city of Aalborg in Denmark. As illuminators sixteen 3.5 meter high RGB LED lamps were used. The activity on the square was monitored by three thermal cameras and analysed by computer vision software from which motion intensity maps and peoples trajectories were estimated and used as input to control the interactive illumination. The paper introduces a 2-layered interactive light strategy addressing ambient and effect illumination criteria totally four light scenarios were designed and tested. The result shows that in general people immersed in the street lighting did not notice that the light changed according to their presence or actions, but people watching from the edge of the square noticed the interaction between the illumination and the immersed persons. The experiment also demonstrated that interactive can give significant power savings. In the current experiment there was a difference of 92% between the most and less energy consuming light scenario.

Pedestrian counting with occlusion handling using stereo thermal cameras

The number of pedestrians walking the streets or gathered in public spaces is a valuable piece of information for shop owners, city governments, event organizers and many others. However, automatic counting that takes place day and night is challenging due to changing lighting conditions and the complexity of scenes with many people occluding one another. To address these challenges, this paper introduces the use of a stereo thermal camera setup for pedestrian counting. We investigate the reconstruction of 3D points in a pedestrian street with two thermal cameras and propose an algorithm for pedestrian counting based on clustering and tracking of the 3D point clouds. The method is tested on two five-minute video sequences captured at a public event with a moderate density of pedestrians and heavy occlusions. The counting performance is compared to the manually annotated ground truth and shows success rates of 95.4% and 99.1% for the two sequences.

Sports Type Classification Using Signature Heatmaps

Automatic classification of activities in a sports arena is important in order to analyse and optimise the use of the arenas. In this work we classify five sports types based only on occupancy heatmaps produced from position data. Due to privacy issues we use thermal imaging for detecting people and then calculate their positions on the court using homography. Heatmaps are produced by summarising Gaussian distributions respresenting people over 10-minute periods. Before classification the heatmaps are projected to a low-dimensional discriminative space using the principle of Fisherfaces. Our result using two weeks of video are very promising with a correct classification of 90.76 %.

Computer vision for sports: Current applications and research topics

Abstract The world of sports intrinsically involves fast and accurate motion that is not only challenging for competitors to master, but can be difficult for coaches and trainers to analyze, and for audiences to follow. The nature of most sports means that monitoring by the use of sensors or other devices fixed to players or equipment is generally not possible. This provides a rich set of opportunities for the application of computer vision techniques to help the competitors, coaches and audience. This paper discusses a selection of current commercial applications that use computer vision for sports analysis, and highlights some of the topics that are currently being addressed in the research community. A summary of on-line datasets to support research in this area is included.

Estimation of Energy Expenditure during Treadmill Exercise via Thermal Imaging.

PURPOSE Noninvasive imaging of oxygen uptake may provide a useful tool for the quantification of energy expenditure during human locomotion. A novel thermal imaging method (optical flow) was validated against indirect calorimetry for the estimation of energy expenditure during human walking and running. METHODS Fourteen endurance-trained subjects completed a discontinuous incremental exercise test on a treadmill. Subjects performed 4-min intervals at 3, 5, and 7 km·h (walking) and at 8, 10, 12, 14, 16, and 18 km·h (running) with 30 s of rest between intervals. Heart rate, gas exchange, and mean accelerations of ankle, thigh, wrist, and hip were measured throughout the exercise test. A thermal camera (30 frames per second) was used to quantify optical flow, calculated as the movements of the limbs relative to the trunk (internal mechanical work) and vertical movement of the trunk (external vertical mechanical work). RESULTS Heart rate, gross oxygen uptake (mL·kg·min) together with gross and net energy expenditure (J·kg·min) rose with increasing treadmill velocities, as did optical flow measurements and mean accelerations (g) of ankle, thigh, wrist, and hip. Oxygen uptake was linearly correlated with optical flow across all exercise intensities (R = 0.96, P < 0.0001; V˙O2 [mL·kg·min] = 7.35 + 9.85 × optical flow [arbitrary units]). Only 3-4 s of camera recording was required to estimate an optical flow value at each velocity. CONCLUSIONS Optical flow measurements provide an accurate estimation of energy expenditure during horizontal walking and running. The technique offers a novel experimental method of estimating energy expenditure during human locomotion, without use of interfering equipment attached to the subject.

Using Human Motion Intensity as Input for Urban Design

This paper presents a study investigating the potential use of human motion intensities as input for parametric urban design. Through a computer vision analysis of thermal images, motion intensity maps are generated and utilized as design drivers for urban design patterns; and, through a case study of a town square, human occupancy and motion intensities are used to generate situated flow topologies presenting new adaptive methods for urban design. These methods incorporate local flow as design drivers for canopy, pavement and furniture layout. The urban design solution may be configured due to various parameters such as security, comfort, navigation, efficiency, or aesthetics.

Audio-Visual Classification of Sports Types

In this work we propose a method for classification of sports types from combined audio and visual features extracted from thermal video. From audio Mel Frequency Cepstral Coefficients (MFCC) are extracted, and PCA are applied to reduce the feature space to 10 dimensions. From the visual modality short trajectories are constructed to represent the motion of players. From these, four motion features are extracted and combined directly with audio features for classification. A k-nearest neighbour classifier is applied for classification of 180 1-minute video sequences from three sports types. Using 10-fold cross validation a correct classification rate of 96.11% is obtained with multimodal features, compared to 86.67% and 90.00% using only visual or audio features, respectively.