Paulo Vinicius Koerich Borges

Video-Based Human Behavior Understanding: A Survey

Understanding human behaviors is a challenging problem in computer vision that has recently seen important advances. Human behavior understanding combines image and signal processing, feature extraction, machine learning, and 3-D geometry. Application scenarios range from surveillance to indexing and retrieval, from patient care to industrial safety and sports analysis. Given the broad set of techniques used in video-based behavior understanding and the fast progress in this area, in this paper we organize and survey the corresponding literature, define unambiguous key terms, and discuss links among fundamental building blocks ranging from human detection to action and interaction recognition. The advantages and the drawbacks of the methods are critically discussed, providing a comprehensive coverage of key aspects of video-based human behavior understanding, available datasets for experimentation and comparisons, and important open research issues.

A Probabilistic Approach for Vision-Based Fire Detection in Videos

Automated fire detection is an active research topic in computer vision. In this paper, we propose and analyze a new method for identifying fire in videos. Computer vision-based fire detection algorithms are usually applied in closed-circuit television surveillance scenarios with controlled background. In contrast, the proposed method can be applied not only to surveillance but also to automatic video classification for retrieval of fire catastrophes in databases of newscast content. In the latter case, there are large variations in fire and background characteristics depending on the video instance. The proposed method analyzes the frame-to-frame changes of specific low-level features describing potential fire regions. These features are color, area size, surface coarseness, boundary roughness, and skewness within estimated fire regions. Because of flickering and random characteristics of fire, these features are powerful discriminants. The behavioral change of each one of these features is evaluated, and the results are then combined according to the Bayes classifier for robust fire recognition. In addition, a priori knowledge of fire events captured in videos is used to significantly improve the classification results. For edited newscast videos, the fire region is usually located in the center of the frames. This fact is used to model the probability of occurrence of fire as a function of the position. Experiments illustrated the applicability of the method.

Correlation-based visual odometry for ground vehicles

Reliable motion estimation is a key component for autonomous vehicles. We present a visual odometry method for ground vehicles using template matching. The method uses a downward-facing camera perpendicular to the ground and estimates the motion of the vehicle by analyzing the image shift from frame to frame. Specifically, an image region (template) is selected, and using correlation we find the corresponding image region in the next frame. We introduce the use of multitemplate correlation matching and suggest template quality measures for estimating the suitability of a template for the purpose of correlation. Several aspects of the template choice are also presented. Through an extensive analysis, we derive the expected theoretical error rate of our system and show its dependence on the template window size and image noise. We also show how a linear forward prediction filter can be used to limit the search area to significantly increase the computation performance. Using a single camera and assuming an Ackerman-steering model, the method has been implemented successfully on a large industrial forklift and a 4×4 vehicle. Over 6 km of field trials from our industrial test site, an off-road area and an urban environment are presented illustrating the applicability of the method as an independent sensor for large vehicle motion estimation at practical velocities.

Text luminance modulation for hardcopy watermarking

This paper improves a recently proposed hardcopy watermarking method by introducing new approaches to decode the embedded information. The proposed method, coined as text luminance modulation (TLM), embeds hidden data in office-like documents, while presenting robustness to the print-scan (PS) channel. The hidden data is embedded by slightly modulating the luminance of characters and symbols. This change can be set unperceivable to the human eye and detected with the aid of a scanner. In previous works, the hidden data was retrieved by using the average luminance level or the halftone pattern as a detection metric. In this paper, however, the detection process combines different metrics into a single metric, significantly improving the detection performance. This paper proposes a new PS model where characteristics induced by the halftoning in the printing process are considered, allowing the use of the variance of a character as a new detection metric. Performance analyses validate the proposed PS model. Experiments illustrate the precision of the analyses and the applicability of the method.

Robust and Transparent Color Modulation for Text Data Hiding

This paper improves the use of text color modulation (TCM) as a reliable text document data hiding method. Using TCM, the characters in a document have their color components modified (possibly unperceptually) according to a side message to be embedded. This work presents a detection metric and an analysis determining the detection error rate in TCM, considering an assumed print and scan (PS) channel model. In addition, a perceptual impact model is employed to evaluate the perceptual difference between a modified and a non-modified character. Combining this perceptual model and the results from the detection error analysis it is possible to determine the optimum color modulation values. The proposed detection metric also exploits the orientation characteristics of color halftoning to reduce the error rate. In particular, because color halftoning algorithms use different screen orientation angles for each color channel, this is used as an effective feature to detect the embedded message. Experiments illustrate the validity of the analysis and the applicability of the method.

Vision-based localization using an edge map extracted from 3D laser range data

Reliable real-time localization is a key component of autonomous industrial vehicle systems. We consider the problem of using on-board vision to determine a vehicles pose in a known, but non-static, environment. While feasible technologies exist for vehicle localization, many are not suited for industrial settings where the vehicle must operate dependably both indoors and outdoors and in a range of lighting conditions. We extend the capabilities of an existing vision-based localization system, in a continued effort to improve the robustness, reliability and utility of an automated industrial vehicle system. The vehicle pose is estimated by comparing an edge-filtered version of a video stream to an available 3D edge map of the site. We enhance the previous system by additionally filtering the camera input for straight lines using a Hough transform, observing that the 3D environment map contains only linear features. In addition, we present an automated approach for generating 3D edge maps from laser point clouds, removing the need for manual map surveying and also reducing the time for map generation down from days to minutes. We present extensive localization results in multiple lighting conditions comparing the system with and without the proposed enhancements.

A vision-based lane detection system combining appearance segmentation and tracking of salient points

Reliable lane detection is a key component of autonomous vehicles supporting navigation in urban environments. This paper introduces the GOLDIE(Geometric Overture for Lane Detection by Intersections Entirety) system, a vision-based software architecture that uses an on-board single camera to determine the position of road lanes with respect to the vehicle. We propose an efficient vision-based lane-detection system that combines an appearance-based analysis with salient point tracking. The appearance-based analysis consists of segmenting high contrast areas that fit inside a Region-Of-Interest(ROI) on the frame. The salient point tracker selects interesting points based in a reference line, that guides a dynamic ROI. The tracking ROI look for paint lane marks close to the last lane reference found, where road marks are likely to emerge, in order to maintain the usability of the salient point tracker. The tracking is performed with the Lucas-Kanade algorithm and the lane points candidates are selected according to a predefined triangular model. Once such lanes points are detected, the vehicle position is estimated based on the intersection of linearised lanes determined through a vanishing point approach. Experiments and comparisons with other algorithms illustrate the applicability of the method.

Pedestrian Detection Based on Blob Motion Statistics

Pedestrian detection based on video analysis is a key functionality in automated surveillance systems. In this paper, we present efficient detection metrics that consider the fact that human movement presents distinctive motion patterns. Contrary to several methods that perform an intrablob analysis based on motion masks, we approach the problem without necessarily considering the periodic pixel motion inside the blob. As such, we do not analyze periodicity in the pixel luminances, but in the motion statistics of the tracked blob as a whole. For this, we propose the use of the following cues: 1) a cyclic behavior in the blob trajectory, and 2) an in-phase relationship between the change in blob size and position. In addition, we also exploit the relationship between blob size and vertical position, assuming that the camera is positioned sufficiently high. If the homography between the camera and the ground is known, the features are normalized by transforming the blob size to the real person size. For improved performance, we combine these features using the Bayes classifier. We also present a theoretical statistical analysis to evaluate the system performance in the presence of noise. We perform online experiments in a real industrial scenario and also with videos from well-known databases. The results illustrate the applicability of the proposed features.

Document Image Processing for Paper Side Communications

This paper proposes the use of higher order statistical moments in document image processing to improve the performance of systems which transmit side information through the print and scan channel. Examples of such systems are multilevel 2-D bar codes and certification via text luminance modulation. These systems print symbols with different luminances, according to the target side information. In previous works, the detection of a received symbol is usually performed by evaluating the average luminance or spectral characteristics of the received signal. This paper points out that, whenever halftoning algorithms are used in the printing process, detection can be improved by observing that third and fourth order statistical moments of the transmitted symbol also change, depending on the luminance level. This work provides a thorough analysis for those moments used as detection metrics. A print and scan channel model is exploited to derive the relationship between the modulated luminance level and the higher order moments of a halftone image. This work employs a strategy to merge the different moments into a single metric to achieve a reduced detection error rate. A transmission protocol for printed documents is proposed which takes advantage of the resulting higher robustness achieved with the combined detection metrics. The applicability of the introduced document image analysis approach is validated by comprehensive computer simulations.

Integrating Off-Board Cameras and Vehicle On-Board Localization for Pedestrian Safety

Situational awareness for industrial vehicles is crucial to ensure safety of personnel and equipment. While human drivers and onboard sensors are able to detect obstacles and pedestrians within line-of-sight, in complex environments, initially occluded or obscured dynamic objects can unpredictably enter the path of a vehicle. We propose a system that integrates a vision-based offboard pedestrian tracking subsystem with an onboard localization and navigation subsystem. This combination enables warnings to be communicated and effectively extends the vehicle controllers field of view to include areas that would otherwise be blind spots. A simple flashing light interface in the vehicle cabin provides a clear and intuitive interface to alert drivers of potential collisions. Alternatively, the system can be also applied to vehicles that have autonomous navigation capabilities, in which case, instead of alert lights, the vehicle is halted or redirected. We implemented and tested the proposed solution on an automated industrial vehicle under autonomous operation and on a human-driven vehicle in a full-scale production facility, over a period of four months.