Martin Winter

Automatic freeze frame detection for video preservation

A significant amount of work in film and video preservation is dedicated to quality assessment of the content to be archived or reused out of the archive. This paper proposes automatic content analysis algorithms which reduce manual inspection time in software based preservation environments. We list the requirements for such algorithms and tools and show exemplarily on a freeze frame impairment detector how analysis algorithms need to be designed for meeting the requirements. The evaluation has shown that robustness against other impairments (noise and flickering) is an essential part of the algorithm. Successful detection of freeze frame impairments with a minimum length of three frames is achieved. The consideration of human perception is important to achieve low false detection rates. Analysis results are represented in a MPEG-7 standard compliant way. The proposed defect summary visualization tool enables efficient human exploration of visually impaired content.

Satellite-based forest monitoring: spatial and temporal forecast of growing index and short-wave infrared band

For detecting anomalies or interventions in the field of forest monitoring we propose an approach based on the spatial and temporal forecast of satellite time series data. For each pixel of the satellite image three different types of forecasts are provided, namely spatial, temporal and combined spatio-temporal forecast. Spatial forecast means that a clustering algorithm is used to group the time series data based on the features normalised difference vegetation index (NDVI) and the short-wave infrared band (SWIR). For estimation of the typical temporal trajectory of the NDVI and SWIR during the vegetation period of each spatial cluster, we apply several methods of functional data analysis including functional principal component analysis, and a novel form of random regression forests with online learning (streaming) capability. The temporal forecast is carried out by means of functional time series analysis and an autoregressive integrated moving average model. The combination of the temporal forecasts, which is based on the past of the considered pixel, and spatial forecasts, which is based on highly correlated pixels within one cluster and their past, is performed by functional data analysis, and a variant of random regression forests adapted to online learning capabilities. For evaluation of the methods, the approaches are applied to a study area in Germany for monitoring forest damages caused by wind-storm, and to a study area in Spain for monitoring forest fires.

Multi-cue learning and visualization of unusual events

Unusual event detection, i.e., identifying unspecified rare/critical events, has become one of the major challenges in visual surveillance. The main solution for this problem is to describe local or global normalness and to report events that do not fit to the estimated models. The majority of existing approaches, however, is limited to a single description (e.g., either appearance or motion) and/or builds on inflexible (unsupervised) learning techniques, both clearly degrading the practical applicability. To overcome these limitations, we demonstrate a system that is capable of extracting and modeling several representations in parallel, while in addition allows for user interaction within a continuous learning setup. Novel yet intuitive concepts of result visualization and user interaction will be presented that allow for exploiting the underlying data.

AVSS 2011 demo session: OUTLIER - online learning and visualization of unusual events

Summary form only given. We introduce to the surveillance community the VIRAT Video Dataset[1], which is a new large-scale surveillance video dataset designed to assess the performance of event recognition algorithms in realistic scenes1.

Automatic Content Based Video Quality Analysis for Media Production and Delivery Processes

Automatic quality control for audiovisual media is an important tool in several steps of the media production, delivery and archiving processes. Today, mainly technical properties of the material are checked, e.g. stream compliance, playtime, aspect ratio, and resolution or MXF compliance. Only some content properties can be checked automatically, e.g. blocking or luma/chroma violation. Other relevant content properties and impairments like noise level, sharpness, large dropouts, flickering or instability are checked by manually exploring the audiovisual content. In this work we focus on challenges and recent results in automatic content based visual quality analysis of video. We first give an overview on which visual impairments are relevant in which stages of the media production, archiving and delivery process. A set of requirements for impairment detection algorithms, tools and systems is presented. We show how impairment detection algorithms need to be designed in order to meet these requirements. Furthermore we show our recent algorithmic research results for two content based impairment detectors (freeze frame and video breakup detection). In order to facilitate interoperability and exchange of impairment metadata between different tools and systems, a standardized way of description is needed. We give an overview on our framework proposed for the description of visual impairments based on MPEG-7. In order to enable efficient human interaction with quality analysis results we present the “Quality Summary Viewer” application which allows a user to quickly grasp the frequency and strengths of visual impairments in the content.

Learning Selection of User Generated Event Videos

User generated images and videos can enhance the coverage of live events on social and online media, as well as in broadcasts. However, the quality, relevance and complementarity of the received contributions varies greatly. In a live scenario, it is often not feasible for the editorial team to review all content and make selections. We propose to support this work by automatic selection based on captured metadata, and extracted quality and content features. It is usually desired to have a human in the loop, thus the automatic system does not make a final decision, but provides a ranked list of content items. As the operator makes selections, the automatic system shall learn from these decisions, which may change over time. Due to the need for online learning and quick adaptation, we propose the use of online random forests for this task. We show on data from three real live events that the approach is able to provide a ranking based on the predicted selection likelihood after an initial adjustment phase.

Recommendation of Counteractions for Prevention of Critical Events in Sub-Surface Drilling Environments

Sub-Surface Drilling is the process of making boreholes into the Earth, which can reach depths of many kilometers. One of the major purposes of such boreholes is the exploration of oil or gas bearing formations with the goal to recover the content of such reservoirs. Problems in drilling operations pose serious risks for the crew and the environment and can cause significant financial losses. Critical events usually do not arise abruptly, but develop over time before they escalate. In this work, the authors present a system that integrates sensor data and machine learning algorithms into a decision support system (DSS), thus helping to avoid critical events by monitoring and recommending preventive measures. The authors describe how the DSS is implemented as a distributed system and how data-driven decision support processes are implemented and integrated into the system. The DSS detects drilling operations by recognizing temporal patterns in the sensor data and uses a combination of detected operational rig-states and sensor data to predict and recommend preventive measures for the stuck pipe problem. The sensor data, detection results and predictions are distributed to all stakeholders and displayed in appropriate user interfaces.