Fabian Nater

Exploiting simple hierarchies for unsupervised human behavior analysis

We propose a data-driven, hierarchical approach for the analysis of human actions in visual scenes. In particular, we focus on the task of in-house assisted living. In such scenarios the environment and the setting may vary considerably which limits the performance of methods with pre-trained models. Therefore our model of normality is established in a completely unsupervised manner and is updated automatically for scene-specific adaptation. The hierarchical representation on both an appearance and an action level paves the way for semantic interpretation. Furthermore we show that the model is suitable for coupled tracking and abnormality detection on different hierarchical stages. As the experiments show, our approach, simple yet effective, yields stable results, e.g. the detection of a fall, without any human interaction.

Beyond Novelty Detection: Incongruent Events, When General and Specific Classifiers Disagree

Unexpected stimuli are a challenge to any machine learning algorithm. Here, we identify distinct types of unexpected events when general-level and specific-level classifiers give conflicting predictions. We define a formal framework for the representation and processing of incongruent events: Starting from the notion of label hierarchy, we show how partial order on labels can be deduced from such hierarchies. For each event, we compute its probability in different ways, based on adjacent levels in the label hierarchy. An incongruent event is an event where the probability computed based on some more specific level is much smaller than the probability computed based on some more general level, leading to conflicting predictions. Algorithms are derived to detect incongruent events from different types of hierarchies, different applications, and a variety of data types. We present promising results for the detection of novel visual and audio objects, and new patterns of motion in video. We also discuss the detection of Out-Of-Vocabulary words in speech recognition, and the detection of incongruent events in a multimodal audiovisual scenario.

Tracker trees for unusual event detection

We present an approach for unusual event detection, based on a tree of trackers. At lower levels, the trackers are trained on broad classes of targets. At higher levels, they aim at more specific targets. For instance, at the root, a general blob tracker could operate which may track any object. The next level could already use information about human appearance to better track people. A further level could go after specific types of actions like walking, running, or sitting. Yet another level up, several walking trackers can be tuned to the gait of a particular person each. Thus, at each layer, one or more families of more specific trackers are available. As long as the target behaves according to expectations, a member of a higher up such family will be better tuned to the data than its parent tracker at a lower level. Typically, a better informed tracker performs more robustly. But in cases where unusual events occur and the normal assumptions about the world no longer hold, they loose their reliability. In such cases, a less informed tracker, not relying on what has now become false information, has a good chance of performing better. Such performance inversion signals an unusual event. Inversions between levels higher up represent deviations that are semantically more subtle than inversions lower down: for instance an unknown intruder entering a house rather than seeing a non-human target.

Temporal Relations in Videos for Unsupervised Activity Analysis.

Observing the different video sequences in Fig. 1, increments between frames are quite small compared to the changes throughout the whole sequence. For instance, the behavior of a tracked person (2nd row) is composed of a certain repertoire of activities with transitions in between that are typically short in comparison. This can also be observed at larger scales, like day-night changes or seasonal changes (3rd and 4th row) and already suggests a hierarchical structure.

Transferring activities: Updating human behavior analysis

One of the great open challenges in visual recognition is the ability to cope with unexpected stimuli. In this work, we present a technique to interpret detected anomalies and update the existing knowledge of normal situations. The addressed context is the analysis of human behavior in indoor surveillance scenarios, where new activities might need to be learned, once the system is already in operation. Our approach is based on human tracking with multiple activity trackers. The main contribution is to integrate a learning stage, where labeled and unlabeled information is collected and analyzed. To this end we develop a new multi-class version of transfer learning which requires minimal human interaction but still provides semantic labels of the new classes. The activity model is then updated with the new activities. Experiments show promising results.

Unsupervised workflow discovery in industrial environments

In this work, we present an approach for the automatic discovery of workflows in industrial environments. In such cluttered scenes, one faces many challenges, which limit the use of state-of-the-art object detection and tracking methods. Instead we propose a purely data-driven method which exploits the temporal structure of the workflow. Our robust technique is free of human intervention and does not need parameter tuning. We show results on two camera views of a working cell in a car assembly line. Workflows are extracted robustly, they match well across the camera views and they are conform with human annotation. Furthermore, we show a simple but efficient extension to analyze the image stream in real time. This assures a smooth running of the workflow and enables the notification of different types of unexpected scenarios.

Discrimination of locomotion direction at different speeds: A comparison between macaque monkeys and algorithms

Models for visual motion perception exist since some time in neurophysiology as well as computer vision. In this paper, we present a comparison between a behavioral study performed with macaque monkeys and the output of a computational model. The tasks include the discrimination between left and right walking directions and forward vs. backward walking. The goal is to measure generalization performance over different walking and running speeds. We show in which cases the results match, and discuss and interpret differences.