Markus Höferlin

Information-based adaptive fast-forward for visual surveillance

Automated video analysis lacks reliability when searching for unknown events in video data. The practical approach is to watch all the recorded video data, if applicable in fast-forward mode. In this paper we present a method to adapt the playback velocity of the video to the temporal information density, so that the users can explore the video under controlled cognitive load. The proposed approach can cope with static changes and is robust to video noise. First, we formulate temporal information as symmetrized Rényi divergence, deriving this measure from signal coding theory. Further, we discuss the animated visualization of accelerated video sequences and propose a physiologically motivated blending approach to cope with arbitrary playback velocities. Finally, we compare the proposed method with the current approaches in this field by experiments and a qualitative user study, and show its advantages over motion-based measures.

State of the Art Report on Video-Based Graphics and Video Visualization

In recent years, a collection of new techniques which deal with video as input data, emerged in computer graphics and visualization. In this survey, we report the state of the art in video‐based graphics and video visualization. We provide a review of techniques for making photo‐realistic or artistic computer‐generated imagery from videos, as well as methods for creating summary and/or abstract visual representations to reveal important features and events in videos. We provide a new taxonomy to categorize the concepts and techniques in this newly emerged body of knowledge. To support this review, we also give a concise overview of the major advances in automated video analysis, as some techniques in this field (e.g. feature extraction, detection, tracking and so on) have been featured in video‐based modelling and rendering pipelines for graphics and visualization.

Visual task solution strategies in tree diagrams

We investigate visual task solution strategies when exploring traditional, orthogonal, and radial node-link tree layouts, four orientations of the non-radial layouts, as well as varying difficulty of the task. The strategies are identified by examining eye movement data recorded in a controlled user study previously conducted by Burch et al. For detailed analysis of the spatio-temporal structures and patterns in the eye tracking data, we employ visual analytics techniques adopted from related methodology for geographic movement data by Andrienko et al. In this way, we complement the statistical analysis of task completion times and error rates reported by Burch et al. with spatio-temporal strategies that explain the variation in completion times. We identify differences between task solution strategies dependent on layout type, orientation, and task difficulty. Furthermore, we examine differences between groups of participants split according to completion time. Our analysis identifies that for all layouts it took nearly the same time to find the task solution node, but in the radial layout the solution was not confirmed directly. Instead, a more frequent cross-checking occurs afterwards, which is the main reason for the impaired performance of radial layouts.

Video visualization for snooker skill training

We present a feasibility study on using video visualization to aid snooker skill training. By involving the coaches and players in the loop of intelligent reasoning, our approach addresses the difficulties of automated semantic reasoning, while benefiting from mature video processing techniques. This work was conducted in conjunction with a snooker club and a sports scientist. In particular, we utilized the principal design of the VideoPerpetuoGram (VPG) to convey spatiotemporal information to the viewers through static visualization, removing the burden of repeated video viewing. We extended the VPG design to accommodate the need for depicting multiple video streams and respective temporal attribute fields, including silhouette extrusion, spatial attributes, and non‐spatial attributes. Our results and evaluation have shown that video visualization can provide snooker coaching with visually quantifiable and comparable summary records, and is thus a cost‐effective means for assessing skill levels and monitoring progress objectively and consistently.

A Survey on Video-based Graphics and Video Visualization

In recent years, a collection of new techniques, which deal with videos a s the input data, emerged in computer graphics and visualization. In this survey, we report the state of the art in video-based graphics and video visualization. We provide a comprehensive review of techniques for making photo- realistic or artistic computer-generated imagery from videos, as well as methods for creating summary and/or abs tract visual representations to reveal important features and events in videos. We propose a new taxonomy to ca tegorize the concepts and techniques in this newly-emerged body of knowledge. To support this review, we als o give a concise overview of the major advances in automated video analysis, as some techniques in this field (e.g ., feature extraction, detection, tracking and so on) have been featured in video-based modeling and rendering p ipelines for graphics and visualization.

Uncertainty-aware video visual analytics of tracked moving objects

Vast amounts of video data render manual video analysis useless while recent automatic video analytics techniques suffer from insufficient performance. To alleviate these issues, we present a scalable and reliable approach exploiting the visual analytics methodology. This involves the user in the iterative process of exploration, hypotheses generation, and their verification. Scalability is achieved by interactive filter definitions on trajectory features extracted by the automatic computer vision stage. We establish the interface between user and machine adopting the VideoPerpetuoGram (VPG) for visual- ization and enable users to provide filter-based relevance feedback. Additionally, users are supported in deriving hypotheses by context-sensitive statistical graphics. To allow for reliable decision making, we gather uncertainties introduced by the computer vision step, communicate these information to users through uncertainty visualization, and grant fuzzy hypothesis formulation to interact with the machine. Finally, we demonstrate the effectiveness of our approach by the video analysis mini challenge which was part of the IEEE Symposium on Visual Analytics Science and Technology 2009.

Interactive Schematic Summaries for Faceted Exploration of Surveillance Video

We present a scalable technique to explore surveillance videos by scatter/gather browsing of trajectories of moving objects. Trajectories are clustered according to a variety of properties, such as location, orientation, and velocity that can be selected by the users. These properties allow for faceted video exploration and refinement of previous browsing steps. The proposed approach facilitates interactive clustering of trajectories by an effective way of cluster visualization that we term schematic summaries. This novel visualization illustrates cluster summaries in a schematic, nonphotorealistic style. To reduce visual clutter, we introduce the trajectory bundling technique. Further, schematic summaries include a timeline view and a showcase view to represent the facets present in a cluster. The fusion of schematic summaries, a variety of facets, and user interaction lead to efficient hierarchical exploration of video data. Examples of different browsing scenarios and initial user feedback demonstrate the potentials of our method.

Layered TimeRadarTrees

We introduce a novel technique for visualizing dense time-varying directed and weighted multi-graphs with an additional hierarchical organization of the graph nodes. Combining Indented Tree Plots and TimeRadarTrees, we show the temporal evolution of relations in a static view. The graph edges are layered around thumbnail wheels consisting of color-coded sectors that are representatives of the graph nodes. These sectors generate implicit representations of graph edges. Start and target vertices are perceived by inspecting the color coding of sectors in the context of other sectors and their orientations. The technique puts emphasis on newer relations and hence, these are mapped to a larger display space in the radial diagram. The benefit of our technique is reduction of visual clutter from which node-link diagrams typically suffer. The visualization focuses on an easy exploration of trends, countertrends, periodicity, temporal shifts, and anomalies in time-varying relational data. We demonstrate the usefulness of the approach by applying it to dense dynamic graph data acquired from a soccer match of the 2D Soccer Simulation League.

Evaluation of Fast-Forward Video Visualization

We evaluate and compare video visualization techniques based on fast-forward. A controlled laboratory user study (n = 24) was conducted to determine the trade-off between support of object identification and motion perception, two properties that have to be considered when choosing a particular fast-forward visualization. We compare four different visualizations: two representing the state-of-the-art and two new variants of visualization introduced in this paper. The two state-of-the-art methods we consider are frame-skipping and temporal blending of successive frames. Our object trail visualization leverages a combination of frame-skipping and temporal blending, whereas predictive trajectory visualization supports motion perception by augmenting the video frames with an arrow that indicates the future object trajectory. Our hypothesis was that each of the state-of-the-art methods satisfies just one of the goals: support of object identification or motion perception. Thus, they represent both ends of the visualization design. The key findings of the evaluation are that object trail visualization supports object identification, whereas predictive trajectory visualization is most useful for motion perception. However, frame-skipping surprisingly exhibits reasonable performance for both tasks. Furthermore, we evaluate the subjective performance of three different playback speed visualizations for adaptive fast-forward, a subdomain of video fast-forward.

Interactive schematic summaries for exploration of surveillance video

We present a new and scalable technique to explore surveillance videos by scatter/gather browsing of trajectories of moving objects. The proposed approach facilitates interactive clustering of trajectories by an effective way of cluster visualization that we term schematic summaries. This novel visualization illustrates cluster summaries in a schematic, non-photorealistic style. To reduce visual clutter, we introduce the trajectory bundling technique. The fusion of schematic summaries and user interaction leads to efficient hierarchical exploration of video data. Examples of different browsing scenarios demonstrate the effectiveness of the proposed method.