David H. S. Chung

Glyph-Based Visualization: Foundations, Design Guidelines, Techniques, and Applications

This state of the art report focuses on glyph-based visualization, a common form of visual design where a data set is depicted by a collection of visual objects referred to as glyphs. Its major strength is that patterns of multivariate data involving more than two attribute dimensions can often be more readily perceived in the context of a spatial relationship, whereas many techniques for spatial data such as direct volume rendering find difficult to depict with multivariate or multi-field data, and many techniques for non-spatial data such as parallel coordinates are less able to convey spatial relationships encoded in the data. This report fills several major gaps in the literature, drawing the link between the fundamental concepts in semiotics and the broad spectrum of glyph-based visualization, reviewing existing design guidelines and implementation techniques, and surveying the use of glyph-based visualization in many applications.

MatchPad : Interactive Glyph-Based Visualization for Real-Time Sports Performance Analysis

Today real‐time sports performance analysis is a crucial aspect of matches in many major sports. For example, in soccer and rugby, team analysts may annotate videos during the matches by tagging specific actions and events, which typically result in some summary statistics and a large spreadsheet of recorded actions and events. To a coach, the summary statistics (e.g., the percentage of ball possession) lacks sufficient details, while reading the spreadsheet is time‐consuming and making decisions based on the spreadsheet in real‐time is thereby impossible. In this paper, we present a visualization solution to the current problem in real‐time sports performance analysis. We adopt a glyph‐based visual design to enable coaching staff and analysts to visualize actions and events “at a glance”. We discuss the relative merits of metaphoric glyphs in comparison with other types of glyph designs in this particular application. We describe an algorithm for managing the glyph layout at different spatial scales in interactive visualization. We demonstrate the use of this technical approach through its application in rugby, for which we delivered the visualization software, MatchPad, on a tablet computer. The MatchPad was used by the Welsh Rugby Union during the Rugby World Cup 2011. It successfully helped coaching staff and team analysts to examine actions and events in detail whilst maintaining a clear overview of the match, and assisted in their decision making during the matches. It also allows coaches to convey crucial information back to the players in a visually‐engaging manner to help improve their performance.

Hierarchical Event Selection for Video Storyboards with a Case Study on Snooker Video Visualization

Video storyboard, which is a form of video visualization, summarizes the major events in a video using illustrative visualization. There are three main technical challenges in creating a video storyboard, (a) event classification, (b) event selection and (c) event illustration. Among these challenges, (a) is highly application-dependent and requires a significant amount of application specific semantics to be encoded in a system or manually specified by users. This paper focuses on challenges (b) and (c). In particular, we present a framework for hierarchical event representation, and an importance-based selection algorithm for supporting the creation of a video storyboard from a video. We consider the storyboard to be an event summarization for the whole video, whilst each individual illustration on the board is also an event summarization but for a smaller time window. We utilized a 3D visualization template for depicting and annotating events in illustrations. To demonstrate the concepts and algorithms developed, we use Snooker video visualization as a case study, because it has a concrete and agreeable set of semantic definitions for events and can make use of existing techniques of event detection and 3D reconstruction in a reliable manner. Nevertheless, most of our concepts and algorithms developed for challenges (b) and (c) can be applied to other application areas.

Glyph sorting: Interactive visualization for multi-dimensional data

Glyph-based visualization is an effective tool for depicting multivariate information. Since sorting is one of the most common analytical tasks performed on individual attributes of a multi-dimensional dataset, this motivates the hypothesis that introducing glyph sorting would significantly enhance the usability of glyph-based visualization. In this article, we present a glyph-based conceptual framework as part of a visualization process for interactive sorting of multivariate data. We examine several technical aspects of glyph sorting and provide design principles for developing effective, visually sortable glyphs. Glyphs that are visually sortable provide two key benefits: (1) performing comparative analysis of multiple attributes between glyphs and (2) to support multi-dimensional visual search. We describe a system that incorporates focus and context glyphs to control sorting in a visually intuitive manner and for viewing sorted results in an interactive, multi-dimensional glyph plot that enables users to perform high-dimensional sorting, analyse and examine data trends in detail. To demonstrate the usability of glyph sorting, we present a case study in rugby event analysis for comparing and analysing trends within matches. This work is undertaken in conjunction with a national rugby team. From using glyph sorting, analysts have reported the discovery of new insight beyond traditional match analysis.

Visual reconstructability as a quality metric for flow visualization

We present a novel approach for the evaluation of 2D flow visualizations based on the visual reconstructability of the input vector fields. According to this metric, a visualization has high quality if the underlying data can be reliably reconstructed from the image. This approach provides visualization creators with a cost‐effective means to assess the quality of visualization results objectively. We present a vision‐based reconstruction system for the three most commonly‐used visual representations of vector fields, namely streamlines, arrow glyphs, and line integral convolution. To demonstrate the use of visual reconstructability as a quality metric, we consider a selection of vector fields obtained from numerical simulations, containing typical flow features. We apply the three types of visualization to each dataset, and compare the visualization results based on their visual reconstructability of the original vector field.

Transformation of an Uncertain Video Search Pipeline to a Sketch-Based Visual Analytics Loop

Traditional sketch-based image or video search systems rely on machine learning concepts as their core technology. However, in many applications, machine learning alone is impractical since videos may not be semantically annotated sufficiently, there may be a lack of suitable training data, and the search requirements of the user may frequently change for different tasks. In this work, we develop a visual analytics systems that overcomes the shortcomings of the traditional approach. We make use of a sketch-based interface to enable users to specify search requirement in a flexible manner without depending on semantic annotation. We employ active machine learning to train different analytical models for different types of search requirements. We use visualization to facilitate knowledge discovery at the different stages of visual analytics. This includes visualizing the parameter space of the trained model, visualizing the search space to support interactive browsing, visualizing candidature search results to support rapid interaction for active learning while minimizing watching videos, and visualizing aggregated information of the search results. We demonstrate the system for searching spatiotemporal attributes from sports video to identify key instances of the team and player performance.

Intelligent filtering by semantic importance for single-view 3D reconstruction from Snooker video

In this paper we investigate the challenge of 3D reconstruction from Snooker video data. We propose a system pipeline for intelligent filtering based on semantic importance in Snooker. The system can be divided into table detection and correction, followed by ball detection, classification and tracking. It is apparent from previous work that there are several challenges presented here. Firstly, previous methods tend to use a fixed top-down camera mounted above the table. To capture a full table view from this is challenging due to space limitations above the table. Instead, we capture video data from a tripod and correct the viewpoint through processing. Secondly, previous methods tend to simply detect the balls without considering other interfering objects such as player and cue. This becomes even more apparent when the player strikes the cue ball. Our intelligent filtering avoids such issues to give accurate 3D table reconstruction.

Automated 3-D Animation From Snooker Videos With Information-Theoretical Optimization

Automated 3-D modeling from real sports videos can provide useful resources for visual design in sports-related computer games, saving a lot of effort in manual design of visual contents. However, image-based 3-D reconstruction usually suffers from inaccuracy caused by statistic image analysis. In this paper, we propose an information-theoretical scheme to minimize errors of automated 3-D modeling from monocular sports videos. In the proposed scheme, mutual information (MI) was exploited to compute the fitting scores of a 3-D model against the observed single-view scene, and the optimization of model fitting was carried out subsequently. With this optimization scheme, errors in model fitting were minimized without human intervention, allowing automated reconstruction of 3-D animation from consecutive monocular video frames at high accuracy. In our work, the Snooker videos were taken as our case study, balls were positioned in 3-D space from single-view frames, and 3-D animation was reproduced from real Snooker videos. Our experimental results validated that the proposed information-theoretical scheme can help attain better accuracy in the automated reconstruction of 3-D animation, and demonstrated that information-theoretical evaluation can be an effective approach for model-based reconstruction from single-view videos.

How ordered is it?: on the perceptual orderability of visual channels

The design of effective glyphs for visualisation involves a number of different visual encodings. Since spatial position is usually already specified in advance, we must rely on other visual channels to convey additional relationships for multivariate analysis. One such relationship is the apparent order present in the data. This paper presents two crowdsourcing empirical studies that focus on the perceptual evaluation of orderability for visual channels, namely Bertins retinal variables. The first study investigates the perception of order in a sequence of elements encoded with different visual channels. We found evidence that certain visual channels are perceived as more ordered (for example, value) while others are perceived as less ordered (for example, hue) than the measured order present in the data. As a result, certain visual channels are more/less sensitive to disorder. The second study evaluates how visual orderability affects min and max judgements of elements in the sequence. We found that visual channels that tend to be perceived as ordered, improve the accuracy of identifying these values.

Knowledge-Assisted Ranking: A Visual Analytic Application for Sports Event Data

Organizing sports video data for performance analysis can be challenging, especially in cases involving multiple attributes and when the criteria for sorting frequently changes depending on the users task. The proposed visual analytic system enables users to specify a sort requirement in a flexible manner without depending on specific knowledge about individual sort keys. The authors use regression techniques to train different analytical models for different types of sorting requirements and use visualization to facilitate knowledge discovery at different stages of the process. They demonstrate the system with a rugby case study to find key instances for analyzing team and player performance. Organizing sports video data for performance analysis can be challenging in cases with multiple attributes, and when sorting frequently changes depending on the users task. As this video shows, the proposed visual analytic system allows interactive data sorting and exploration.https://youtu.be/Cs6SLtPVDQQ.