Janez Perš

A trajectory-based analysis of coordinated team activity in a basketball game

This paper proposes a novel, trajectory-based approach to the automatic recognition of complex multi-player behavior in a basketball game. First, a probabilistic play model is applied to the player-trajectory data in order to segment the play into game phases (offense, defense, time out). In this way, both the temporal boundaries of the observed activity and its broader context are obtained. Next, the teams activity is analyzed in more detail by detecting the key elements of basketball play. Following basketball theory, these key elements (starting formation, screen, and move) are the building blocks of basketball play, and therefore their temporal order is used to produce a semantic description of the observed activity. Finally, the activity is recognized by comparing its semantic description with the descriptions of manually defined templates, stored in a database. The effectiveness and robustness of the proposed approach is demonstrated on two championship games and 71 examples of three types of basketball offense.

Observation and analysis of large-scale human motion

Many team sports include complex human movement, which can be observed at different levels of detail. Some aspects of the athletes motion can be studied in detail using commercially available high-speed, high-accuracy biomechanical measurement systems. However, due to their limitations, these devices are not appropriate for studying large-scale motion during a game (for example, the motion of a player running across the entire playing field). We describe an alternative approach to studying such large-scale motion, and present a video-based, computer-aided system, developed specifically for the purpose of acquiring large-scale motion data. The baseline of our approach consists of sacrificing much of the spatial accuracy and temporal resolution of widely used biomechanical measurement systems, to obtain data on human movement that span large areas and long intervals of time. Data can be obtained for each of the observed athletes with reasonable amount of operator work. The system was developed using the recordings of a handball match. Several field tests were performed to assess measurement error, including comparison to one of the widely available biomechanical measurement systems. With the help of the system presented, we could obtain position data for all 14 handball players on a 40 x 20 m large court with RMS error better than 0.6 m, covering 1 h of action. Several results, obtained during the handball match study are presented, in order to highlight the importance of large-scale motion acquisition.

A Bayes-spectral-entropy-based measure of camera focus using a discrete cosine transform

In this paper we present a novel measure of camera focus based on the Bayes spectral entropy of an image spectrum. In order to estimate the degree of focus, the image is divided into non-overlapping sub-images of 8x8 pixels. Next, sharpness values are calculated separately for each sub-image and their mean is taken as a measure of the overall focus. The sub-image spectra are obtained by an 8x8 discrete cosine transform (DCT). Comparisons were made against four well-known measures that were chosen as reference, on images captured with a standard visible-light camera and a thermal camera. The proposed measure outperformed the reference measures by exhibiting a wider working range and a smaller failure rate. To assess its robustness to noise, additional tests were conducted with noisy images.

Closed-world tracking of multiple interacting targets for indoor-sports applications

In this paper we present an efficient algorithm for tracking multiple players during indoor sports matches. A sports match can be considered as a semi-controlled environment for which a set of closed-world assumptions regarding the visual as well as the dynamical properties of the players and the court can be derived. These assumptions are then used in the context of particle filtering to arrive at a computationally fast, closed-world, multi-player tracker. The proposed tracker is based on multiple, single-player trackers, which are combined using a closed-world assumption about the interactions among players. With regard to the visual properties, the robustness of the tracker is achieved by deriving a novel sports-domain-specific likelihood function and employing a novel background-elimination scheme. The restrictions on the players dynamics are enforced by employing a novel form of local smoothing. This smoothing renders the tracking more robust and reduces the computational complexity of the tracker. We evaluated the proposed closed-world, multi-player tracker on a challenging data set. In comparison with several similar trackers that did not utilize all of the closed-world assumptions, the proposed tracker produced better estimates of position and prediction as well as reducing the number of failures.

A Two-Stage Dynamic Model for Visual Tracking

We propose a new dynamic model which can be used within blob trackers to track the targets center of gravity. A strong point of the model is that it is designed to track a variety of motions which are usually encountered in applications such as pedestrian tracking, hand tracking, and sports. We call the dynamic model a two-stage dynamic model due to its particular structure, which is a composition of two models: a liberal model and a conservative model. The liberal model allows larger perturbations in the targets dynamics and is able to account for motions in between the random-walk dynamics and the nearly constant-velocity dynamics. On the other hand, the conservative model assumes smaller perturbations and is used to further constrain the liberal model to the targets current dynamics. We implement the two-stage dynamic model in a two-stage probabilistic tracker based on the particle filter and apply it to two separate examples of blob tracking: 1) tracking entire persons and 2) tracking of a persons hands. Experiments show that, in comparison to the widely used models, the proposed two-stage dynamic model allows tracking with smaller number of particles in the particle filter (e.g., 25 particles), while achieving smaller errors in the state estimation and a smaller failure rate. The results suggest that the improved performance comes from the models ability to actively adapt to the targets motion during tracking.

Histograms of optical flow for efficient representation of body motion

A novel method for efficient encoding of human body motion, extracted from image sequences is presented. Optical flow field is calculated from sequential images, and the part of the flow field containing a person is subdivided into six segments. For each of the segments, a two dimensional, eight-bin histogram of optical flow is calculated. A symbol is generated, corresponding to the bin with the maximum sample count. Since the optical flow sequences before and after the temporal reference point are processed separately, twelve symbol sequences are obtained from the whole image sequence. Symbol sequences are purged of all symbol repetitions. To establish the similarity between two motion sequences, two sets of symbol sequences are compared. In our case, this is done by the means of normalized Levenshtein distance. Due to use of symbol sequences, the method is extremely storage efficient. It is also performance efficient, as it could be performed in near-realtime using the motion vectors from MPEG4 encoded video sequences. The approach has been tested on video sequences of persons entering restricted area using keycard and fingerprint reader. We show that it could be applied both to verification of person identities due to minuscule differences in their motion, and to detection of unusual behavior, such as tailgating.

Trajectory based assessment of coordinated human activity

Most approaches to detection and classification of human activity deal with observing individual persons. However, people often tend to organize into groups to achieve certain goals, and human activity is sometimes more readily defined and observed in the context of whole group, where the activity is coordinated among its members. An excellent example of this are team sports, which can provide valuable test ground for development of methods for analysis of coordinated group activity. We used basketball play in this work and developed a probabilistic model of a team play, which is based on the detection of key events in the team behavior. The model is based on expert coach knowledge and has been used to assess the team performance in three different types of basketball offense, based on trajectories of all players, obtained by whole-body tracker. Results show that our high-level behaviour model may be used both for activity recognition and performance evaluation in certain basketball activities.

Tracking People in Sport: Making Use of Partially Controlled Environment

Many different methods for tracking humans were proposed in the past several years, yet surprisingly only a few authors examined the accuracy of the proposed systems. As the accuracy analysis is impossible without the well-defined ground truth, some kind of at least partially controlled environment is needed. Analysis of an athlete motion in sport match is well suited for that purpose, and it coincides with the need of the sport research community for accurate and reliable results of motion acquisition. This paper presents a development of a two-camera people tracker, incorporating two complementary tracking algorithms. The developed system is suited for simultaneously tracking several people on a large area of a handball court, using a sequence of 384-by-288 pixel images from fixed cameras. We also examine the level of accuracy that this kind of computer vision system setup is capable of.

Tracking by Identification Using Computer Vision and Radio

We present a novel system for detection, localization and tracking of multiple people, which fuses a multi-view computer vision approach with a radio-based localization system. The proposed fusion combines the best of both worlds, excellent computer-vision-based localization, and strong identity information provided by the radio system, and is therefore able to perform tracking by identification, which makes it impervious to propagated identity switches. We present comprehensive methodology for evaluation of systems that perform person localization in world coordinate system and use it to evaluate the proposed system as well as its components. Experimental results on a challenging indoor dataset, which involves multiple people walking around a realistically cluttered room, confirm that proposed fusion of both systems significantly outperforms its individual components. Compared to the radio-based system, it achieves better localization results, while at the same time it successfully prevents propagation of identity switches that occur in pure computer-vision-based tracking.

Fast Image-Based Obstacle Detection From Unmanned Surface Vehicles

Obstacle detection plays an important role in unmanned surface vehicles (USVs). The USVs operate in a highly diverse environments in which an obstacle may be a floating piece of wood, a scuba diver, a pier, or a part of a shoreline, which presents a significant challenge to continuous detection from images taken on board. This paper addresses the problem of online detection by constrained, unsupervised segmentation. To this end, a new graphical model is proposed that affords a fast and continuous obstacle image-map estimation from a single video stream captured on board a USV. The model accounts for the semantic structure of marine environment as observed from USV by imposing weak structural constraints. A Markov random field framework is adopted and a highly efficient algorithm for simultaneous optimization of model parameters and segmentation mask estimation is derived. Our approach does not require computationally intensive extraction of texture features and comfortably runs in real time. The algorithm is tested on a new, challenging, dataset for segmentation, and obstacle detection in marine environments, which is the largest annotated dataset of its kind. Results on this dataset show that our model outperforms the related approaches, while requiring a fraction of computational effort.