Rok Mandeljc

Tracking by Identification Using Computer Vision and Radio

We present a novel system for detection, localization and tracking of multiple people, which fuses a multi-view computer vision approach with a radio-based localization system. The proposed fusion combines the best of both worlds, excellent computer-vision-based localization, and strong identity information provided by the radio system, and is therefore able to perform tracking by identification, which makes it impervious to propagated identity switches. We present comprehensive methodology for evaluation of systems that perform person localization in world coordinate system and use it to evaluate the proposed system as well as its components. Experimental results on a challenging indoor dataset, which involves multiple people walking around a realistically cluttered room, confirm that proposed fusion of both systems significantly outperforms its individual components. Compared to the radio-based system, it achieves better localization results, while at the same time it successfully prevents propagation of identity switches that occur in pure computer-vision-based tracking.

Visual re-identification across large, distributed camera networks

We propose a holistic approach to the problem of re-identification in an environment of distributed smart cameras. We model the re-identification process in a distributed camera network as a distributed multi-class classifier, composed of spatially distributed binary classifiers. We treat the problem of re-identification as an open-world problem, and address novelty detection and forgetting. As there are many tradeoffs in design and operation of such a system, we propose a set of evaluation measures to be used in addition to the recognition performance. The proposed concept is illustrated and evaluated on a new many-camera surveillance dataset and SAIVT-SoftBio dataset. Display Omitted Formalization of object re-identification problem in a distributed environmentRe-identification treated as an open-world problemNovelty detection and forgetting included in the schemeA set of performance measures, geared towards open-world, distributed surveillanceExperiments on a many-camera (36) surveillance dataset and publicly available source code

Dana36: A Multi-camera Image Dataset for Object Identification in Surveillance Scenarios

We present a novel dataset for evaluation of object matching and recognition methods in surveillance scenarios. Dataset consists of more than 23,000 images, depicting 15 persons and nine vehicles. A ground truth data - the identity of each person or vehicle - is provided, along with the coordinates of the bounding box in the full camera image. The dataset was acquired from 36 stationary camera views using a variety of surveillance cameras with resolutions ranging from standard VGA to three megapixel. 27 cameras observed the persons and vehicles in an outdoor environment, while the remaining nine observed the same persons indoors. The activity of persons was planned in advance, they drive the cars to the parking lot, exit the cars and walk around the building, through the main entrance, and up the stairs, towards the first floor of the building. The intended use of the dataset is performance evaluation of computer vision methods that aim to (re)identify people and objects from many different viewpoints in different environments and under variable conditions. Due to variety of camera locations, vantage points and resolutions, the dataset provides means to adjust the difficulty of the identification task in a controlled and documented manner. An interface for easy use of dataset within Matlab is provided as well, and the data is complemented by baseline results using a basic color histogram-based descriptor. While the cropped images of persons and vehicles represent the primary data in our dataset, we also provide full-frame images and a set of tracklets for each object as a courtesy to the dataset users.

Fusion of non-visual modalities into the Probabilistic Occupancy Map framework for person localization

In this paper we investigate the possibilities for fusion of non-visual sensor modalities into state-of-the-art vision-based framework for person detection and localization, the Probabilistic Occupancy Map (POM), with the aim of improving the frame-by-frame localization results in a realistic (cluttered) indoor environment. We point out the aspects that need to be considered when fusing non-visual sensor information into POM and provide a mathematical model for it. We demonstrate the proposed fusion method on the example of multi-camera and radio-based person localization setup. The performance of both systems is evaluated, showing their strengths and weaknesses. We show that localization results may be significantly improved by fusing the information from the radio-based system into the camera-based POM framework using the proposed model.

Towards commoditized smart-camera design

We propose a set of design principles for a cost-effective embedded smart camera. Our aim is to alleviate the shortcomings of the existing designs, such as excessive reliance on battery power and wireless networking, over-emphasized focus on specific use cases, and use of specialized technologies. In our opinion, these shortcomings prevent widespread commercialization and adoption of embedded smart cameras, especially in the context of visual-sensor networks. The proposed principles lead to a distinctively different design, which relies on commoditized, standardized and widely-available components, tools and knowledge. As an example of using these principles in practice, we present a smart camera, which is inexpensive, easy to build and support, capable of high-speed communication and enables rapid transfer of computer-vision algorithms to the embedded world.

Part-based room categorization for household service robots

A service robot that operates in a previously-unseen home environment should be able to recognize the functionality of the rooms it visits, such as a living room, a bathroom, etc. We present a novel part-based model and an approach for room categorization using data obtained from a visual sensor. Images are represented with sets of unordered parts that are obtained by object-agnostic region proposals, and encoded using state-of-the-art image descriptor extractor - a convolutional neural network (CNN). An approach is proposed that learns category-specific discriminative parts for the part-based model. The proposed approach was compared to the state-of-the-art CNN trained specifically for place recognition. Experimental results show that the proposed approach outperforms the holistic CNN by being robust to image degradation, such as occlusions, modifications of image scaling, and aspect changes. In addition, we report non-negligible annotation errors and image duplicates in a popular dataset for place categorization and discuss annotation ambiguities.

Effects of rule changes on physical demands and shot characteristics of elite-standard men’s squash and implications for training

ABSTRACT The physical demands and rally characteristics of elite-standard men’s squash have not been well documented since recent rule changes (scoring and tin height). This information is needed to design optimal training drills for physical conditioning provided here based on an analysis of movement and shot information. Matches at the 2010 (n = 14) and 2011 (n = 27) Rowe British Grand Prix were analysed. Rallies were split into four ball-in-play duration categories using the 25th (short), 75th (medium), 95th percentiles (long) and maximum values. Cohen’s d and chi-squared tests of independence evaluated effects of rally and rule changes on patterns of play. The proportion of long, middle and short shots was related to the duration of the rally with more shots played in the middle and front of the court in short rallies (phi = 0.12). The frequencies of shots played from different areas of the court have not changed after the adoption of new rules but there is less time available to return shots that reflect the attacking nature of match play for elite-standard men players. Aspiring and current elite-standard players need to condition themselves to improve their ability to cope with these demands using the ghosting patterns presented that mimic demands of modern match play.

Using a situation awareness approach to determine decision-making behaviour in squash

ABSTRACT Situation awareness (SA) refers to the awareness of all relevant sources of information, an ability to synthesise this information using domain knowledge gained from past experiences and the ability to physically respond to a situation. Expert-novice differences have been widely reported in decision-making in complex situations although determining the small differences in expert behaviour are more elusive. This study considered how expert squash players use SA to decide on what shot to play. Matches at the 2010 (n = 14) and 2011 (n = 27) Rowe British Grand Prix were recorded and processed using Tracker software. Shot type, ball location, players’ positions on court and movement parameters between the time an opponent played a shot prior to the player’s shot to the time of the opponent’s following shot were captured 25 times per second. Six SA clusters were named to relate to the outcome of a shot ranging from a defensive shot played under pressure to create time to an attempted winner played under no pressure with the opponent out of position. This new methodology found fine-grained SA differences in expert behaviour, even for the same shot type played from the same court area, beyond the usual expert-novice differences.

Non-sequential multi-view detection, localization and identification of people using multi-modal feature maps

We present a novel multi-modal fusion framework for non-sequential person detection, localization and identification from multiple views. Our goal is independent processing of randomly-accessed sections of video, either individual frames or small batches thereof. This way, we aim to limit the error propagation that makes the existing approaches unsuitable for fully-autonomous tracking of multiple people in long video sequences. Our framework uses one or more trained classifiers to fuse multiple weak feature maps. We perform experimental validation on a challenging dataset, demonstrating how the framework can, depending on the provided feature maps, be used either only to improve generic person detection, or enable simultaneous detection and recognition of individuals. Finally, we show that tracking-by-identification using the output of the proposed framework outperforms the state-of-the-art identification-by-tracking approach in terms of preserved track identities.

Stereo obstacle detection for unmanned surface vehicles by IMU-assisted semantic segmentation

Abstract A new obstacle detection algorithm for unmanned surface vehicles (USVs) is presented. A state-of-the-art graphical model for semantic segmentation is extended to incorporate boat pitch and roll measurements from the on-board inertial measurement unit (IMU), and a stereo verification algorithm that consolidates tentative detections obtained from the segmentation is proposed. The IMU readings are used to estimate the location of horizon line in the image, which automatically adjusts the priors in the probabilistic semantic segmentation model. We derive the equations for projecting the horizon into images, propose an efficient optimization algorithm for the extended graphical model, and offer a practical IMU–camera–USV calibration procedure. Using an USV equipped with multiple synchronized sensors, we captured a new challenging multi-modal dataset, and annotated its images with water edge and obstacles. Experimental results show that the proposed algorithm significantly outperforms the state of the art, with nearly 30% improvement in water-edge detection accuracy, an over 21% reduction of false positive rate, an almost 60% reduction of false negative rate, and an over 65% increase of true positive rate, while its Matlab implementation runs in real-time.