Tomislav Hrkać

Infrared-visual image registration based on corners and hausdorff distance

The paper presents an approach to multimodal image registration. The method is developed for aligning infrared (IR) and visual (RGB) images of facades. It is based on mapping clouds of points extracted by a corner detector applied to both images. The experiments show that corners are suitable features for our application. In the alignment process a number of transformation hypotheses is generated and evaluated. The evaluation is performed by measuring similarity between the RGB corners and the transformed corners from IR image. Directed partial Hausdorff distance is used as a robust similarity measure. The implemented system has been tested on various IR-RGB pairs of images of buildings. The results show that the method can be used for image registration, but also expose some typical problems.

A model of fuzzy spatio-temporal knowledge representation and reasoning based on high-level Petri nets

In many application areas there is a need to represent human-like knowledge related to spatio-temporal relations among multiple moving objects. This type of knowledge is usually imprecise, vague and fuzzy, while the reasoning about spatio-temporal relations is intuitive. In this paper we present a model of fuzzy spatio-temporal knowledge representation and reasoning based on high-level Petri nets. The model should be suitable for the design of a knowledge base for real-time, multi-agent-based intelligent systems that include expert or user human-like knowledge. The central part of the model is the knowledge representation scheme called FuSpaT, which supports the representation and reasoning for domains that include imprecise and fuzzy spatial, temporal and spatio-temporal relationships. The scheme is based on the high-level Petri nets called Petri nets with fuzzy spatio-temporal tokens (PeNeFuST). The FuSpaT scheme integrates the theory of the PeNeFuST and 117 spatio-temporal relations. The reasoning in the proposed model is a spatio-temporal data-driven process based on the dynamical properties of the scheme, i.e., the execution of the Petri nets with fuzzy spatio-temporal tokens. An illustrative example of the spatio-temporal reasoning for two agents in a simplified robot-soccer scene is given.

A Knowledge Representation and Reasoning Based on Petri Nets with Spatio-Temporal Tokens

Knowledge representation of time and space, and reasoning about temporal and spatial relations, are important areas in fields such as visual object tracking, robot vision, multimedia, geographical information systems, etc. In this paper we present a formal model of knowledge representation and reasoning for spatio-temporal domains. The model is based on the high-level Petri nets called the Petri Nets with Spatio-Temporal Tokens (PNSTT), which are used as the main building block of the knowledge representation scheme called SpaTem. The SpaTem scheme integrates the theory of PNSTT and 117 spatio-temporal relations. Spatio-temporal reasoning is a spatio-temporal data-driven process based on a combination of nine spatial and thirteen temporal relations adopted for robot-vision applications, where the objects in the scene are represented by their centroids. An illustrative example of spatio-temporal reasoning for two mobile robots on a two-dimensional grid world is described.

TeMAS–a multi-agent system for temporally rich domains

In this paper, we present the model and simulator of a multi-agent system (MAS) for temporally rich domains. The theoretical foundations of the model include a knowledge representation scheme based on an original modification of Petri nets, called Petri nets with time tokens (PNTTs), as well as temporal reasoning based on the extension of Allens temporal logic. The proposed MAS, called TeMAS, has a hierarchical structure, consisting of different levels, where each level contains clusters of agents. A paradigm of hierarchically organized blackboards is used for the communication among agents, clusters, as well as levels. We describe an object-oriented implementation of a program simulator of TeMAS and give an example of the use of the simulator for interpretation of events in a dynamic scene.

Deep metric learning for person Re-identification and De-identification

Large amounts of visual data are gathered from various surveillance systems across different places and times, and have to be processed in order to infer the current state of the world. One of the common problems in surveillance scenarios is person re-identification, the task of associating a person across different cameras. On the other hand, these scenarios raise privacy concerns, which lead to the need for person de-identification, i.e. concealing person identity. This task is related to the re-identification in two aspects: (i) multiple appearances of the same person could be de-identified in similar manner; and (ii) if we discover the features useful for re-identification, we could try to hide the identity by modifying those features. Re-identification can be addressed as a classification problem. The state-of-the-art classification methods are based on deep learning. In this paper we explore the applicability of the recently proposed Triplet network architecture to the person re-identification problem, by applying it on VIPeR dataset. We show that the network is able to learn useful feature-space embeddings, and analyze its benefits and limitations.

I Know That Person: Generative Full Body and Face De-identification of People in Images

We propose a model for full body and face deidentification of humans in images. Given a segmentation of the human figure, our model generates a synthetic human image with an alternative appearance that looks natural and fits the segmentation outline. The model is usable with various levels of segmentation, from simple human figure blobs to complex garment-level segmentations. The level of detail in the de-identified output depends on the level of detail in the input segmentation. The model de-identifies not only primary biometric identifiers (e.g. the face), but also soft and non-biometric identifiers including clothing, hairstyle, etc. Quantitative and perceptual experiments indicate that our model produces de-identified outputs that thwart human and machine recognition, while preserving data utility and naturalness.

Towards neural art-based face de-identification in video data

We propose a computer vision-based pipeline that enables altering the appearance of faces in videos. Assuming a surveillance scenario, we combine GMM-based background subtraction with an improved version of the GrabCut algorithm to find and segment pedestrians. Independently, we detect faces using a standard face detector. We apply the neural art algorithm, utilizing the responses of a deep neural network to obfuscate the detected faces through style mixing with reference images. The altered faces are combined with the original frames using the extracted pedestrian silhouettes as a guideline. Experimental evaluation indicates that our method has potential in producing de-identified versions of the input frames while preserving the utility of the de-identified data.

Two-stage cascade model for unconstrained face detection

In this paper, we propose a two-stage model for unconstrained face detection. The first stage is based on the normalized pixel difference (NPD) method, and the second stage uses the deformable part model (DPM) method. The NPD method applied to in the wild image datasets outputs the unbalanced ratio of false positive to false negative face detection when the main goal is to achieve minimal false negative face detection. In this case, false positive face detection is typically an order of magnitude higher. The result of the NPD-based detector is forwarded to the DPM-based detector in order to reduce the number of false positive detections. In this paper, we compare the results obtained by the NPD and DPM methods on the one hand, and the proposed two-stage model on the other. The preliminary experimental results on the Annotated Faces in the Wild (AFW) and the Face Detection Dataset and Benchmark (FDDB) show that the two-stage model significantly reduces false positive detections while simultaneously the number of false negative detections is increased by only a few.

Iterative Automated Foreground Segmentation in Video Sequences Using Graph Cuts

In this paper we propose a method for foreground object segmentation in videos using an improved version of the GrabCut algorithm. Motivated by applications in de-identification, we consider a static camera scenario and take into account common problems with the original algorithm that can result in poor segmentation. Our improvements are as follows: (i) using background subtraction, we build GMM-based segmentation priors; (ii) in building foreground and background GMMs, the contributions of pixels are weighted depending on their distance from the boundary of the object prior; (iii) probabilities of pixels belonging to foreground or background are modified by taking into account the prior pixel classification as well as its estimated confidence; and (iv) the smoothness term of GrabCut is modified by discouraging boundaries further away from the object prior. We perform experiments on CDnet 2014 Pedestrian Dataset and show considerable improvements over a reference implementation of GrabCut.

Automatic recognition of handwritten corrections for multiple-choice exam answer sheets

Automated grading of multiple-choice exams is of great interest in university courses with a large number of students. We consider an existing system in which exams are automatically graded using simple answer sheets that are annotated by the student. A sheet consists of a series of circles representing possible answers. As annotation errors are possible, a student is permitted to alter the annotated answer by annotating the“error” circle and handwriting the letter of the correct answer next to the appropriate row. During the scanning process, if an annotated“error” circle is detected, the system raises an alarm and requires intervention from a human operator to determine which answer to consider valid. We propose rather simple and effecive computer vision algorithm which enables automated reading of a limited set of handwritten answers and minimizes the need for a human intervention in the scanning process. We test our algorithm on a large dataset of real scanned answer sheets, and report encouraging performance rates.