Anton Konushin

Fast Automatic Single-View 3-d Reconstruction of Urban Scenes

We consider the problem of estimating 3-d structure from a single still image of an outdoor urban scene. Our goal is to efficiently create 3-d models which are visually pleasant. We chose an appropriate 3-d model structure and formulate the task of 3-d reconstruction as model fitting problem. Our 3-d models are composed of a number of vertical walls and a ground plane, where ground-vertical boundary is a continuous polyline. We achieve computational efficiency by special preprocessing together with stepwise search of 3-d model parameters dividing the problem into two smaller sub-problems on chain graphs. The use of Conditional Random Field models for both problems allows to various cues. We infer orientation of vertical walls of 3-d model vanishing points.

Depth image-based representation and compression for static and animated 3-D objects

This paper describes a new family of three-dimensional (3-D) representations for computer graphics and animation, called depth image-based representations (DIBR), which have been adopted into MPEG-4 Part16: Animation Framework eXtension (AFX). Idea of the approach is to build a compact and photorealistic representation of a 3-D object or scene without using polygonal mesh. Instead, images accompanied by depth values for each pixel are used. This type of representation allows us to build and render novel views of objects and scene with an interactive rate. There are many different methods for the image-based rendering with depths, and the DIBR format is designed to efficiently represent the information necessary for such methods. The main formats of the DIBR family are SimpleTexture (an image together with depth array), PointTexture (an image with multiple pixels along each line of sight), and OctreeImage (octree-like data structure together with a set of images containing viewport parameters). In order to store and transmit the DIBR object, we develop a compression algorithm and bitstream format for OctreeImage representation.

Depth image-based representations for static and animated 3D objects

We describe a novel depth image-based representation (DIBR) that has been adopted into the MPEG-4 animation framework extension (AFX). The idea of this approach is to build a compact representation of a 3D object or scene without storing the geometry information in traditional polygonal form. The main formats of the DIBR family are simple texture (an image together with depth array), point texture (a view of a scene from a single input camera but with multiple pixels along each line of sight), and octree image (octree data structure together with a set of images and their viewport parameters). The designed node specifications and rendering algorithms are addressed. The experimental results show the efficacy and fidelity of the proposed approach.

Evaluation of Traffic Sign Recognition Methods Trained on Synthetically Generated Data

Most of todays machine learning techniques requires large manually labeled data. This problem can be solved by using synthetic images. Our main contribution is to evaluate methods of traffic sign recognition trained on synthetically generated data and show that results are comparable with results of classifiers trained on real dataset. To get a representative synthetic dataset we model different sign image variations such as intra-class variability, imprecise localization, blur, lighting, and viewpoint changes. We also present a new method for traffic sign segmentation, based on a nearest neighbor search in the large set of synthetically generated samples, which improves current traffic sign recognition algorithms.

An improvement on an MCMC-based video tracking algorithm

This paper presents an approach to fully automatic people tracking in surveillance video recorded by stable camera. We propose an improvement on Benfold et al. tracking-by-detection algorithm [1]. We extend the basic algorithm through filtering of person detector results and the scene entrance/exit positions construction. Moreover, the paper presents a modified method for tracklet position estimation. We compare several tracklet construction algorithms such as “Flock of Features” and normalized cross correlation. Our experiments reveal that all the proposed modifications improve both robustness and precision of tracks compared to the basic algorithm.

Simultaneous classification of several features of a person's appearance using a deep convolutional neural network

In this paper, we describe a model of a convolutional neural network for automatic simultaneous extraction of several features of the person’s appearance from an image. The proposed model has the form of a deep convolutional neural network with common initial layers and several probabilistic outputs. This neural network has the high accuracy of a convolutional network and can simultaneously extract a number of features of the appearance in the time that is required to extract one feature of the appearance. The neural network is tested using photographs from the LWF database. As features of the appearance of a person, we use the person’s sex, a mustache, and a beard. The accuracy of identifying each feature is no less than 91.5%, which is one of the best results for the LWF database. This model of a neural network can be used for simultaneous identification of a greater number of features of the person’s appearance without a significant increase in operating time.

The matching of infrared markers for tracking objects using stereo pairs

This paper presents a new algorithm for obtaining inter-frame and inter-view (inter-camera) correspondences to solve the problem of tracking an object labeled with infrared markers using a stereo pair taken simultaneously in the infrared region. In practice it is often necessary to track an object when it is impossible to have contact with it, for example, the tracking of facial movements using the motion capture technique (Motion Capture, [9, 10]) to create realistic animation or the tracking of object movements when interacting with the augmented reality. In such cases contactless object tracking methods are used. In the classic version of this problem, two or more cameras are used to capture the object of interest. In order to restore three-dimensional coordinates of object points, it is necessary to triangulate the received projections of the points. In the case of the visible range, the problem of finding and matching points on the object can be solved using interest point descriptors [2]. However, there are situations in which it is impossible to use the visible range data, for example, a uniformly colored or regularly textured object, which makes it senseless to use interest point descriptors. Thus, the use of interest point descriptors significantly limits the scope of application of algorithms because of the imposition of severe restrictions on the class of tracked objects, objects should have uneven texture. In turn, the motion capture method implies the tracking of an object of a predetermined shape, when it is not always possible to establish its shape. In this work, an alternative approach is proposed, i.e., the use of infrared markers and cameras that capture frames in the infrared range which makes the task of finding critical points irrelevant. On the other hand, infrared markers on cameras that operate in the infrared range are indistinguishable from each other. Therefore, there is the problem of finding correspondences which in the case of interest point descriptors is solved by the very nature of the descriptors. In this paper, we describe algorithms that make it possible to restore point correspondences by a sequence of stereo pair images (Fig. 1) and its calibrations. In this case, epipolar constraints and a voting scheme based on the greedy algorithm are used.

Alpha-Flow for video matting

This work addresses the problem of video matting, that is extracting the opacity-layer of a foreground object from a video sequence. We introduce the notion of alpha-flow which corresponds to the flow in the opacity layer. The idea is derived from the process of rotoscoping, where a user-supplied object mask is smoothly interpolated between keyframes while preserving its correspondence with the underlying image. Our key contribution is an algorithm which infers both the opacity masks and the alpha-flow in an efficient and unified manner. We embed our algorithm in an interactive video matting system where the first and last frame of a sequence are given as keyframes, and additional user strokes may be provided in intermediate frames. We show high quality results on various challenging sequences, and give a detailed comparison to competing techniques.

Video tracking and behaviour segmentation of laboratory rodents

In this paper a system for laboratory rodent video tracking and behavior segmentation is proposed. A new real-time mouse pose estimation method is proposed based on semi-automatically generated animal shape model. Behavior segmentation into separate behavior acts is considered as a signal segmentation problem using hidden Markov models (HMM). Conventional first order HMM supposes a geometric prior distribution on segment’s length, which is inadequate for behavior segmentation. We propose a modification of conventional first order HMM that allows any prior distribution on segment’s length. Experiments show that the developed approach can lead to more adequate results comparing to conventional HMM.

Markov chain Monte Carlo based video tracking algorithm

The paper considers a problem of multiple person tracking. We present the algorithm to automatic people tracking on surveillance videos recorded by static cameras. Proposed algorithm is an extension of approach based on tracking-by-detection of people heads and data association using Markov chain Monte Carlo (MCMC). Short track fragments (tracklets) are built by local tracking of people heads. Tracklet postprocessing and accurate results interpolation were shown to reduce number of false positives. We use position deviations of tracklets and revised entry/exit points factor to separate pedestrians from false positives. The paper presents a new method to estimate body position, that increases precision of tracker. Finally, we switched HOG-based detector to cascade one. Our evaluation shows proposed modifications significantly increase tracking quality.