Csaba Domokos

FuseNet: Incorporating Depth into Semantic Segmentation via Fusion-Based CNN Architecture

In this paper we address the problem of semantic labeling of indoor scenes on RGB-D data. With the availability of RGB-D cameras, it is expected that additional depth measurement will improve the accuracy. Here we investigate a solution how to incorporate complementary depth information into a semantic segmentation framework by making use of convolutional neural networks (CNNs). Recently encoder-decoder type fully convolutional CNN architectures have achieved a great success in the field of semantic segmentation. Motivated by this observation we propose an encoder-decoder type network, where the encoder part is composed of two branches of networks that simultaneously extract features from RGB and depth images and fuse depth features into the RGB feature maps as the network goes deeper. Comprehensive experimental evaluations demonstrate that the proposed fusion-based architecture achieves competitive results with the state-of-the-art methods on the challenging SUN RGB-D benchmark obtaining 76.27% global accuracy, 48.30% average class accuracy and 37.29% average intersection-over-union score.

Nonlinear Shape Registration without Correspondences

In this paper, we propose a novel framework to estimate the parameters of a diffeomorphism that aligns a known shape and its distorted observation. Classical registration methods first establish correspondences between the shapes and then compute the transformation parameters from these landmarks. Herein, we trace back the problem to the solution of a system of nonlinear equations which directly gives the parameters of the aligning transformation. The proposed method provides a generic framework to recover any diffeomorphic deformation without established correspondences. It is easy to implement, not sensitive to the strength of the deformation, and robust against segmentation errors. The method has been applied to several commonly used transformation models. The performance of the proposed framework has been demonstrated on large synthetic data sets as well as in the context of various applications.

Parametric estimation of affine deformations of planar shapes

We consider the estimation of affine transformations aligning a known 2D shape and its distorted observation. The classical way to solve this registration problem is to find correspondences between the shapes and then compute the transformation parameters from these landmarks. Here we propose a novel approach where the exact transformation is obtained as the solution of a polynomial system of equations. The method has been tested on synthetic as well as on real images and its robustness in the presence of segmentation errors and additive geometric noise has also been demonstrated. We have successfully applied the method for the registration of hip prosthesis X-ray images. The advantage of the proposed solution is that it is fast, easy to implement, has linear time complexity, works without established correspondences and provides an exact solution regardless of the magnitude of transformation.

Recovering planar homographies between 2D shapes

Images taken from different views of a planar object are related by planar homography. Recovering the parameters of such transformations is a fundamental problem in computer vision with various applications. This paper proposes a novel method to estimate the parameters of a homography that aligns two binary images. It is obtained by solving a system of nonlinear equations generated by integrating linearly independent functions over the domains determined by the shapes. The advantage of the proposed solution is that it is easy to implement, less sensitive to the strength of the deformation, works without established correspondences and robust against segmentation errors. The method has been tested on synthetic as well as on real images and its efficiency has been demonstrated in the context of two different applications: alignment of hip prosthesis X-ray images and matching of traffic signs.

Parametric estimation of affine deformations of binary images

We consider the problem of planar object registration on binary images where the aligning transformation is restricted to the group of affine transformations. Previous approaches usually require established correspondences or the solution of nonlinear optimization problems. Herein we show that it is possible to formulate the problem as the solution of a system of up to third order polynomial equations. These equations are constructed in a simple way using some basic geometric information of binary images. It does not need established correspondences nor the solution of complex optimization problems. The resulting algorithm is fast and provides a direct solution regardless of the magnitude of transformation.

Recovering Affine Deformations of Fuzzy Shapes

Fuzzy sets and fuzzy techniques are attracting increasing attention nowadays in the field of image processing and analysis. It has been shown that the information preserved by using fuzzy representation based on area coverage may be successfully utilized to improve precision and accuracy of several shape descriptors; geometric moments of a shape are among them. We propose to extend an existing binary shape matching method to take advantage of fuzzy object representation. The result of a synthetic test show that fuzzy representation yields smaller registration errors in average. A segmentation method is also presented to generate fuzzy segmentations of real images. The applicability of the proposed methods is demonstrated on real X-ray images of hip replacement implants.

Affine puzzle: realigning deformed object fragments without correspondences

This paper is addressing the problem of realigning broken objects without correspondences. We consider linear transformations between the object fragments and present the method through 2D and 3D affine transformations. The basic idea is to construct and solve a polynomial system of equations which provides the unknown parameters of the alignment. We have quantitatively evaluated the proposed algorithm on a large synthetic dataset containing 2D and 3D images. The results show that the method performs well and robust against segmentation errors. We also present experiments on 2D real images as well as on volumetric medical images applied to surgical planning.

Nonlinear registration of binary shapes

A novel approach is proposed to estimate the parameters of a diffeomorphism that aligns two binary images. Classical approaches usually define a cost function based on a similarity metric and then find the solution via optimization. Herein, we trace back the problem to the solution of a system of nonlinear equations which directly provides the parameters of the aligning transformation. The proposed method works without any time consuming optimization step or established correspondences. The advantage of our algorithm is that it is easy to implement, less sensitive to the strength of the deformation, and robust against segmentation errors. The efficiency of the proposed approach has been demonstrated on a large synthetic dataset as well as in the context of an industrial application.

Binary Image Registration Using Covariant Gaussian Densities

We consider the estimation of 2D affine transformations aligning a known binary shape and its distorted observation. The classical way to solve this registration problem is to find correspondences between the two images and then compute the transformation parameters from these landmarks. In this paper, we propose a novel approach where the exact transformation is obtained as a least-squares solution of a linear system. The basic idea is to fit a Gaussian density to the shapes which preserves the effect of the unknown transformation. It can also be regarded as a consistent coloring of the shapes yielding two rich functions defined over the two shapes to be matched. The advantage of the proposed solution is that it is fast, easy to implement, works without established correspondences and provides a unique and exact solution regardless of the magnitude of transformation.

Affine Shape Alignment Using Covariant Gaussian Densities: A Direct Solution

We propose a novel approach for the estimation of 2D affine transformations aligning a planar shape and its distorted observation. The exact transformation is obtained as a least-squares solution of a linear system of equations constructed by fitting Gaussian densities to the shapes which preserve the effect of the unknown transformation. In the case of compound shapes, we also propose a robust and efficient numerical scheme achieving near real-time performance. The method has been tested on synthetic as well as on real images. Its robustness in the case of segmentation errors, missing data, and modelling error has also been demonstrated. The proposed method does not require point correspondences nor the solution of complex optimization problems, has linear time complexity and provides an exact solution regardless of the magnitude of deformation.