Christian Schellewald

Binary partitioning, perceptual grouping, and restoration with semidefinite programming

We introduce a novel optimization method based on semidefinite programming relaxations to the field of computer vision and apply it to the combinatorial problem of minimizing quadratic functionals in binary decision variables subject to linear constraints. The approach is (tuning) parameter-free and computes high-quality combinatorial solutions using interior-point methods (convex programming) and a randomized hyperplane technique. Apart from a symmetry condition, no assumptions (such as metric pairwise interactions) are made with respect to the objective criterion. As a consequence, the approach can be applied to a wide range of problems. Applications to unsupervised partitioning, figure-ground discrimination, and binary restoration are presented along with extensive ground-truth experiments. From the viewpoint of relaxation of the underlying combinatorial problem, we show the superiority of our approach to relaxations based on spectral graph theory and prove performance bounds.

Probabilistic subgraph matching based on convex relaxation

We present a novel approach to the matching of subgraphs for object recognition in computer vision. Feature similarities between object model and scene graph are complemented with a regularization term that measures differences of the relational structure. For the resulting quadratic integer program, a mathematically tight relaxation is derived by exploiting the degrees of freedom of the embedding space of positive semidefinite matrices. We show that the global minimum of the relaxed convex problem can be interpreted as probability distribution over the original space of matching matrices, providing a basis for efficiently sampling all close-to-optimal combinatorial matchings within the original solution space. As a result, the approach can even handle completely ambiguous situations, despite uniqueness of the relaxed convex problem. Exhaustive numerical experiments demonstrate the promising performance of the approach which – up to a single inevitable regularization parameter that weights feature similarity against structural similarity – is free of any further tuning parameters.

Evaluation of Convex Optimization Techniques for the Weighted Graph-Matching Problem in Computer Vision

We present a novel approach to the weighted graph-matching problem in computer vision, based on a convex relaxation of the underlying combinatorial optimization problem. The approach always computes a lower bound of the objective function, which is a favorable property in the context of exact search algorithms. Furthermore, no tuning parameters have to be selected by the user, due to the convexity of the relaxed problem formulation. For comparison, we implemented a recently published deterministic annealing approach and conducted numerous experiments for both established benchmark experiments from combinatorial mathematics, and for random ground-truth experiments using computer-generated graphs. Our results show similar performance for both approaches. In contrast to the convex approach, however, four parameters have to be determined by hand for the annealing algorithm to become competitive.

Subgraph Matching with Semidefinite Programming

Abstract We present a convex programming approach to the problem of matching subgraphs which represent object views against larger graphs which represent scenes. Starting from a linear programming formulation for computing optimal matchings in bipartite graphs, we extend the linear objective function in order to take into account the relational constraints given by both graphs. The resulting combinatorial optimization problem is approximately solved by a semidefinite program. Preliminary results are promising with respect to view-based object recognition subject to relational constraints.

Evaluation of a convex relaxation to a quadratic assignment matching approach for relational object views

We introduce a convex relaxation approach for the quadratic assignment problem to the field of computer vision. Due to convexity, a favourable property of this approach is the absence of any tuning parameters and the computation of high-quality combinatorial solutions by solving a mathematically simple optimization problem. Furthermore, the relaxation step always computes a tight lower bound of the objective function and thus can additionally be used as an efficient subroutine of an exact search algorithm. We report the results of both established benchmark experiments from combinatorial mathematics and random ground-truth experiments using computer-generated graphs. For comparison, a deterministic annealing approach is investigated as well. Both approaches show similarly good performance. In contrast to the convex approach, however, the annealing approach yields no problem relaxation, and four parameters have to be tuned by hand for the annealing algorithm to become competitive.

Diffusion-Snakes Using Statistical Shape Knowledge

We present a novel extension of the Mumford–Shah functional that allows to incorporate statistical shape knowledge at the computational level of image segmentation. Our approach exhibits various favorable properties: non-local convergence, robustness against noise, and the ability to take into consideration both shape evidence in given image data and knowledge about learned shapes. In particular, the latter property distinguishes our approach from previous work on contour–evolution based image segmentation. Experimental results confirm these properties.

Convex Relaxations for Binary Image Partitioning and Perceptual Grouping

We consider approaches to computer vision problems which require the minimization of a global energy functional over binary variables and take into account both local similarity and spatial context. The combinatorial nature of such problems has lead to the design of various approximation algorithms in the past which often involve tuning parameters and tend to get trapped in local minima. In this context, we present a novel approach to the field of computer vision that amounts to solving a convex relaxation of the original problem without introducing any additional parameters. Numerical ground truth experiments reveal a relative error of the convex minimizer with respect to the global optimum of below 2% on the average. We apply our approach by discussing two specific problem instances related to image partitioning and perceptual grouping. Numerical experiments illustrate the quality of the approach which, in the partitioning case, compares favorably with established approaches like the ICM-algorithm.

Image Labeling and Grouping by Minimizing Linear Functionals over Cones

We consider energy minimization problems related to image labeling, partitioning, and grouping, which typically show up at mid-level stages of computer vision systems. A common feature of these problems is their intrinsic combinatorial complexity from an optimization point-of-view. Rather than trying to compute the global minimum - a goal we consider as elusive in these cases - we wish to design optimization approaches which exhibit two relevant properties: First, in each application a solution with guaranteed degree of suboptimality can be computed. Secondly, the computations are based on clearly defined algorithms which do not comprise any (hidden) tuning parameters. In this paper, we focus on the second property and introduce a novel and general optimization technique to the field of computer vision which amounts to compute a suboptimal solution by just solving a convex optimization problem. As representative examples, we consider two binary quadratic energy functionals related to image labeling and perceptual grouping. Both problems can be considered as instances of a general quadratic functional in binary variables, which is embedded into a higher-dimensional space such that suboptimal solutions can be computed as minima of linear functionals over cones in that space (semidefinite programs). Extensive numerical results reveal that, on the average, suboptimal solutions can be computed which yield a gap below 5% with respect to the global optimum in case where this is known.

Unsupervised Image Partitioning with Semidefinite Programming

We apply a novel optimization technique, semidefinite programming, to the unsupervised partitioning of images. Representing images by graphs which encode pairwise (dis)similarities of local image features, a partition of the image into coherent groups is computed by determining optimal balanced graph cuts. Unlike recent work in the literature, we do not make any assumption concerning the objective criterion like metric pairwise interactions, for example. Moreover, no tuning parameter is necessary to compute the solution. We prove that, from the optimization point of view, our approach cannot perform worse than spectral relaxation approaches which, conversely, may completely fail for the unsupervised choice of the eigenvector threshold.

A Convex Relaxation Bound for Subgraph Isomorphism

In this work a convex relaxation of a subgraph isomorphism problem is proposed, which leads to a new lower bound that can provide a proof that a subgraph isomorphism between two graphs can not be found. The bound is based on a semidefinite programming relaxation of a combinatorial optimisation formulation for subgraph isomorphism and is explained in detail. We consider subgraph isomorphism problem instances of simple graphs which means that only the structural information of the two graphs is exploited and other information that might be available (e.g., node positions) is ignored. The bound is based on the fact that a subgraph isomorphism always leads to zero as lowest possible optimal objective value in the combinatorial problem formulation. Therefore, for problem instances with a lower bound that is larger than zero this represents a proof that a subgraph isomorphism can not exist. But note that conversely, a negative lower bound does not imply that a subgraph isomorphism must be present and only indicates that a subgraph isomorphism can not be excluded. In addition, the relation of our approach and the reformulation of the largest common subgraph problem into a maximum clique problem is discussed.