Ming Ye

Estimating piecewise-smooth optical flow with global matching and graduated optimization

This paper presents a new method for estimating piecewise-smooth optical flow. We propose a global optimization formulation with three-frame matching and local variation and develop an efficient technique to minimize the resultant global energy. This technique takes advantage of local gradient, global gradient, and global matching methods and alleviates their limitations. Experiments on various synthetic and real data show that this method achieves highly competitive accuracy.

Algorithm performance contest

This contest involved the running and evaluation of computer vision and pattern recognition techniques on different data sets with known groundwidth. The contest included three areas; binary shape recognition, symbol recognition and image flow estimation. A package was made available for each area. Each package contained either real images with manual groundtruth or programs to generate data sets of ideal as well as noisy images with known groundtruth. They also contained programs to evaluate the results of an algorithm according to the given groundtruth. These evaluation criteria included the generation of confusion matrices, computation of the misdetection and false alarm rates and other performance measures suitable for the problems. The paper summarizes the data generation for each area and experimental results for a total of six participating algorithms.

Optical flow from a least-trimmed squares based adaptive approach

Optical flow estimation can be formulated as two regression stages: derivative estimation and optical flow constraints (OFC) solving. Traditional approaches use least-squares at both stages and are sensitive to assumption violations. To improve estimation accuracy, especially near motion boundaries, we use a least trimmed squares (LTS) estimator to solve the OFC, obtaining a confidence measure for each estimate; and at place with low confidence, we use another LTS estimator to make the derivative estimation robust. This adaptive two-stage robust scheme has significantly higher accuracy than non-robust algorithms and those only using robust methods at the OFC stage. Advantages are illustrated on both synthetic and real data.

Document image matching and annotation lifting

Two images of the same document could differ significantly due to faxing, scanner distortions, or degradation through multigeneration copying. Additionally, one of the images may have extensive annotations not present in the other. We give a method for registering two such images and separating out annotations. We further present an algorithm for detecting and repairing broken strokes in the annotations. Our methods have been tested on a wide variety of documents with reliable results; for the special case of form dropout our results are better than had been obtained previously with special-purpose algorithms.

Learning to Group Text Lines and Regions in Freeform Handwritten Notes

This paper proposes a machine learning approach to grouping problems in ink parsing. Starting from an initial segmentation, hypotheses are generated by perturbing local configurations and processed in a high-confidence-first fashion, where the confidence of each hypothesis is produced by a data-driven AdaBoost decision-tree classifier with a set of intuitive features. This framework has successfully applied to grouping text lines and regions in complex freeform digital ink notes from real TabletPC users. It holds great potential in solving many other grouping problems in the ink parsing and document image analysis domains.

Estimating optical flow using a global matching formulation and graduated optimization

In this paper we consider the problem of optimal optical flow estimation assuming brightness conservation and piecewise smoothness. We propose a formulation based on three-frame matching and global optimization allowing local variation. It is superior to popular gradient-based models and more justifiable than existing global methods. We also develop an efficient technique to minimize the resultant global energy. It takes advantage of local gradient, global gradient and global matching methods and overcomes their limitations. Experiments on various synthetic and real data and comparison with state-of-the-art techniques show that the method achieves discontinuity preserving capability and sub-pixel accuracy.

Two-stage robust optical flow estimation

We formulate optical flow estimation as a two-stage regression problem. Based on characteristics of these two regression models and conclusions on modern regression methods, we choose a least trimmed squares followed by a weighted least squares estimator to solve the optical flow constraint (OFC); and at places where this one-stage robust method fails due to poor derivative quality, we use a least trimmed squares estimator to make the facet model fitting robust. This two-stage robust scheme produces significantly higher accuracy than non-robust algorithms and those only using robust methods at the OFC stage. On the synthetic data, the one-stage robust method has an average error of 7.7% against 24% of Blacks and 19% of the pure LS method; and the two-stage robust method further reduces the error by half near motion boundaries. Advantages are also demonstrated on real data.

Context Aware On-line Diagramming Recognition

This paper presents a context aware, online immediate-mode diagramming recognition and beautification software for hand-sketched diagrams. The system is independent of stroke-order, -number, -direction and is invariant to scaling, translation and rotation. In our stroke-based recognition model, we propose convexity features along with spatial and temporal proximity features to prune the combinatorial search space of possible stroke configurations to form shapes. This reduces the problem of exponential complexity to polynomial one while reducing the error by 24% compared to temporal proximity based criterion. The strokes are then recognized using geometric polygonal features against a neural-net based classifier for 17 classes. The diagramming system is based on stroke-based classifier combination model where an arbitrator makes context aware decisions using suggestions from shape, connector and writing-drawing experts. We achieved an accuracy of 92.7%, 81.4% and 91.5% on the respective experts for a collection of 700,000 online shapes.

Local gradient, global matching, piecewise-smooth optical flow

In this paper we discuss a hybrid technique for piecewise-smooth optical flow estimation. We first pose optical flow estimation as a gradient-based local regression problem and solve it under a high-breakdown robust criterion. Then taking the output from the first step as the initial guess, we recast the problem in a robust matching-based global optimization framework. We have developed novel fast-converging deterministic algorithms for both optimization problems and incorporated a hierarchical scheme to handle large motions. This technique inherits the good subpixel accuracy from the local gradient approach and the insensitivity to local perturbation and derivative quality from the global matching approach, and it overcomes the limitations of both. Significant advantages over competing techniques are demonstrated on various standard synthetic and real image sequences.