Rami Ben-Ari

Variational Stereo Vision with Sharp Discontinuities and Occlusion Handling

This paper addresses the problem of correspondence establishment in binocular stereo vision. We suggest a novel variational approach that considers both the discontinuities and occlusions. It deals with color images as well as gray levels. The proposed method divides the image domain into the visible and occluded regions where each region is handled differently. The depth discontinuities in the visible domain are preserved by use of the total variation term in conjunction with the Mumford-Shah framework. In addition to the dense disparity and the occlusion maps, our method also provides a discontinuity function revealing the location of the boundaries in the disparity map. We evaluate our method on data sets from Middlebury site showing superior performance in comparison to the state of the art variational technique.

Stereo Matching with Mumford-Shah Regularization and Occlusion Handling

This paper addresses the problem of correspondence establishment in binocular stereo vision. We suggest a novel spatially continuous approach for stereo matching based on the variational framework. The proposed method suggests a unique regularization term based on Mumford-Shah functional for discontinuity preserving, combined with a new energy functional for occlusion handling. The evaluation process is based on concurrent minimization of two coupled energy functionals, one for domain segmentation (occluded versus visible) and the other for disparity evaluation. In addition to a dense disparity map, our method also provides an estimation for the half-occlusion domain and a discontinuity function allocating the disparity/depth boundaries. Two new constraints are introduced improving the revealed discontinuity map. The experimental tests include a wide range of real data sets from the Middlebury stereo database. The results demonstrate the capability of our method in calculating an accurate disparity function with sharp discontinuities and occlusion map recovery. Significant improvements are shown compared to a recently published variational stereo approach. A comparison on the Middlebury stereo benchmark with subpixel accuracies shows that our method is currently among the top-ranked stereo matching algorithms.

A Region Based Convolutional Network for Tumor Detection and Classification in Breast Mammography

This paper addresses the problem of detection and classification of tumors in breast mammograms. We introduce a novel system that integrates several modules including a breast segmentation module and a fibroglandular tissue segmentation module into a modified cascaded region-based convolutional network. The method is evaluated on a large multi-center clinical dataset and compared to ground truth annotated by expert radiologists. Preliminary experimental results show the high accuracy and efficiency obtained by the suggested network structure. As the volume and complexity of data in healthcare continues to accelerate generalizing such an approach may have a profound impact on patient care in many applications.

A Unified Approach for Registration and Depth in Depth from Defocus

Depth from Defocus (DFD) suggests a simple optical set-up to recover the shape of a scene through imaging with shallow depth of field. Although numerous methods have been proposed for DFD, less attention has been paid to the particular problem of alignment between the captured images. The inherent shift-variant defocus often prevents standard registration techniques from achieving the accuracy needed for successful shape reconstruction. In this paper, we address the DFD and registration problem in a unified framework, exploiting their mutual relation to reach a better solution for both cues. We draw a formal connection between registration and defocus blur, find its limitations and reveal the weakness of the standard isolated approaches of registration and depth estimation. The solution is approached by energy minimization. The efficiency of the associated numerical scheme is justified by showing its equivalence to the celebrated Newton-Raphson method and proof of convergence of the emerged linear system. The computationally intensive approach of DFD, newly combined with simultaneous registration, is handled by GPU computing. Experimental results demonstrate the high sensitivity of the recovered shapes to slight errors in registration and validate the superior performance of the suggested approach over two, separately applying registration and DFD alternatives.

Variational Depth from Defocus in real-time

With emerging of next generation of digital cameras offering a 3D reconstruction of a viewed scene, Depth from Defocus (DFD) presents an attractive option. In this approach the depth profile of the scene is recovered from two views captured in different focus setting. The DFD is well known as a computationally-intensive method due to the shift-variant filtering involved with its estimation. In this paper we present a parallel GPGPU implementation of DFD based on the variational framework, enabling computation up to 15 frames/sec for a SVGA sequence. This constitutes the first GPU application and the fastest implementation known for passive DFD. The speed-up is obtained by using the novel Fast Explicit Diffusion approach and the fine grain data parallelism in an explicit scheme. We evaluate our method on publicly available real data and compare its results to a recently published PDE based method. The proposed method outperforms previous DFD techniques in terms of accuracy/runtime, suggesting the DFD as an alternative for 3D reconstruction in real-time.

A Geometric Framework and a New Criterion in Optical Flow Modeling

We evaluate the dense optical flow between two frames via variational approach. In this paper, a new framework for deriving the regularization term is introduced giving a geometric insight into the action of a smoothing term. The framework is based on the Beltrami paradigm in image denoising. It includes a general formulation that unifies several previous methods. Using the proposed framework we also derive two novel anisotropic regularizers incorporating a new criterion that requires co-linearity between the gradients of optical flow components and possibly the intensity gradient. We call this criterion “alignment” and reveal its existence also in the celebrated Nagel and Enkelmann’s formulation. It is shown that the physical model of rotational motion of a rigid body, pure divergent/convergent flow and irrotational fluid flow, satisfy the alignment criterion in the flow field. Experimental tests in comparison to a recently published method show the capability of the new criterion in improving the optical flow estimations.

Geodesic Active Contours with Combined Shape and Appearance Priors

We present a new object segmentation method that is based on geodesic active contours with combined shape and appearance priors. It is known that using shape priors can significantly improve object segmentation in cluttered scenes and occlusions. Within this context, we add a new prior, based on the appearance of the object, (i.e., an image) to be segmented. This method enables the appearance pattern to be incorporated within the geodesic active contour framework with shape priors, seeking for the object whose boundaries lie on high image gradients and that best fits the shape and appearance of a reference model. The output contour results from minimizing an energy functional built of these three main terms. We show that appearance is a powerful term that distinguishes between objects with similar shapes and capable of successfully segment an object in a very cluttered environment where standard active contours (even those with shape priors) tend to fail.

A General Framework and New Alignment Criterion for Dense Optical Flow

The problem of dense optical flow computation is addressed from a variational viewpoint. A new geometric framework is introduced. It unifies previous art and yields new efficient methods. Along with the framework a new alignment criterion suggests itself. It is shown that the alignment between the gradients of the optical flow components and between the latter and the intensity gradients is an important measure of the flow’s quality. Adding this criterion as a requirement in the optimization process improves the resulting flow. This is demonstrated in synthetic and real sequences.

A Geometric Approach for Regularization of the Data Term in Stereo-Vision

Every stereovision application must cope with the correspondence problem. The space of the matching variables, often consisting of spatial coordinates, intensity and disparity, is commonly referred as the data term (space). Since the data is often noisy a-priori, preference is required to result a smooth disparity (or piecewise smooth). To this end, each local method (e.g. window correlation techniques) performs a regularization of the data space. In this paper we propose a geometric framework for anisotropic regularization of the data space seeking to preserve the discontinuities in this space when filtering out the noise. On the other hand, the global methods consider a non-regularized data term with a smoothing constraint imposed directly on the disparity. This paper also proposes a new idea where the data space is regularized in a global method prior to the disparity evaluation. The idea is implemented on the state of the art variational method. Experimental results on the Middlebury real images demonstrate the advantages of the proposed approach.

Hybrid Mass Detection in Breast MRI Combining Unsupervised Saliency Analysis and Deep Learning

To interpret a breast MRI study, a radiologist has to examine over 1000 images, and integrate spatial and temporal information from multiple sequences. The automated detection and classification of suspicious lesions can help reduce the workload and improve accuracy. We describe a hybrid mass-detection algorithm that combines unsupervised candidate detection with deep learning-based classification. The detection algorithm first identifies image-salient regions, as well as regions that are cross-salient with respect to the contralateral breast image. We then use a convolutional neural network (CNN) to classify the detected candidates into true-positive and false-positive masses. The network uses a novel multi-channel image representation; this representation encompasses information from the anatomical and kinetic image features, as well as saliency maps. We evaluated our algorithm on a dataset of MRI studies from 171 patients, with 1957 annotated slices of malignant (59%) and benign (41%) masses. Unsupervised saliency-based detection provided a sensitivity of 0.96 with 9.7 false-positive detections per slice. Combined with CNN classification, the number of false positive detections dropped to 0.7 per slice, with 0.85 sensitivity. The multi-channel representation achieved higher classification performance compared to single-channel images. The combination of domain-specific unsupervised methods and general-purpose supervised learning offers advantages for medical imaging applications, and may improve the ability of automated algorithms to assist radiologists.