Ohad Ben-Shahar

Geometrical computations explain projection patterns of long-range horizontal connections in visual cortex

Neurons in primary visual cortex respond selectively to oriented stimuli such as edges and lines. The long-range horizontal connections between them are thought to facilitate contour integration. While many physiological and psychophysical findings suggest that collinear or association field models of good continuation dictate particular projection patterns of horizontal connections to guide this integration process, significant evidence of interactions inconsistent with these hypotheses is accumulating. We first show that natural random variations around the collinear and association field models cannot account for these inconsistencies, a fact that motivates the search for more principled explanations. We then develop a model of long-range projection fields that formalizes good continuation based on differential geometry. The analysis implicates curvature(s) in a fundamental way, and the resulting model explains both consistent data and apparent outliers. It quantitatively predicts the (typically ignored) spread in projection distribution, its nonmonotonic variance, and the differences found among individual neurons. Surprisingly, and for the first time, this model also indicates that texture (and shading) continuation can serve as alternative and complementary functional explanations to contour integration. Because current anatomical data support both (curve and texture) integration models equally and because both are important computationally, new testable predictions are derived to allow their differentiation and identification.

Computer vision for fruit harvesting robots state of the art and challenges ahead

Despite extensive research conducted in machine vision for harvesting robots, practical success in this field of agrobotics is still limited. This article presents a comprehensive review of classical and state-of-the-art machine vision solutions employed in such systems, with special emphasis on the visual cues and machine vision algorithms used. We discuss the advantages and limitations of each approach and we examine these capacities in light of the challenges ahead. We conclude with suggested directions from the general computer vision literature which could assist our research community meet these challenges and bring us closer to the goal of practical selective fruit harvesting robots.

Stochastic Emergence of Repeating Cortical Motifs in Spontaneous Membrane Potential Fluctuations In Vivo

It was recently discovered that subthreshold membrane potential fluctuations of cortical neurons can precisely repeat during spontaneous activity, seconds to minutes apart, both in brain slices and in anesthetized animals. These repeats, also called cortical motifs, were suggested to reflect a replay of sequential neuronal firing patterns. We searched for motifs in spontaneous activity, recorded from the rat barrel cortex and from the cat striate cortex of anesthetized animals, and found numerous repeating patterns of high similarity and repetition rates. To test their significance, various statistics were compared between physiological data and three different types of stochastic surrogate data that preserve dynamical characteristics of the recorded data. We found no evidence for the existence of deterministically generated cortical motifs. Rather, the stochastic properties of cortical motifs suggest that they appear by chance, as a result of the constraints imposed by the coarse dynamics of subthreshold ongoing activity.

Toward a Theory of Shape from Specular Flow

The image of a curved, specular (mirror-like) surface is a distorted reflection of the environment. The goal of our work is to develop a framework for recovering general shape from such distortions when the environment is neither calibrated nor known. To achieve this goal we consider far-field illumination, where the object-environment distance is relatively large, and we examine the dense specular flow that is induced on the image plane through relative object-environment motion. We show that under these very practical conditions the observed specular flow can be related to surface shape through a pair of coupled nonlinear partial differential equations. Importantly, this relationship depends only on the environments relative motion and not its content. We examine the qualitative properties of these equations, present analytic methods for recovery of the shape in several special cases, and empirically validate our results using captured data. We also discuss the relevance to both computer vision and human perception.

Practical pushing planning for rearrangement tasks

Rearrangement of objects by pushing is a basic manipulation task. The authors (1995) presented a resolution-complete algorithm that plans optimal pushing manipulations for rearrangement tasks but operates in high time and space complexity. In this paper the authors address the issue of practical planning for the same kind of problems. Rather than using a classical heuristic method, the authors propose an alternative approach. The authors present a hierarchical classification of the pushing problems domain into several classes, each characterized by properties of the plans that can solve it. Such a classification allows the authors to consider each class individually, analyze and exploit properties of each class, and suggest individual planning methods. Algorithms for two of the defined classes are presented. Both algorithms were tested in a simulated environment, with up to 32 movable objects and 66 combined DOF. Some of these simulations are presented here.

Sensitivity to curvatures in orientation-based texture segmentation

Texture segregation has long been attributed to changes in the distribution of elementary features across the visual field [Nature 290 (12) (1981) 91; Biol. Cybernet. 54 (1986) 245]. The study of orientation, a conspicuous feature, has led to models of orientation-based texture segmentation (OBTS) that depend on the magnitude of one or two orientation gradients [Vis. Res. 31 (4) (1991) 679; Vis. Res. 31 (6) (1991) 1073] and influenced further by the relative configuration between the orientation textons and the global orientation edge [Percept. Psychophys. 52 (4) (1992) 255; Vis. Res. 35 (20) (1995) 2863]. Here we show that these models are at best partial and that the notion of orientation gradient has been incompletely used in the study of OBTS. To do so, we first study the behavior of orientation in orientation-defined texture patches. Geometrical analysis identifies two texture curvatures and reveals the incompleteness of previous stimuli. Psychophysical experimentation then demonstrates that segmentation is strongly affected by discontinuities in these curvatures. Importantly, we show that this sensitivity to curvature is independent of the orientation gradients and inconsistent with the simple configural considerations proposed in the past.

A Tangent Bundle Theory for Visual Curve Completion

Visual curve completion is a fundamental perceptual mechanism that completes the missing parts (e.g., due to occlusion) between observed contour fragments. Previous research into the shape of completed curves has generally followed an “axiomatic” approach, where desired perceptual/geometrical properties are first defined as axioms, followed by mathematical investigation into curves that satisfy them. However, determining psychophysically such desired properties is difficult and researchers still debate what they should be in the first place. Instead, here we exploit the observation that curve completion is an early visual process to formalize the problem in the unit tangent bundle R2 × S1, which abstracts the primary visual cortex (V1) and facilitates exploration of basic principles from which perceptual properties are later derived rather than imposed. Exploring here the elementary principle of least action in V1, we show how the problem becomes one of finding minimum-length admissible curves in R2 × S1. We formalize the problem in variational terms, we analyze it theoretically, and we formulate practical algorithms for the reconstruction of these completed curves. We then explore their induced visual properties vis-à-vis popular perceptual axioms and show how our theory predicts many perceptual properties reported in the corresponding perceptual literature. Finally, we demonstrate a variety of curve completions and report comparisons to psychophysical data and other completion models.

Shape from Specular Flow

An image of a specular (mirror-like) object is nothing but a distorted reflection of its environment. When the environment is unknown, reconstructing shape from such an image can be very difficult. This reconstruction task can be made tractable when, instead of a single image, one observes relative motion between the specular object and its environment, and therefore, a motion field-or specular flow-in the image plane. In this paper, we study the shape from specular flow problem and show that observable specular flow is directly related to surface shape through a nonlinear partial differential equation. This equation has the key property of depending only on the relative motion of the environment while being independent of its content. We take first steps toward understanding and exploiting this PDE, and we examine its qualitative properties in relation to shape geometry. We analyze several cases in which the surface shape can be recovered in closed form, and we show that, under certain conditions, specular shape can be reconstructed when both the relative motion and the content of the environment are unknown. We discuss numerical issues related to the proposed reconstruction algorithms, and we validate our findings using both real and synthetic data.

A linear formulation of shape from specular flow

When a curved mirror-like surface moves relative to its environment, it induces a motion field—or specular flow— on the image plane that observes it. This specular flow is related to the mirrors shape through a non-linear partial differential equation, and there is interest in understanding when and how this equation can be solved for surface shape. Existing analyses of this ‘shape from specular flow equation’ have focused on closed-form solutions, and while they have yielded insight, their critical reliance on externally-provided initial conditions and/or specific motions makes them difficult to apply in practice. This paper resolves these issues. We show that a suitable reparameterization leads to a linear formulation of the shape from specular flow equation. This formulation radically simplifies the reconstruction process and allows, for example, both motion and shape to be recovered from as few as two specular flows even when no externally-provided initial conditions are available. Our analysis moves us closer to a practical method for recovering shape from specular flow that operates under arbitrary, unknown motions in unknown illumination environments and does not require additional shape information from other sources.

Dense specular shape from multiple specular flows

The inference of specular (mirror-like) shape is a particularly difficult problem because an image of a specular object is nothing but a distortion of the surrounding environment. Consequently, when the environment is unknown, such an image would seem to convey little information about the shape itself. It has recently been suggested (Adato et al., ICCV 2007) that observations of relative motion between a specular object and its environment can dramatically simplify the inference problem and allow one to recover shape without explicit knowledge of the environment content. However, this approach requires solving a non-linear PDE (the dasiashape from specular flow equationpsila) and analytic solutions are only known to exist for very constrained motions. In this paper, we consider the recovery of shape from specular flow under general motions. We show that while the dasiashape from specular flowpsila PDE for a single motion is non-linear, we can combine observations of multiple specular flows from distinct relative motions to yield a linear set of equations. We derive necessary conditions for this procedure, discuss several numerical issues with their solution, and validate our results quantitatively using image data.