Ethan Fetaya

Atoms of recognition in human and computer vision

Significance Discovering the visual features and representations used by the brain to recognize objects is a central problem in the study of vision. Recent successes in computational models of visual recognition naturally raise the question: Do computer systems and the human brain use similar or different computations? We show by combining a novel method (minimal images) and simulations that the human recognition system uses features and learning processes, which are critical for recognition, but are not used by current models. The study uses a “phase transition” phenomenon in minimal images, in which minor changes to the image have a drastic effect on its recognition. The results show fundamental limitations of current approaches and suggest directions to produce more realistic and better-performing models. Discovering the visual features and representations used by the brain to recognize objects is a central problem in the study of vision. Recently, neural network models of visual object recognition, including biological and deep network models, have shown remarkable progress and have begun to rival human performance in some challenging tasks. These models are trained on image examples and learn to extract features and representations and to use them for categorization. It remains unclear, however, whether the representations and learning processes discovered by current models are similar to those used by the human visual system. Here we show, by introducing and using minimal recognizable images, that the human visual system uses features and processes that are not used by current models and that are critical for recognition. We found by psychophysical studies that at the level of minimal recognizable images a minute change in the image can have a drastic effect on recognition, thus identifying features that are critical for the task. Simulations then showed that current models cannot explain this sensitivity to precise feature configurations and, more generally, do not learn to recognize minimal images at a human level. The role of the features shown here is revealed uniquely at the minimal level, where the contribution of each feature is essential. A full understanding of the learning and use of such features will extend our understanding of visual recognition and its cortical mechanisms and will enhance the capacity of computational models to learn from visual experience and to deal with recognition and detailed image interpretation.

Human Pose Estimation Using Deep Consensus Voting

In this paper we consider the problem of human pose estimation from a single still image. We propose a novel approach where each location in the image votes for the position of each keypoint using a convolutional neural net. The voting scheme allows us to utilize information from the whole image, rather than rely on a sparse set of keypoint locations. Using dense, multi-target votes, not only produces good keypoint predictions, but also enables us to compute image-dependent joint keypoint probabilities by looking at consensus voting. This differs from most previous methods where joint probabilities are learned from relative keypoint locations and are independent of the image. We finally combine the keypoints votes and joint probabilities in order to identify the optimal pose configuration. We show our competitive performance on the MPII Human Pose and Leeds Sports Pose datasets.

StixelNet: A Deep Convolutional Network for Obstacle Detection and Road Segmentation.

General obstacle detection is a key enabler for obstacle avoidance in mobile robotics and autonomous driving. In this paper we address the task of detecting the closest obstacle in each direction from a driving vehicle. As opposed to existing methods based on 3D sensing we use a single color camera. The main novelty in our approach is the reduction of the task to a column-wise regression problem. The regression is then solved using a deep convolutional neural network (CNN). In addition, we introduce a new loss function based on a semi-discrete representation of the obstacle position probability to train the network. The network is trained using ground truth automatically generated from a laser-scanner point cloud. Using the KITTI dataset, we show that the our monocularbased approach outperforms existing camera-based methods including ones using stereo. We also apply the network on the related task of road segmentation achieving among the best results on the KITTI road segmentation challenge.

Bounding the distance of quantum surface codes

Homological quantum codes (also called topological codes) are low density parity check error correcting codes that come from surfaces and higher dimension manifolds. Homological codes from surfaces, i.e., surface codes, have also been suggested as a possible way to construct stable quantum memory and fault-tolerant computation. It has been conjectured that all homological codes have a square root bound on there distance and therefore cannot produce good codes. This claim has been disputed in dimension four using the geometric property of systolic freedom. We will show in this paper that the conjecture holds in dimension two due to the negation of systolic freedom, i.e., systolic rigidity.

Object interpretation: extending and validating object recognition

1) Extract image measurements for candidate primitives of type points, contours, and regions 2) Score combinations of primitive candidates by their comparability with learned relations. 3) Select the maximum-score combination as the final interpretation of the object structure. • local intensity extrema • parallelism and continuity between two contours • containment of point feature in region • ‘ends-in’ relation between contour and point/region • Cover of point feature by contour • Segmentation and texture support along contours and between contours The Interpretation process: