Erhardt Barth

Fundamental limits of linear filters in the visual processing of two-dimensional signals

Since Lettvin, Maturana, McCulloch and Pitts (1959), neurophysiologists have known that the visual system contains detectors that respond to stimulus features such as “bugs”, line ends, bars, comers, etc. (Hubel & Wiesel, 1965). During the same period, the theory of linear systems has been applied successfully to the analysis and modelling of visual functions (DeValois & DeValois, 1980). Interestingly, however, there is a fundamental incompatibility between these two approaches that has not yet received adequate attention. Linear filtering, even if modified by common nonlinear&& like thresholding or rectification, will generally confound straight signals with signals that show essentially two-dimensional variations. This principle deficiency is illustrated for a curvature detector recently suggested by Dobbins, Zucker and Cynader (1987, 1989), which is based on a nonlinear combination of linear filters. However, the problem can be solved by using the mathematical formalism of differential geometry. We employ the concept of “Gaussian curvature” of surfaces to derive a class of physiologically plausible operators for the detection of two-dimensional signal variations. Two essential properties of these detectors turn out to be necessary: the use of “and” operations, that are impossible with linear filters, and a specific “compensation principle” corresponding to inhibitory interactions between orientation selective filters. One example for the encoding of essentially two-dimensional signal variations is the detection of curvature. According to a recent hypothesis by Dobbins et al. (1987, 1989), this can be accomplished by using the difference between the outputs of two simple cells with different receptive field sizes to generate “endstopped” responses that proportionally vary to stimulus length and curvature. It can be shown, however, that this particular model, as well as any essentially linear system, is subject to response ambiguities in that it is always possible to find a stimulus of zero curvature that erroneously elicits a response. Consider the stimulus configuration shown in Fig. la. While the curved lines and short bars give rise to appropriate responses of the Dobbins et al. detector, it also reacts erroneously to certain straight stimuli on the right side (Fig. lb). The corresponding critical spectral area within which such false responses can occur is indicated in Fig. 2. The very reason for the occurrence of such ambiguous responses has to be sought in a fundamental limitation of linear filters in the processing of two-dimensional signals. Such signals can be classified into three elementary categories: (1) constant signals that show no variation at all; (2) intrinsically onedimensional signals that are constant along one orientation and can, therefore, be completely characterized by their variation along the orthogonal orientation (here: lD-signals); (3) actually two-dimensional signals that vary along all orientations (here: ZD-signals). Obvious examples of 1-D signals are straight lines, straight edges, or sinusoidal gratings with arbitrary orientation. Curved lines, curved edges and junctions, intersections, terminations, etc. are typical 2D-signals (Marko, 1974; Julesz, 1981). An essential requirement for all detectors which encode ZD-signal properties is that they should not erroneously respond to ID-signals. Curvature detectors, for example, should not respond to straight stimuli. We will show that such an unambiguous detection of ZD-signal properties necessarily employs “and” operations. Such

How honeybees make grazing landings on flat surfaces.

Abstract. Freely flying bees were filmed as they landed on a flat, horizontal surface, to investigate the underlying visuomotor control strategies. The results reveal that (1) landing bees approach the surface at a relatively shallow descent angle; (2) they tend to hold the angular velocity of the image of the surface constant as they approach it; and (3) the instantaneous speed of descent is proportional to the instantaneous forward speed. These characteristics reflect a surprisingly simple and effective strategy for achieving a smooth landing, by which the forward and descent speeds are automatically reduced as the surface is approached and are both close to zero at touchdown. No explicit knowledge of flight speed or height above the ground is necessary. A model of the control scheme is developed and its predictions are verified. It is also shown that, during landing, the bee decelerates continuously and in such a way as to keep the projected time to touchdown constant as the surface is approached. The feasibility of this landing strategy is demonstrated by implementation in a robotic gantry equipped with vision.

Time-of-Flight Sensors in Computer Graphics

A growing number of applications depend on accurate and fast 3D scene analysis. Examples are model and lightfield acquisition, collision prevention, mixed reality, and gesture recognition. The estimation of a range map by image analysis or laser scan techniques is still a time-consuming and expensive part of such systems. A lower-priced, fast and robust alternative for distance measurements are Time-of-Flight (ToF) cameras. Recently, significant advances have been made in producing low-cost and compact ToF-devices, which have the potential to revolutionize many fields of research, including Computer Graphics, Computer Vision and Human Machine Interaction (HMI). These technologies are starting to have an impact on research and commercial applications. The upcoming generation of ToF sensors, however, will be even more powerful and will have the potential to become “ubiquitous real-time geometry devices” for gaming, web-conferencing, and numerous other applications. This STAR gives an account of recent developments in ToF-technology and discusses the current state of the integration of this technology into various graphics-related applications.

Time‐of‐Flight Cameras in Computer Graphics

A growing number of applications depend on accurate and fast 3D scene analysis. Examples are model and lightfield acquisition, collision prevention, mixed reality and gesture recognition. The estimation of a range map by image analysis or laser scan techniques is still a time‐consuming and expensive part of such systems.

Simple Method for High-Performance Digit Recognition Based on Sparse Coding

In this brief paper, we propose a method of feature extraction for digit recognition that is inspired by vision research: a sparse-coding strategy and a local maximum operation. We show that our method, despite its simplicity, yields state-of-the-art classification results on a highly competitive digit-recognition benchmark. We first employ the unsupervised Sparsenet algorithm to learn a basis for representing patches of handwritten digit images. We then use this basis to extract local coefficients. In a second step, we apply a local maximum operation to implement local shift invariance. Finally, we train a support vector machine (SVM) on the resulting feature vectors and obtain state-of-the-art classification performance in the digit recognition task defined by the MNIST benchmark. We compare the different classification performances obtained with sparse coding, Gabor wavelets, and principal component analysis (PCA). We conclude that the learning of a sparse representation of local image patches combined with a local maximum operation for feature extraction can significantly improve recognition performance.

Analysis of Superimposed Oriented Patterns

Estimation of local orientation in images may be posed as the problem of finding the minimum gray-level variance axis in a local neighborhood. In bivariate images, the solution is given by the eigenvector corresponding to the smaller eigenvalue of a 2times2 tensor. For an ideal single orientation, the tensor is rank-deficient, i.e., the smaller eigenvalue vanishes. A large minimal eigenvalue signals the presence of more than one local orientation, what may be caused by non-opaque additive or opaque occluding objects, crossings, bifurcations, or corners. We describe a framework for estimating such superimposed orientations. Our analysis is based on the eigensystem analysis of suitably extended tensors for both additive and occluding superpositions. Unlike in the single-orientation case, the eigensystem analysis does not directly yield the orientations, rather, it provides so-called mixed-orientation parameters (MOPs). We, therefore, show how to decompose the MOPs into the individual orientations. We also show how to use tensor invariants to increase efficiency, and derive a new feature for describing local neighborhoods which is invariant to rigid transformations. Applications are, e.g., in texture analysis, directional filtering and interpolation, feature extraction for corners and crossings, tracking, and signal separation

Automated Indirect Immunofluorescence Evaluation of Antinuclear Autoantibodies on HEp-2 Cells

Indirect immunofluorescence (IIF) on human epithelial (HEp-2) cells is considered as the gold standard screening method for the detection of antinuclear autoantibodies (ANA). However, in terms of automation and standardization, it has not been able to keep pace with most other analytical techniques used in diagnostic laboratories. Although there are already some automation solutions for IIF incubation in the market, the automation of result evaluation is still in its infancy. Therefore, the EUROPattern Suite has been developed as a comprehensive automated processing and interpretation system for standardized and efficient ANA detection by HEp-2 cell-based IIF. In this study, the automated pattern recognition was compared to conventional visual interpretation in a total of 351 sera. In the discrimination of positive from negative samples, concordant results between visual and automated evaluation were obtained for 349 sera (99.4%, kappa = 0.984). The system missed out none of the 272 antibody-positive samples and identified 77 out of 79 visually negative samples (analytical sensitivity/specificity: 100%/97.5%). Moreover, 94.0% of all main antibody patterns were recognized correctly by the software. Owing to its performance characteristics, EUROPattern enables fast, objective, and economic IIF ANA analysis and has the potential to reduce intra- and interlaboratory variability.

Sparse Coding Neural Gas: Learning of overcomplete data representations

We consider the problem of learning an unknown (overcomplete) basis from data that are generated from unknown and sparse linear combinations. Introducing the Sparse Coding Neural Gas algorithm, we show how to employ a combination of the original Neural Gas algorithm and Ojas rule in order to learn a simple sparse code that represents each training sample by only one scaled basis vector. We generalize this algorithm by using Orthogonal Matching Pursuit in order to learn a sparse code where each training sample is represented by a linear combination of up to k basis elements. We evaluate the influence of additive noise and the coherence of the original basis on the performance with respect to the reconstruction of the original basis and compare the new method to other state of the art methods. For this analysis, we use artificial data where the original basis is known. Furthermore, we employ our method to learn an overcomplete representation for natural images and obtain an appealing set of basis functions that resemble the receptive fields of neurons in the primary visual cortex. An important result is that the algorithm converges even with a high degree of overcompleteness. A reference implementation of the methods is provided.

Analytic solutions for multiple motions

A novel framework for single and multiple motion estimation is presented. It is based on a generalized structure tensor that contains blurred products of directional derivatives. The order of differentiation increases with the number of motions but more general linear filters can be used instead of derivatives. From the general framework, a hierarchical algorithm for motion estimation is derived and its performance is demonstrated on a synthetic sequence.

Estimation of multiple local orientations in image signals

Local orientation estimation can be posed as the problem of finding the minimum grey level variance axis within a local neighbourhood. In 2D image signals, this corresponds to the eigensystem analysis of a 2 /spl times/ 2-tensor, which yields valid results for single orientations. We describe extensions to multiple overlaid orientations, which may be caused by transparent objects, crossings, bifurcations, corners etc. Multiple orientation detection is based on the eigensystem analysis of an appropriately extended tensor, yielding so-called mixed orientation parameters. These mixed orientation parameters can be regarded as another tensor built from the sought individual orientation parameters. We show how the mixed orientation tensor can be decomposed into the individual orientations by finding the roots of a polynomial. Applications are, e.g., in directional filtering and interpolation, feature extraction for corners or crossings, and signal separation.