Jim Kay

A study of methods of choosing the smoothing parameter in image restoration by regularization

The method of regularization is portrayed as providing a compromise between fidelity to the data and smoothness, with the tradeoff being determined by a scalar smoothing parameter. Various ways of choosing this parameter are discussed in the case of quadratic regularization criteria. They are compared algebraically, and their statistical properties are comparatively assessed from the results of all extensive simulation study based on simple images. >

Gender recognition from point-light walkers.

Point-light displays of human gait provide information sufficient to recognize the gender of a walker and are taken as evidence of the exquisite tuning of the visual system to biological motion. The authors revisit this topic with the goals of quantifying human efficiency at gender recognition. To achieve this, the authors first derive an ideal observer for gender recognition on the basis of center of moment (J. E. Cutting, D. R. Proffitt, & L. T. Kozlowski, 1978) and, with the use of anthropometric data from various populations, show optimal recognition of approximately 79% correct. Next, they perform a meta-analysis of 21 experiments examining gender recognition, obtaining accuracies of 66% correct for a side view and 71% for other views. Finally, results of the meta-analysis and the ideal observer are combined to obtain estimates of human efficiency at gender recognition of 26% for the side view and 47% for other views.

A Critical Comparison of Two Methods of Statistical Discrimination

Important clinical differences arising in the application of commonly advocated discriminant or diagnostic methods demand a thorough assessment of the realism of their different assessments. Recent theoretical work on the estimation of density functions provides reasons for these differences and suggests which methods should provide greater realism. These suggestions are strongly supported by a simulation study. Specific recommendations are made concerning statistical diagnostic practice.

Activation functions, computational goals, and learning rules for local processors with contextual guidance

Information about context can enable local processors to discover latent variables that are relevant to the context within which they occur, and it can also guide short-term processing. For example, Becker and Hinton (1992) have shown how context can guide learning, and Hummel and Biederman (1992) have shown how it can guide processing in a large neural net for object recognition. This article studies the basic capabilities of a local processor with two distinct classes of inputs: receptive field inputs that provide the primary drive and contextual inputs that modulate their effects. The contextual predictions are used to guide processing without confusing them with receptive field inputs. The processors transfer function must therefore distinguish these two roles. Given these two classes of input, the information in the output can be decomposed into four disjoint components to provide a space of possible goals in which the unsupervised learning of Linsker (1988) and the internally supervised learning of Becker and Hinton (1992) are special cases. Learning rules are derived from an information-theoretic objective function, and simulations show that a local processor trained with these rules and using an appropriate activation function has the elementary properties required.

The discovery of structure by multi-stream networks of local processors with contextual guidance

We study multi-stream networks in which feature discovery and associative learning interact cooperatively at the level of the local processors. These processors select and recode the information in their receptive field (RF) inputs that is predictably related to the context within which it occurs. To enable them to do this they are provided with local contextual input in addition to their receptive field input. This input guides both learning and processing to the RF information that is related to the context, but without confounding the information that the processor transmits about the RF. We show that these nets can discover linear functions of their inputs that are predictably related across streams. They can do so whether or not these variables are the most informative within streams, and when there is no evidence within streams as to the existence of these variables. They discover the relevant variables concurrently with, and because of, discovering the predictive relations between them. Two-stage m...

On estimation of noise variance in two-dimensional signal processing

Estimation of noise variance is an important component of digital signal processing, in particular of image processing. In this paper we develop methods for estimating the variance of white noise in a two-dimensional degraded signal. We discuss optimal configurations of pixels for difference-based estimation, and describe asymptotically optimal selection of weights for the component pixels. After extensive analysis of possible configurations we recommend averaging linear configurations over a variety of different orientations (usually two or four). This approach produces estimators with properties of both statistical and numerical efficiency.

Contextually guided unsupervised learning using local multivariate binary processors

We consider the role of contextual guidance in learning and processing within multi-stream neural networks. Earlier work ([Kay and Phillips, 1994][Kay and Phillips, 1996]; [Phillips et al., 1995]) showed how the goals of feature discovery and associative learning could be fused within a single objective and made precise using information theory in such a way that local binary processors could extract a single feature that is coherent across streams. In this paper, we consider multi-unit local processors with multivariate binary outputs that enable a greater number of coherent features to be extracted. Using the Ising model, we define a class of information-theoretic objective functions and also local approximations and derive the learning rules in both cases. These rules have similarities to, and differences from, the celebrated BCM rule. Local and global versions of infomax appear as by-products of the general approach, as well as multivariate versions of coherent infomax. Focussing on the more biologically plausible local rules, we describe some computational experiments designed to investigate specific properties of the processors and the general approach. The main conclusions are: (1) the local methodology introduced in the paper has the required functionality. (2) Different units within the multi-unit processors learned to respond to different aspects of their receptive fields. (3) The units within each processor generally produced a distributed code in which the outputs were correlated and which was robust to damage; in the special case where the number of units available was only just sufficient to transmit the relevant information, a form of competitive learning was produced. (4) The contextual connections enabled the information correlated across streams to be extracted and, by improving feature detection with weak or noisy inputs, they played a useful role in short-term processing and in improving generalization. (5) The methodology allows the statistical associations between distributed self-organizing population codes to be learned.

Feature discovery under contextual supervision using mutual information

The author considers a neural network in which the inputs may be divided into two groups, termed primary inputs and contextual inputs. The goal of the network is to discover those linear functions of the primary inputs that are maximally related to the information contained in the contextual units. The strength of the relationship between the two sets of inputs is measured by using their average mutual information. In the situation where the inputs follow a multivariate, elliptically symmetric probability model, this is equivalent to performing a canonical correlation analysis. A stochastic algorithm is introduced to achieve this analysis. Some theoretical details including a convergence results are presented. Some possible nonlinear extensions are discussed.<<ETX>>

Coherent Infomax as a Computational Goal for Neural Systems

Signal processing in the cerebral cortex is thought to involve a common multi-purpose algorithm embodied in a canonical cortical micro-circuit that is replicated many times over both within and across cortical regions. Operation of this algorithm produces widely distributed but coherent and relevant patterns of activity. The theory of Coherent Infomax provides a formal specification of the objectives of such an algorithm. It also formally derives specifications for both the short-term processing dynamics and for the learning rules whereby the connection strengths between units in the network can be adapted to the environment in which the system finds itself. A central assumption of the theory is that the local processors can combine reliable signal coding with flexible use of those codes because they have two classes of synaptic connection: driving connections which specify the information content of the neural signals, and contextual connections which modulate that signal processing. Here, we make the biological relevance of this theory more explicit by putting more emphasis upon the contextual guidance of ongoing processing, by showing that Coherent Infomax is consistent with a particular Bayesian interpretation for the contextual guidance of learning and processing, by explicitly specifying rules for on-line learning, and by suggesting approximations by which the learning rules can be made computationally feasible within systems composed of very many local processors.

Preservation and Change in the Upland Landscape: The Public Benefits of Grazing Management

A contingent valuation survey was undertaken to estimate the public benefits of landscapechanges that could arise from reductions in grazing levels using the example of the Central Southern Uplands of Scotland. A dichotomous choice with continuous follow-up format was used to quantify the environmentalbenefit in terms of the willingness to pay of the general public and visitors. The paper discusses the merits of this format and the evidence of starting-point bias. In addition, the results are compared with the preferences of locals and interest groups as expressed through focus group sessions and subsets of the survey. A strong preference for more tree cover was evident, a landscape feature not well represented in the current landscape.