Tomas Maul

Local receptive field constrained deep networks

Extension of RBMs with Gaussian local receptive field constraints on hidden nodes.Method for automatically learning advantageous, non-uniform feature detector placement from data.Novel biologically inspired deep network models (LRF-DBN and LRF-DNN) with features of gradually increasing size.Superior face completion and dimensionality reduction results achieved.Feature hierarchy of the proposed model demonstrates part-based compositionality. Automatic extraction of distinctive features from a visual information stream is challenging due to the large amount of information contained in most image data. In recent years deep neural networks (DNNs) have gained outstanding popularity for solving visual information processing tasks. This study reports novel contributions, including a new DNN architecture and training method, which increase the fidelity of DNN-based representations to encodings extracted by visual processing neurons. Our local receptive field constrained DNNs (LRF-DNNs) are pre-trained with a modified restricted Boltzmann machine, the LRF-RBM, which utilizes biologically inspired Gaussian receptive field constraints to encourage the emergence of local features. Moreover, we propose a method for concurrently finding advantageous receptive field centers, while training the LRF-RBM. By utilizing LRF-RBMs with gradually increasing receptive field sizes on each layer, our LRF-DNN learns features of increasing complexity and demonstrates hierarchical part-based compositionality. We show superior face completion and reconstruction results on the challenging LFW face dataset.

A REVIEW OF RETINAL PROSTHESIS APPROACHES

Age-related macular degeneration and retinitis pigmentosa are two of the most common diseases that cause degeneration in the outer retina, which can lead to several visual impairments up to blindness. Vision restoration is an important goal for which several different research approaches are currently being pursued. We are concerned with restoration via retinal prosthetic devices. Prostheses can be implemented intraocularly and extraocularly, which leads to different categories of devices. Cortical Prostheses and Optic Nerve Prostheses are examples of extraocular solutions while Epiretinal Prostheses and Subretinal Prostheses are examples of intraocular solutions. Some of the prostheses that are successfully implanted and tested in animals as well as humans can restore basic visual functions but still have limitations. This paper will give an overview of the current state of art of Retinal Prostheses and compare the advantages and limitations of each type. The purpose of this review is thus to summarize the current technologies and approaches used in developing Retinal Prostheses and therefore to lay a foundation for future designs and research directions.

Early experiments with neural diversity machines

The current paper introduces the concept of neural diversity machines (NDM) which, refers to hybrid artificial neural networks (HANN) with conditions on the minimum number of functions available to the network, amongst several other properties. The paper demonstrates how NDM networks can be optimized for solving different problems. The results demonstrate the feasibility of the approach and bolster some of the biological and computational arguments in favor of neural diversity. A substantial number of optimization experiments were conducted, generating a corresponding number of diverse neural architectures, which revealed several unexpected statistics, including the relative commonality of nodes combining inner-product and Gaussian functions. The paper confirms the advantages of HANN, demonstrates the potential of increasing the focus on neural diversity and hints at possible new neural computational strategies.

Simulation Modelling Study of Self-Assembled Nanoparticle Coatings for Retinal Implants

The electrode resolution of current retinal prostheses is still far from matching the densities of retinal neurons. Decreasing electrode diameter increases impedance levels thus deterring effective stimulation of neurons. One solution is to increase the surface roughness of electrodes, which can be done via nanoparticle coatings. This paper explores a Lattice Gas Model of the drying-mediated self-assembly of nanoparticle mixtures. The model includes representations for different types of nanoparticles, solvent, vapour, substrate and the energetic relationships between these elements. The dynamical aspect of the model is determined by energy minimization, stochastic fluctuations and physical constraints. The model attempts to unravel the rela-tionships between different experimental conditions (e.g. evaporation rate, substrate characteristics and solvent viscosity) and the surface roughness of resulting assemblies. Some of the main results include the facts that the assemblies formed by nanoparticles of different sizes can boost roughness in specific circumstances and that the optimized assemblies can exhibit walled or stalagmite structures. This study provides a set of simulation modelling experiments that if confirmed in the laboratory may result in new and useful materials.

Cybernetics of Vision Systems: Toward an Understanding of Putative Functions of the Outer Retina

The retina still poses many structural and computational questions. Structurally, for example, it is not yet clear how many distinct horizontal cell (HC) types the primate retina contains and what the exact patterns of connections between photoreceptors (PRs) and HCs consist of. Computationally, it is not yet clear, for instance, what functions are present and how they are being implemented. This paper proposes a model (a linear recurrent neural network defined by 31 parameters) of the outer retina and an optimization methodology that hopes to shed some light on these questions. This paper shows that a simplified model of the outer retina can implement several low-level visual functions involving the modulation of noise, brightness, contrast, saturation, and even color. The results demonstrate that contrast control functions can be implemented with a minimum of two HC types and that spectral specificity between PRs and HCs is a common and important feature. It is also shown that several different spectrally specific patterns can emerge in order to implement the same function. One interesting microcircuit that naturally emerged from our experiments involves nonblurry denoising via interchromatic gap junctions and compensatory resaturation via HC circuits, a strategy that we hypothesize to exist in some biological retinae.

Swiping with luminophonics

Luminophonics is a system that aims to maximize cross-modality conversion of information, specifically from the visual to auditory modalities, with the motivation to develop a better assistive technology for the visually impaired by using image sonification techniques. The project aims to research and develop generic and highly-configurable components concerned with different image processing techniques, attention mechanisms, orchestration approaches and psychological constraints. The swiping method that is introduced in this paper combines several techniques in order to explicitly convert the colour, size and position of objects. Preliminary tests suggest that the approach is valid and deserves further investigation.

Modelling Retinal Feature Detection With Deep Belief Networks In A Simulated Environment.

Recent research has demonstrated the great capability of deep belief networks for solving a variety of visual recognition tasks. However, primary focus has been on modelling higher level visual features and later stages of visual processing found in the brain. Lower level processes such as those found in the retina have gone ignored. In this paper, we address this issue and demonstrate how the retina’s inherent multi-layered structure lends itself naturally for modelling with deep networks. We introduce a method for simulating the retinal photoreceptor input and show the efficacy of deep networks in learning feature detectors similar to retinal ganglion cells. We thereby demonstrate the potential of deep belief networks for modelling the earliest stages of visual processing.

Investigations into Particle Swarm Optimization for Multi-class Shape Recognition

There has been a significant drop in the cost as well as an increase in the quality of imaging sensors due to stiff competition as well as production improvements. Consequently, real-time surveillance of private or public spaces which relies on such equipment is gaining wider acceptance. While the human brain is very good at image analysis, fatigue and boredom may contribute to a less-than-optimum level of monitoring performance. Clearly, it would be good if highly accurate vision systems could complement the role of humans in round-the-clock video surveillance. This paper addresses an image analysis problem for video surveillance based on the particle swarm computing paradigm. In this study three separate datasets were used. The overall finding of the paper suggests that clustering using Particle Swarm Optimization leads to better and more consistent results, in terms of both cluster characteristics and subsequent recognition, as compared to traditional techniques such as K-Means.

A parallel circuit approach for improving the speed and generalization properties of neural networks

One of the common problems of neural networks, especially those with many layers consists of their lengthy training times. We attempted to solve this problem at the algorithmic (not hardware) level, proposing a simple parallel design inspired by the parallel circuits found in the human retina. To avoid large matrix calculations, we split the original network vertically into parallel circuits and let the BP algorithm flow in each subnetwork independently. Experimental results have shown the speed advantage of the proposed approach but also pointed out that the reduction is affected by multiple dependencies. The results also suggest that parallel circuits improve the generalization ability of neural networks presumably due to automatic problem decomposition.

Towards evolutionary deep neural networks

This paper is concerned with the problem of optimizing deep neural networks with diverse transfer functions using evolutionary methods. Standard evolutionary (SEDeeNN) and cooperative coevolutionary methods (CoDeeNN) were applied to three different architectures characterized by different constraints on neural diversity. It was found that (1) SEDeeNN (but not CoDeeNN) changes parameters uniformly across all layers, (2) both evolutionary approaches can exhibit good convergence and generalization properties, and (3) increased neural diversity improves both convergence and generalization. In addition to clarifying the feasibility of evolutionary deep neural networks, we suggests a guiding principle for synergizing evolutionary and error gradient based approaches through layerchange analysis.