Matthew D. Luciw

Dually Optimal Neuronal Layers: Lobe Component Analysis

Development imposes great challenges. Internal ldquocorticalrdquorepresentations must be autonomously generated from interactive experiences. The eventual quality of these developed representations is of course important. Additionally, learning must be as fast as possible-to quickly derive better representation from limited experiences. Those who achieve both of these will have competitive advantages. We present a cortex-inspired theory called lobe component analysis (LCA) guided by the aforementioned dual criteria. A lobe component represents a high concentration of probability density of the neuronal input space. We explain how lobe components can achieve a dual-spatiotemporal (ldquobestrdquo and ldquofastestrdquo)-optimality, through mathematical analysis, in which we describe how lobe components plasticity can be temporally scheduled to take into account the history of observations in the best possible way. This contrasts with using only the last observation in gradient-based adaptive learning algorithms. Since they are based on two cell-centered mechanisms-Hebbian learning and lateral inhibition-lobe components develop in-place, meaning every networked neuron is individually responsible for the learning of its signal-processing characteristics within its connected network environment. There is no need for a separate learning network. We argue that in-place learning algorithms will be crucial for real-world large-size developmental applications due to their simplicity, low computational complexity, and generality. Our experimental results show that the learning speed of the LCA algorithm is drastically faster than other Hebbian-based updating methods and independent component analysis algorithms, thanks to its dual optimality, and it does not need to use any second- or higher order statistics. We also introduce the new principle of fast learning from stable representation.

Incremental slow feature analysis: Adaptive low-complexity slow feature updating from high-dimensional input streams

We introduce here an incremental version of slow feature analysis (IncSFA), combining candid covariance-free incremental principal components analysis (CCIPCA) and covariance-free incremental minor components analysis (CIMCA). IncSFAs feature updating complexity is linear with respect to the input dimensionality, while batch SFAs (BSFA) updating complexity is cubic. IncSFA does not need to store, or even compute, any covariance matrices. The drawback to IncSFA is data efficiency: it does not use each data point as effectively as BSFA. But IncSFA allows SFA to be tractably applied, with just a few parameters, directly on high-dimensional input streams (e.g., visual input of an autonomous agent), while BSFA has to resort to hierarchical receptive-field-based architectures when the input dimension is too high. Further, IncSFAs updates have simple Hebbian and anti-Hebbian forms, extending the biological plausibility of SFA. Experimental results show IncSFA learns the same set of features as BSFA and can handle a few cases where BSFA fails.

An intrinsic value system for developing multiple invariant representations with incremental slowness learning

Curiosity Driven Modular Incremental Slow Feature Analysis (CD-MISFA;) is a recently introduced model of intrinsically-motivated invariance learning. Artificial curiosity enables the orderly formation of multiple stable sensory representations to simplify the agents complex sensory input. We discuss computational properties of the CD-MISFA model itself as well as neurophysiological analogs fulfilling similar functional roles. CD-MISFA combines 1. unsupervised representation learning through the slowness principle, 2. generation of an intrinsic reward signal through learning progress of the developing features, and 3. balancing of exploration and exploitation to maximize learning progress and quickly learn multiple feature sets for perceptual simplification. Experimental results on synthetic observations and on the iCub robot show that the intrinsic value system is essential for representation learning. Representations are typically explored and learned in order from least to most costly, as predicted by the theory of curiosity.

Artificial curiosity with planning for autonomous perceptual and cognitive development

Autonomous agents that learn from reward on high-dimensional visual observations must learn to simplify the raw observations in both space (i.e., dimensionality reduction) and time (i.e., prediction), so that reinforcement learning becomes tractable and effective. Training the spatial and temporal models requires an appropriate sampling scheme, which cannot be hard-coded if the algorithm is to be general. Intrinsic rewards are associated with samples that best improve the agents model of the world. Yet the dynamic nature of an intrinsic reward signal presents a major obstacle to successfully realizing an efficient curiosity-drive. TD-based incremental reinforcement learning approaches fail to adapt quickly enough to effectively exploit the curiosity signal. In this paper, a novel artificial curiosity system with planning is implemented, based on developmental or continual learning principles. Least-squares policy iteration is used with an agents internal forward model, to efficiently assign values for maximizing combined external and intrinsic reward. The properties of this system are illustrated in a high-dimensional, noisy, visual environment that requires the agent to explore. With no useful external value information early on, the self-generated intrinsic values lead to actions that improve both its spatial (perceptual) and temporal (cognitive) models. Curiosity also leads it to learn how it could act to maximize external reward.

Sequential constant size compressors for reinforcement learning

Traditional Reinforcement Learning methods are insufficient for AGIs who must be able to learn to deal with Partially Observable Markov Decision Processes. We investigate a novel method for dealing with this problem: standard RL techniques using as input the hidden layer output of a Sequential Constant-Size Compressor (SCSC). The SCSC takes the form of a sequential Recurrent Auto-Associative Memory, trained through standard back-propagation. Results illustrate the feasibility of this approach -- this system learns to deal with highdimensional visual observations (up to 640 pixels) in partially observable environments where there are long time lags (up to 12 steps) between relevant sensory information and necessary action.

Learning skills from play: Artificial curiosity on a Katana robot arm

Artificial curiosity tries to maximize learning progress. We apply this concept to a physical system. Our Katana robot arm curiously plays with wooden blocks, using vision, reaching, and grasping. It is intrinsically motivated to explore its world. As a by-product, it learns how to place blocks stably, and how to stack blocks.

Multi-channel EEG recordings during 3,936 grasp and lift trials with varying weight and friction

WAY-EEG-GAL is a dataset designed to allow critical tests of techniques to decode sensation, intention, and action from scalp EEG recordings in humans who perform a grasp-and-lift task. Twelve participants performed lifting series in which the object’s weight (165, 330, or 660 g), surface friction (sandpaper, suede, or silk surface), or both, were changed unpredictably between trials, thus enforcing changes in fingertip force coordination. In each of a total of 3,936 trials, the participant was cued to reach for the object, grasp it with the thumb and index finger, lift it and hold it for a couple of seconds, put it back on the support surface, release it, and, lastly, to return the hand to a designated rest position. We recorded EEG (32 channels), EMG (five arm and hand muscles), the 3D position of both the hand and object, and force/torque at both contact plates. For each trial we provide 16 event times (e.g., ‘object lift-off’) and 18 measures that characterize the behaviour (e.g., ‘peak grip force’).

Low complexity proto-value function learning from sensory observations with incremental slow feature analysis

We show that Incremental Slow Feature Analysis (IncSFA) provides a low complexity method for learning Proto-Value Functions (PVFs). It has been shown that a small number of PVFs provide a good basis set for linear approximation of value functions in reinforcement environments. Our method learns PVFs from a high-dimensional sensory input stream, as the agent explores its world, without building a transition model, adjacency matrix, or covariance matrix. A temporal-difference based reinforcement learner improves a value function approximation upon the features, and the agent uses the value function to achieve rewards successfully. The algorithm is local in space and time, furthering the biological plausibility and applicability of PVFs.

Where-What Network 3: Developmental top-down attention for multiple foregrounds and complex backgrounds

The Where-What Network 3 (WWN-3) is an artificial developmental network modeled after visual cortical pathways, for the purpose of attention and recognition in the presence of complex natural backgrounds. It is general-purpose and not pre-determined to detect a certain type of stimulus. It is a learning network, which develops its weights from images using a supervised paradigm and a local Hebbian learning algorithm. Attention has been thought of as bottom-up or top-down. This paper focuses on the biologically-inspired mechanisms of top-down attention in WWN-3, through top-down excitation that interacts with bottom-up activity at every layer within the network. Top-down excitation in WWN-3 can control the location of attention by imposing a certain location or disengaging from the current location. It can also control what type of object to search for. Paired layers and sparse coding deal with potential hallucination problems. Top-down attention in WWN occurs as soon as an action emerges at a motor layer, which could be imposed by a teacher or internally selected. Given two competing foregrounds in the same scene, WWN showed effective performance in all the attention modes tested.

Topographic Class Grouping with applications to 3D object recognition

The cerebral cortex uses a large number of top-down connections, but the roles of the top-down connections remain unclear. Through end-to-end (sensor-to-motor) multilayered networks that use three types of connections (bottom-up, lateral, and top-down), the new topographic class grouping (TCG) mechanism shown in this paper explains how the top-down connections influence (1) the type of feature detectors (neurons) developed and (2) their placement in the neuronal plane. The top-down connections boost the variations in the neuronal between class directions during the training phase. The first outcome of this top-down boosted input space is the facilitation of the emergence of feature detectors that are purer, measured statistically by the average entropy of the neuronspsila development. The relatively purer neurons are more ldquoabstract,rdquo i.e., characterizing class-specific (or motor-specific) input information, resulting in better classification rates. The second outcome of this top-down boosted input space is the increase of the distance between input samples that belong to different classes, resulting in a farther separation of neurons according to their class. Therefore, neurons that respond to the same class become relatively nearer. This results in TCG, measured statistically by a smaller within-class scatter of responses when the neuronal plane has a fixed size. Although these mechanisms are potentially applicable to any pattern recognition applications, we report quantitative effects of these mechanisms for 3D object recognition of center-normalized, background-controlled objects. TCG has enabled a significant reduction of the recognition errors.