Stephen D. Milligan

Evolution of internal dynamics for neural network nodes

Most artificial neural networks have nodes that apply a simple static transfer function, such as a sigmoid or gaussian, to their accumulated inputs. This contrasts with biological neurons, whose transfer functions are dynamic and driven by a rich internal structure. Our artificial neural network approach, which we call state-enhanced neural networks, uses nodes with dynamic transfer functions based on n-dimensional real-valued internal state. This internal state provides the nodes with memory of past inputs and computations. The state update rules, which determine the internal dynamics of a node, are optimized by an evolutionary algorithm to fit a particular task and environment. We demonstrate the effectiveness of the approach in comparison to certain types of recurrent neural networks using a suite of partially observable Markov decision processes as test problems. These problems involve both sequence detection and simulated mice in mazes, and include four advanced benchmarks proposed by other researchers.

Genomic computing networks learn complex POMDPs

A genomic computing network is a variant of a neural network for which a genome encodes all aspects, both structural and functional, of the network. The genome is evolved by a genetic algorithm to fit particular tasks and environments. The genome has three portions: one for specifying links and their initial weights, a second for specifying how a node updates its internal state, and a third for specifying how a node updates the weights on its links. Preliminary experiments demonstrate that genomic computing networks can use node internal state to solve POMDPs more complex than those solved previously using neural networks.