Vladimir G. Red'ko

Theory of functional systems, adaptive critics and neural networks

We propose a general scheme of intelligent adaptive control system based on the Petr K. Anokhins theory of functional systems. This scheme is aimed at controlling adaptive purposeful behavior of an animat (a simulated animal) that has several natural needs (e.g., energy replenishment, reproduction). The control system consists of a set of hierarchically linked functional systems and enables predictive and goal-directed behavior. Each functional system includes a neural network based adaptive critic design. We also discuss schemes of prognosis, decision making, action selection and learning that occur in the functional systems and in the whole control system of the animat.

2005 Special Issue: A model of evolution and learning

We study a model of evolving populations of self-learning agents and analyze the interaction between learning and evolution. We consider an agent-broker that predicts stock price changes and uses its predictions for selecting actions. Each agent is equipped with a neural network adaptive critic design for behavioral adaptation. We discuss three cases in which either evolution or learning, or both, are active in our model. We show that the Baldwin effect can be observed in our model, viz. originally acquired adaptive policy of best agent-brokers becomes inherited over the course of the evolution. We also compare the behavioral tactics of our agents to the searching behavior of simple animals.

Project Animat Brain: Designing the Animat Control System on the Basis of the Functional Systems Theory

The paper proposes the framework for an animat control system (the Animat Brain) that is based on the Petr K. Anokhins theory of functional systems. We propose the animat control system that consists of a set of functional systems (FSs) and enables predictive and purposeful behavior. Each FS consists of two neural networks: the actor and the predictor. The actors are intended to form chains of actions and the predictors are intended to make prognoses of future events. There are primary and secondary repertoires of behavior: the primary repertoire is formed by evolution; the secondary repertoire is formed by means of learning. This paper describes both principles of the Animat Brain operation and the particular model of predictive behavior in a cellular landmark environment.

Model of Evolutionary Emergence of Purposeful Adaptive Behavior. The Role of Motivation

The process of evolutionary emergence of purposeful adaptive behavior is investigated by means of computer simulations. The model proposed implies that there is an evolving population of simple agents, which have two natural needs: energy and reproduction. Any need is characterized quantitatively by a corresponding motivation. Motivations determine goal-directed behavior of agents. The model demonstrates that purposeful behavior does emerge in the simulated evolutionary processes. Emergence of purposefulness is accompanied by origin of a simple hierarchy in the control system of agents.

A model of Baldwin effect in populations of self-learning agents

We study an evolution model of adaptive self-learning agents. The control system of agents is based on a neural network adaptive critic design. Each agent is a broker that predicts stock price changes and uses its predictions for action selection. The agent tries to get rich by buying and selling stocks. We demonstrate that the Baldwin effect takes place in our model, viz., originally acquired adaptive policy of an agent-broker becomes inherited in the course of the evolution. In addition, we compare agent behavioral tactics with searching behavior of simple animals.

Estimation of the evolution speed for the quasispecies model: arbitrary alphabet case

The efficiency of the evolutionary search in M. Eigens quasispecies model for the case of an arbitrary alphabet (the arbitrary number of possible string symbols) is estimated. Simple analytical formulas for the evolution rate and the total number of fitness function calculations are obtained. Analytical estimations are proved by computer simulations. It is shown that for the case of unimodal fitness function of A-ary strings of length N, the optimal string can be found during (λ-1)N generations under condition that the total number of fitness function calculations is of the order of [(λ-1)N] 2 .