Ivan Tyukin

Adaptive observers and parameter estimation for a class of systems nonlinear in the parameters

We consider the problem of asymptotic reconstruction of the state and parameter values in systems of ordinary differential equations. A solution to this problem is proposed for a class of systems of which the unknowns are allowed to be nonlinearly parameterized functions of state and time. Reconstruction of state and parameter values is based on the concepts of weakly attracting sets and non-uniform convergence and is subjected to persistency of excitation conditions. In the absence of nonlinear parametrization the resulting observers reduce to standard estimation schemes. In this respect, the proposed method constitutes a generalization of the conventional canonical adaptive observer design.

Feasibility of random basis function approximators for modeling and control

We discuss the role of random basis function approximators in modeling and control. We analyze the published work on random basis function approximators and demonstrate that their favorable error rate of convergence O(1/n) is guaranteed only with very substantial computational resources. We also discuss implications of our analysis for applications of neural networks in modeling and control.

The economics of motion perception and invariants of visual sensitivity

Neural systems face the challenge of optimizing their performance with limited resources, just as economic systems do. Here, we use tools of neoclassical economic theory to explore how a frugal visual system should use a limited number of neurons to optimize perception of motion. The theory prescribes that vision should allocate its resources to different conditions of stimulation according to the degree of balance between measurement uncertainties and stimulus uncertainties. We find that human vision approximately follows the optimal prescription. The equilibrium theory explains why human visual sensitivity is distributed the way it is and why qualitatively different regimes of apparent motion are observed at different speeds. The theory offers a new normative framework for understanding the mechanisms of visual sensitivity at the threshold of visibility and above the threshold and predicts large-scale changes in visual sensitivity in response to changes in the statistics of stimulation and system goals.

Invariant template matching in systems with spatiotemporal coding: A matter of instability

We consider the design principles of algorithms that match templates to images subject to spatiotemporal encoding. Both templates and images are encoded as temporal sequences of samplings from spatial patterns. Matching is required to be tolerant to various combinations of image perturbations. These include ones that can be modeled as parameterized uncertainties such as image blur, luminance, and, as special cases, invariant transformation groups such as translation and rotations, as well as unmodeled uncertainties (noise). For a system to deal with such perturbations in an efficient way, they are to be handled through a minimal number of channels and by simple adaptation mechanisms. These normative requirements can be met within the mathematical framework of weakly attracting sets. We discuss explicit implementation of this principle in neural systems and show that it naturally explains a range of phenomena in biological vision, such as mental rotation, visual search, and the presence of multiple time scales in adaptation. We illustrate our results with an application to a realistic pattern recognition problem.

Nonuniform Small-Gain Theorems for Systems with Unstable Invariant Sets

We consider the problem of asymptotic convergence to invariant sets in interconnected nonlinear dynamical systems. Standard approaches often require that the invariant sets be uniformly attracting, e.g., stable in the Lyapunov sense. This, however, is neither a necessary requirement nor is always useful. Systems may, for instance, be inherently unstable (e.g., intermittent, itinerant, meta-stable) or the problem statement may include requirements that cannot be satisfied with stable solutions. This is often the case in general optimization problems and in nonlinear parameter identification or adaptation. Conventional techniques for these cases either rely on detailed knowledge of the systems vector-fields or require boundedness of its states. The presently proposed method relies only on estimates of the input-output maps and steady-state characteristics. The method requires the possibility of representing the system as an interconnection of a stable and contracting part with an unstable and exploratory part. We illustrate with examples how the method can be applied to problems of analyzing the asymptotic behavior of locally unstable systems as well as to problems of parameter identification and adaptation in the presence of nonlinear parametrizations. The relation of our results to conventional small-gain theorems is discussed.

SUFFICIENT CONDITIONS FOR SYNCHRONIZATION IN AN ENSEMBLE OF HINDMARSH AND ROSE NEURONS: PASSIVITY-BASED APPROACH

Abstract We investigate synchronization in a system of globally, uniformly and linearly coupled Hindmarsh and Rose oscillators. These oscillators are physiologically realistic models of neural dynamics at the level of a single cell. Aplying a recently developed framework for the analysis of synchronization phenomena, passivitybased approach (A. Pogromsky, H. Nijmeijer), we derive sufficient conditions for global and local asymptotic synchronization in the system. Apart from showing the possibility of synchronization, we concentrate on estimating the least possible values for the coupling connections that are sufficient for convergence of the trajectories to the synchronization manifold.

Approximation with random bases

In this work we discuss the problem of selecting suitable approximators from families of parameterized elementary functions that are known to be dense in a Hilbert space of functions. We consider and analyze published procedures, both randomized and deterministic, for selecting elements from these families that have been shown to ensure the rate of convergence in L2 norm of order O(1/N), where N is the number of elements. We show that both randomized and deterministic procedures are successful if additional information about the families of functions to be approximated is provided. In the absence of such additional information one may observe exponential growth of the number of terms needed to approximate the function and/or extreme sensitivity of the outcome of the approximation to parameters. Implications of our analysis for applications of neural networks in modeling and control are illustrated with examples.

Adaptation in Dynamical Systems

In the context of this book, adaptation is taken to mean a feature of a system aimed at achieving the best possible performance when mathematical models of the environment and the system itself are not fully available. This has applications ranging from theories of visual perception and the processing of information to the more technical problems of friction compensation and adaptive classification of signals in fixed-weight recurrent neural networks. Largely devoted to the problems of adaptive regulation, tracking and identification, this book presents a unifying system-theoretic view on the problem of adaptation in dynamical systems. Special attention is given to systems with nonlinearly parametrized models of uncertainty. Concepts, methods, and algorithms given in the text can be successfully employed in wider areas of science and technology. The detailed examples and background information make this book suitable for a wide range of researchers and graduates in cybernetics, mathematical modeling, and neuroscience.

Observers for Canonic Models of Neural Oscillators

We consider the problem of state and parameter estimation for a class of nonlinear oscillators defined as a system of coupled nonlinear ordinary differential equations. Observable variables are limited to a few components of state vector and an input signal. This class of systems describes a set of canonic models governing the dynamics of evoked potential in neural mem- branes, including Hodgkin-Huxley, Hindmarsh-Rose, FitzHugh-Nagumo, and Morris-Lecar mod- els. We consider the problem of state and parameter reconstruction for these models within the classical framework of observer design. This framework offers computationally-efficient solutions to the problem of state and parameter reconstruction of a system of nonlinear differential equa- tions, provided that these equations are in the so-called adaptive observer canonic form. We show that despite typical neural oscillators being locally observable they are not in the adaptive canonic observer form. Furthermore, we show that no parameter-independent diffeomorphism exists such that the original equations of these models can be transformed into the adaptive canonic observer form. We demonstrate, however, that for the class of Hindmarsh-Rose and FitzHugh-Nagumo models, parameter-dependent coordinate transformations can be used to render these systems into the adaptive observer canonical form. This allows reconstruction, at least partially and up to a (bi)linear transformation, of unknown state and parameter values with exponential rate of conver- gence. In order to avoid the problem of only partial reconstruction and at the same time to be able to deal with more general nonlinear models in which the unknown parameters enter the system nonlinearly, we present a new method for state and parameter reconstruction for these systems. The method combines advantages of standard Lyapunov-based design with more flexible design

Observers for canonic models of neural oscillators

We consider the problem of state and parameter estimation for a class of nonlinear oscillators defined as a system of coupled nonlinear ordinary differential equations. Observable variables are limited to a few components of state vector and an input signal. This class of systems describes a set of canonic models governing the dynamics of evoked potential in neural mem- branes, including Hodgkin-Huxley, Hindmarsh-Rose, FitzHugh-Nagumo, and Morris-Lecar mod- els. We consider the problem of state and parameter reconstruction for these models within the classical framework of observer design. This framework offers computationally-efficient solutions to the problem of state and parameter reconstruction of a system of nonlinear differential equa- tions, provided that these equations are in the so-called adaptive observer canonic form. We show that despite typical neural oscillators being locally observable they are not in the adaptive canonic observer form. Furthermore, we show that no parameter-independent diffeomorphism exists such that the original equations of these models can be transformed into the adaptive canonic observer form. We demonstrate, however, that for the class of Hindmarsh-Rose and FitzHugh-Nagumo models, parameter-dependent coordinate transformations can be used to render these systems into the adaptive observer canonical form. This allows reconstruction, at least partially and up to a (bi)linear transformation, of unknown state and parameter values with exponential rate of conver- gence. In order to avoid the problem of only partial reconstruction and at the same time to be able to deal with more general nonlinear models in which the unknown parameters enter the system nonlinearly, we present a new method for state and parameter reconstruction for these systems. The method combines advantages of standard Lyapunov-based design with more flexible design