Ziang Lv

Control Double Inverted Pendulum by Reinforcement Learning with Double CMAC Network

To accelerate the learning of reinforcement learning, many types of function approximation are used to represent state value. However function approximation reduces the accuracy of state value, and brings difficulty in the convergence. To solve the problems of tradeoff between the generalization and accuracy in reinforcement learning, we represent state-action value by two CMAC networks with different generalization parameters. The accuracy CMAC network can represent values exactly, which achieves precise control in the states around target area. And the generalization CMAC network can extend experiences to unknown area, and guide the learning of accuracy CMAC network. The algorithm proposed in this paper can effectively avoid the dilemma of achieving tradeoff between generalization and accuracy. Simulation results for the control of double inverted pendulum are presented to show effectiveness of the proposed algorithm

Active Exploration Planning in Reinforcement Learning for Inverted Pendulum System Control

Reinforcement learning method usually require that all actions be tried in all state infinitely often for convergence. Such algorithms are impractical to be applied to sophisticated systems due to its low learning efficiency. This paper analyses the problem of limit cycles exist in reinforcement learning for inverted pendulum system control and proposed active exploration planning policy. The algorithm sufficiently makes use of characteristics, active detects limit cycles and plan exploration instead by random exploration. The algorithm action improved the learning efficiency by selecting sub-optimal control action and limiting the exploration to the controllable areas, which can make the number of trials not grow exponentially with the state space. Simulation results for the control of single and double inverted pendulum are presented to show effectiveness of the proposed algorithm

A neural network structure evolution algorithm based on e, m projections and model selection criterion

According to biological and neurophysiologic research, there is a bloom bursting of synapses in brain’s physiological growing process of newborn infants. These jillion nerve connections will be pruned and the dendrites of neurons can change their conformation in infants’ proceeding cognition process. Simulating this pruning process, a new neural network structure evolution algorithm is proposed based on e and m projections in information geometry and model selection criterion. This structure evolution process is formulated in iterative e, m projections and stopped by using model selection criterion. Experimental results prove the validation of the algorithm.

Model Selection Criterion Applied to Visual Perception

Model selection, which means selecting a suitable model from a class of plausible candidate models to explain the given data, is important in almost every field of scientific research including visual perception process. In fact, model selection bears a striking similarity to visual perception in view of inverse problem, and model selection is an all-pervading problem in visual perception. This paper elucidates the relation between model selection and visual perception and shows an illustrative application example of quantitative characterization of Gestalt cues for contour grouping by using a new information geometric model selection criterion IGMSC. The experimental results show that applying model selection criterion in visual perception is helpful to solve the ill-posedness of perception problem and determine the suitable quantitative model

The negative effect on the control of inverted pendulum caused by the limit cycle in reinforcement learning

Control inverted pendulum is one of important applied regions of reinforcement learning. This paper analyzes negative effect on the control of inverted pendulum caused by the limit cycle. It points out the limit cycle will make Q-value converge to zero, and destroy the stabilization of the optimal control policy. Moreover higher degree of exploration can not overcome this problem, but rather intensify it. This paper discuss many solutions to this limit cycle, which succeed controlling inverted pendulum system and keep the stabilization of control policy

A Model of Perceptual Learning Integrated with Top-Down Information

Recent psychological and neurobiological experiments results show that top-down information such as attention and other higher cortical processes play an important role in perceptual learning issues, while current neural network models, mostly concerned with bottom-up information process only, do not combine the top-down information. In this paper, we give a model of perceptual learning that takes top-down information into account, and explain the mechanism of this model in the framework of information geometry.

Perceptual learning inspired model selection method of neural networks

Perceptual learning is the improvement in performance on a variety of simple sensory tasks. Current neural network models mostly concerned with bottom-up processes, and do not incorporate top-down information. Model selection is the crux of learning. To obtain good model we must make balance between the goodness of fit and the complexity of the model. Inspired by perceptual learning, we studied on the model selection of neuro-manifold, use the geometrical method. We propose that the Gauss-Kronecker curvature of the statistical manifold is the natural measurement of the nonlinearity of the manifold. This approach provides a clear intuitive understanding of the model complexity.

Information Geometric Model Selection Criterion and its Application in Cognition

Model selection is important in deciding among competing computational models in many scientific research domains including in cognition processing. This paper presents an information geometric model selection criterion GMSC and shows its application in cognition. IGMSC computes the geometric complexity of the model by regarding the model space as the manifold and estimates the model-data geometric fitness by using the divergence between the true distribution and the asymptotic distribution, enduing complexity and fitness with clear geometric significance. The comparison experiment shows the effect of IGMSC in cognition

Information Geometry Approach to the Model Selection of Neural Networks

Model selection is an efficient method to overcome the over-fitting problem of large-scale neural networks. The crux of model selection is generalization. To obtain good generalization we must make balance between the goodness of fit and the complexity of the model. Most of present methods only focus on the parameters of model, which cannot describe the intrinsic complexity of the model. Information geometry is the application of differential geometry in statistical. We studied on the model selection of neural networks use the information geometry method. We propose that the Gauss-Kronecker curvature of the statistical manifold is the natural measurement of the non-linearity of the manifold. This approach provides a clear intuitive understanding of the model complexity

A New Geometric Approach to the Complexity of Model Selection

Model selection is one of the central problems of machine learning. The goal of model selection is to select from a set of competing explanations the best one that capture the underlying regularities of given observations. The criterion of a good model is generalizability. We must make balance between the goodness of fit and the complexity of the model to obtain good generalization. Most of present methods are consistent in goodness of fit and differ in complexity. But they only focus on the free parameters of the model; hence they cannot describe the intrinsic complexity of the model and they are not invariant under re-parameterization of the model. This paper uses a new geometrical method to study the complexity of the model selection problem. We propose that the integral of the Gauss-Kronecker curvature of the statistical manifold is the natural measurement of the non-linearity of the manifold of the model. This approach provides a clear intuitive understanding of the intrinsic complexity of the model We use an experiment to verify the criterion based on this method