Mitsuo Kawato

The cerebellum and VOR/OKR learning models

Although one particular model of the cerebellum, as proposed by Marr and Albus, provides a formal framework for understanding how heterosynaptic plasticity of Purkinje cells might be used for motor learning, the physiological details remain largely an engima. Developments in computational neuroscience and artificial neural networks applied to real control problems are essential to understand fully how workspace errors associated with movement performances can be converted into motor-command errors, and how these errors can then be used as one kind of synaptic input by motor-learning algorithms that are based on biologically plausible rules involving heterosynaptic plasticity. These developments, as well as recent advances in the study of cellular mechanisms of synaptic plasticity, form the basis for the detailed computational models of cerebellar motor learning that have been proposed. These models provide hints toward resolving a long-standing controversy in the oculomotor literature regarding the sites of adaptive changes in the vestibuloocular reflex (VOR) and the optokinetic eye movement response (OKR), and suggest new experiments to elucidate general mechanisms of sensory motor learning.

Neural network control for a closed-loop System using Feedback-error-learning

This paper presents new learning schemes using feedback-error-learning for a neural network model applied to adaptive nonlinear feedback control. Feedback-error-learning was proposed as a learning method for forming a feedforward controller that uses the output of a feedback controller as the error for training a neural network model. Using new schemes for nonlinear feedback control, the actual responses after learning correspond to the desired responses which are defined by an inverse reference model implemented as a conventional feedback controller. In this respect, these methods are similar to Model Reference Adaptive Control (MRAC) applied to linear or linearized systems. It is shown that learning impedance control is derived when one proposed scheme is used in Cartesian space. We show the results of applying these learning schemes to an inverted pendulum and a 2-link manipulator. We also discuss the convergence properties of the neural network models employed in these learning schemes by applying the Lyapunov method to the averaged equations associated with the stochastic differential equations which describe the system dynamics.

A forward-inverse optics model of reciprocal connections between visual cortical areas

We propose that the feedforward connection from the lower visual cortical area to the higher visual cortical area provides an approximated inverse model of the imaging process (optics), while the backprojection connection from the higher area to the lower area provides a forward model of the optics. By mathematical analysis and computer simulation, we show that a small number of relaxation computations circulating this forward-inverse optics hierarchy achieves fast and reliable integration of vision modules, and therefore might resolve the following problems. (i) How are parallel visual modules (multiple visual cortical areas) integrated to allow a coherent scene perception? (ii) How can ill-posed vision problems be solved by the brain within several hundreds of milliseconds?

A computational model for shape estimation by integration of shading and edge information

Abstract In constructing a computational model for human perception of shape-from-shading, we must solve two major problems: (1) How do we estimate the shape within a small number of iterations? and (2) How do we integrate the shading and edge information for shape estimation? To solve the first problem, we propose a solution for the shape-from-shading problem by using forward and approximated inverse optics. Because the surface normal modification required due to the brightness error is separated from that required due to the smoothness constraint, we can use a simple and fast algorithm for each modification. To solve the second problem, we propose a computational model for shape estimation by integration of shading and edge information. The model is comprised of three modules for the surface normal, discontinuity, and light direction estimation. A rather weak interaction among the three modules can undo the shape-from-shading problem that resembles the chicken-and-egg problem .

Perceived motion in structure from motion: Pointing responses to the axis of rotation

We investigated the ability to match finger orientation to the direction of the axis of rotation in structure-from-motion displays. Preliminary experiments verified that subjects could accurately use the index finger to report direction. The remainder of the experiments studied the perception of the axis of rotation from full rotations of a group of discrete points, the profiles of a rotating ellipsoid, and two views of a group of discrete points. Subjects’ responses were analyzed by decomposing the pointing responses into their slant and tilt components. Overall, the results indicated that subjects were sensitive to both slant and tilt. However, when the axis of rotation was near the viewing direction, subjects had difficulty reporting tilt with profiles and two views and showed a large bias in their slant judgments with two views and full rotations. These results are not entirely consistent with theoretical predictions. The results, particularly for two views, suggest that additional constraints are used by humans in the recovery of structure from motion.

Estimation of Movement from Surface EMG Signals Using a Neural Network Model

Over the years, the neurophysiology and biomechanics of muscle systems have been investigated quite extensively in order to characterize the relations between muscle activity (EMG) and various dynamical and/or kinematic aspects of the ensuing movement behavior. There have been numerous efforts to correlate the duration, magnitude and timing of phasic EMG bursts with the amplitude, duration, and maximum speed of limb motion (Gottlieb, Corcos, and Agarwal 1989; Brown and Cooke 1990; Karst and Hasan 1991). Although the complexity of musculoskeletal systems has made it difficult to reconstruct movement accurately from EMG signals, this goal is central to efforts to model motor control mechanisms of the central nervous system (CNS) computationally.

Correspondence in pictorial space.

ATR Human Information Processing Research Laboratories, Kyoto, Japan We have investigated psychophysically determined image correspondences between pairs of photographs of a single three-dimensional (3-D) object in various poses. These correspondences were obtained by presenting the pictures simultaneously, side by side, and letting the subject match a marker in one picture with a marker (under manual control) in the other picture. Between poses, the object was rotated about a fixed vertical axis; thus, the shifts of the veridical correspondences (with respect to the surface of the object) were very nearly horizontal. In fact, the subjects produced appreciable scatter in both horizontal and vertical directions. The scatter in repeated sessions and between data depends on the local (landmarks) and global (interpolation) structure of the pictures. Since the object was fairly smooth (white semigloss finish) and nontextured, the only way to establish the correspondence is by way of the “pictorial relief.” The relief is some largely unknown function of the image structure and the observer. Apparently, more immediate entities (e.g., the shading or the contour) cannot be used as such, since they vary with the pose. We compare these data with results obtained with a surface attitude probe on a single picture. We studied various measures of consistency both within a single method and between methods. We found that subjects were confident in establishing correspondences, but results scattered appreciably in a way that depended on both global and local image structure. Correspondence results for various pose angles were mutually very consistent, but only to a minor extent with results of attitude measurements. The main finding was that subjects could establish correspondence on the basis of their 3-D interpretation (pictorial relief), even if the 2-D graytone distributions are quite different.

Perception of local orientation from shaded images

The perception of local orientation from shaded images was examined. In Experiment 1, subjects viewed a boundaryless Gaussian hill and judged local orientation using both a gauge figure and a pointing method. One subject reported an internally consistent surface which was incompatible with the judged light-source direction and model used to generate the image. The remaining subjects reported a surface similar to the generating one, and analysis of their results indicated a contour of zero difference between response and generating slants. This contour of zero slant difference was explored in three subsequent experiments using the pointing technique. These experiments investigated possible influences of luminance artifact (Experiment 2), perception of global orientation (Experiment 3), and self-occluding contours (Experiment 4). All three of these experiments yielded results similar to those of Experiment 1, with distinct contours of zero slant difference. This contour was explored for relationships with the simulated slant of the generating surface and the differential structure of image intensity. This analysis indicated that the contour of zero slant difference was approximately a line of constant slant which shared large regions of adjacency to the zero crossings of the second directional derivative of image intensity.

Using motor tasks to quantitatively judge 3-D surface curvatures.

The primary objective of this study was to quantitatively investigate the human perception of surface curvature by using virtual surfaces and motor tasks along with data analysis methods to estimate surface curvature from drawing movements. Three psychophysical experiments were conducted. In Experiment 1, we looked at subjects’ sensitivity to the curvature of a curve lying on a surface and changes in the curvature as defined byEuler’s formula, which relates maximum and minimum principal curvatures and their directions. Regardless of direction and surface shape (elliptic and hyperbolic), subjects could report the curvature of a curve lying on a surface through a drawing task. In addition, multiple curves drawn by subjects were used to reconstruct the surface. These reconstructed surfaces could be better accounted for by analysis that treated the drawing data as a set of curvatures rather than as a set of depths. A pointing task was utilized in Experiment 2, and subjects could report principal curvature directions of a surface rather precisely and consistently when the difference between principal curvatures was sufficiently large, but performance was poor for the direction of zero curvature (asymptotic direction) on a hyperbolic surface. In Experiment 3, it was discovered that sensitivity to the sign of curvature was different for perceptual judgments and motor responses, and there was also a difference for that of a curve itself and the same curve embedded in a surface. These findings suggest that humans are sensitive to relative changes in curvature and are able to comprehend quantitative surface curvature for some motor tasks.

Studies of Human Arm Control Using a Neural Network Model and Surface EMG Signals

Abstract The human arm has at least seven degrees of freedom: the shoulder has three, the elbow has one and the wrist has three. Number of related muscles are about thirty. Quantitative dynamical models of the arm have been playing critical roles in developing recent computational theories of motor control. Unfortunately, construction of a reliable quantitative model based just on reductionistic approach turned out quite difficult if not entirely impossible. We have focused on at constructing a forward dynamics model (FDM) of the human arm in the form of an artificial neural network while using physiological recordings of EMG signals and simultaneous measurement of movement trajectories. In previous studies we have already succeeded in: (1) estimating joint torques under isometric conditions in the horizontal plane (2) estimating four degrees-of-freedom posture in 3D space and (3) estimating joint angular acceleration and reconstructing trajectories in the horizontal plane from surface EMG signals. In this paper, as the final step of our previous efforts, dynamic joint torques at the elbow and shoulder during movements in the horizontal plane are estimated from the surface EMG signals of 10 flexor and extensor muscles using a neural network model with a modular architecture. Moreover different trajectories are reliably reconstructed only from the arm initial condition and the EMG time course using this network and Lagrangean equations of the arm dynamics. This is the first demonstration that multi-joint movements and posture maintenance can be quantitatively and accurately predicted from multiple surface EMG signals while including complicated via-point movements as well as co-contraction of muscles.