René Felix Reinhart

Online learning and generalization of parts-based image representations by non-negative sparse autoencoders

We present an efficient online learning scheme for non-negative sparse coding in autoencoder neural networks. It comprises a novel synaptic decay rule that ensures non-negative weights in combination with an intrinsic self-adaptation rule that optimizes sparseness of the non-negative encoding. We show that non-negativity constrains the space of solutions such that overfitting is prevented and very similar encodings are found irrespective of the network initialization and size. We benchmark the novel method on real-world datasets of handwritten digits and faces. The autoencoder yields higher sparseness and lower reconstruction errors than related offline algorithms based on matrix factorization. It generalizes to new inputs both accurately and without costly computations, which is fundamentally different from the classical matrix factorization approaches.

Neural learning and dynamical selection of redundant solutions for inverse kinematic control

We introduce a novel recurrent neural network controller that learns and maintains multiple solutions of the inverse kinematics. Redundancies are resolved dynamically by means of multi-stable attractor dynamics. The associative network comprises a combined forward and inverse model of the robots kinematics and enables flexible selection of control spaces by mixing constraints in task space and joint space. The network is integrated into a feedforward-feedback control framework which enables dynamical movement generation. We show results for the humanoid robot iCub in simulation.

Recurrent Neural Associative Learning of Forward and Inverse Kinematics for Movement Generation of the Redundant PA-10 Robot

We present a connectionist approach to learn forward and redundant inverse kinematics in a single recurrent network. The network architecture extends the reservoir computing idea, i.e. to read out the state of a fixed dynamic system, into an associative setting, which learns the forward and backward mapping simultaneously. For output learning we use efficient Backpropagation-Decorrelation learning while the recurrent dynamics is adjusted by an unsupervised biologically inspired learning rule based on intrinsic plasticity. Including linear connections between input and output allows to train the network for autonomous movement generation. We show results for the 7-DOF redundant PA-10 robot arm in simulation.

Regularization and stability in reservoir networks with output feedback

Output feedback is crucial for autonomous and parameterized pattern generation with reservoir networks. Read-out learning affects the output feedback loop and can lead to error amplification. Regularization is therefore important for both generalization and reduction of error amplification. We show that regularization of the reservoir and the read-out layer reduces the risk of error amplification, mitigates parameter dependency and boosts the task-specific performance of reservoir networks with output feedback. We discuss the deeper connection between regularization of the learning process and stability of the trained network.

Dynamics identification of a damped multi elastic link robot arm under gravity.

The infinite dimensionality, varying, uncertainties or even unknown boundary conditions render the derivation and - in particular - the identification of accurate dynamics models for elastic link robots tedious and error prone. This contribution circumvents these challenges by the prior application of a model-free inner loop oscillation damping controller before modelling the robots dynamics. Then, the damped dynamics of a multi elastic link robot arm under gravity can be modelled with high accuracy. An analytical and a data-driven model for the damped dynamics are proposed and quantitatively compared. Both models can explain motor currents as well as link strain measurements in real-time. The paper includes an experimental model validation with different payloads in the entire workspace of the robot.

Neural learning of vector fields for encoding stable dynamical systems

Abstract The data-driven approximation of vector fields that encode dynamical systems is a persistently hard task in machine learning. If data is sparse and given in the form of velocities derived from few trajectories only, state-space regions exist, where no information on the vector field and its induced dynamics is available. Generalization towards such regions is meaningful only if strong biases are introduced, for instance assumptions on global stability properties of the to-be-learned dynamics. We address this issue in a novel learning scheme that represents vector fields by means of neural networks, where asymptotic stability of the induced dynamics is explicitly enforced through utilizing knowledge from Lyapunov׳s stability theory, in a predefined workspace. The learning of vector fields is constrained through point-wise conditions, derived from a suitable Lyapunov function candidate, which is first adjusted towards the training data. We point out the significance of optimized Lyapunov function candidates and analyze the approach in a scenario where trajectories are learned and generalized from human handwriting motions. In addition, we demonstrate that learning from robotic data obtained by kinesthetic teaching of the humanoid robot iCub leads to robust motion generation.

Representation and generalization of bi-manual skills from kinesthetic teaching

The paper presents a modular architecture for bi-manual skill acquisition from kinesthetic teaching. Skills are learned and embedded over several representational levels comprising a compact movement representation by means of movement primitives, a task space description of the bi-manual tool constraint, and the particular redundancy resolution of the inverse kinematics. A comparative evaluation of different architectural configurations identifies a specific modulation scheme for skill execution to achieve optimal spatial generalization from few training samples. Based on this architectural layout together with a novel stabilization approach for dynamical movement primitives, the robust teaching and execution of complex skill sequences is demonstrated on the humanoid robot iCub.

Recurrence enhances the spatial encoding of static inputs in reservoir networks

We shed light on the key ingredients of reservoir computing and analyze the contribution of the network dynamics to the spatial encoding of inputs. Therefore, we introduce attractor-based reservoir networks for processing of static patterns and compare their performance and encoding capabilities with a related feedforward approach. We show that the network dynamics improve the nonlinear encoding of inputs in the reservoir state which can increase the task-specific performance.

An active compliant control mode for interaction with a pneumatic soft robot

Bionic soft robots offer exciting perspectives for more flexible and safe physical interaction with the world and humans. Unfortunately, their hardware design often prevents analytical modeling, which in turn is a prerequisite to apply classical automatic control approaches. On the other hand, also modeling by means of learning is hardly feasible due to many degrees of freedom, high-dimensional state spaces and the softness properties like e.g. mechanical elasticity, which cause limited repeatability and complex dynamics. Nevertheless, the realization of basic control modes is important to leverage the potential of soft robots for applications. We therefore propose a hybrid approach combining classical and learning elements for the realization of an interactive control mode for an elastic bionic robot. It superimposes a low-gain feedback control with a feed-forward control based on a learned simplified model of the inverse dynamics which considers only equilibria of the robots dynamics. We demonstrate on the Bionic Handling Assistant how a respective inverse equilibrium model can be learned and effectively exploited for quick and agile control. In a second step, the control scheme is extended to an active compliant control mode. It implements a kind of gravitation compensation to allow for kinesthetic teaching of the robot based on the implicit knowledge of gravitational and mechanical forces that are encoded in the learned equilibrium model. We finally discuss that this control scheme may be implemented also on other soft robots to provide the avenue towards their applications in general manipulation tasks.

A multi-level control architecture for the bionic handling assistant

The bionic handling assistant is one of the largest soft continuum robots and very special in being a pneumatically operated platform that is able to bend, stretch, and grasp in all directions. It nevertheless shares many challenges with smaller continuum and other soft robots such as parallel actuation, complex movement dynamics, slow pneumatic actuation, non-stationary behavior, and a lack of analytic models. To master the control of this challenging robot, we argue for a tight integration of standard analytic tools, simulation, control, and state-of-the-art machine learning into an overall architecture that can serve as blueprint for control design also beyond the BHA. To this aim, we show how to integrate specific modes of operation and different levels of control in a synergistic manner, which is enabled by using modern paradigms of software architecture and middleware. We thereby achieve an architecture with unique overall control abilities for a soft continuum robot that allow for flexible experimentation toward compliant user-interaction, grasping, and online learning of internal models. Graphical Abstract