Felix Reinhart

Rare neural correlations implement robotic conditioning with delayed rewards and disturbances

Neural conditioning associates cues and actions with following rewards. The environments in which robots operate, however, are pervaded by a variety of disturbing stimuli and uncertain timing. In particular, variable reward delays make it difficult to reconstruct which previous actions are responsible for following rewards. Such an uncertainty is handled by biological neural networks, but represents a challenge for computational models, suggesting the lack of a satisfactory theory for robotic neural conditioning. The present study demonstrates the use of rare neural correlations in making correct associations between rewards and previous cues or actions. Rare correlations are functional in selecting sparse synapses to be eligible for later weight updates if a reward occurs. The repetition of this process singles out the associating and reward-triggering pathways, and thereby copes with distal rewards. The neural network displays macro-level classical and operant conditioning, which is demonstrated in an interactive real-life human-robot interaction. The proposed mechanism models realistic conditioning in humans and animals and implements similar behaviors in neuro-robotic platforms.

Generalizing a learned inverse dynamic model of KUKA LWR IV+ for load variations using regression in the model space

In this paper, we show the generalization of an inverse dynamic model for KUKA LWR IV+ under load mass variations. We use a modular approach based on regression in the model space. First, inverse dynamic models for the known masses are learned using a recently proposed approach called Independent Joint Learning (IJL). In IJL the torque errors due to unmodeled dynamics of the real robot are estimated using only joint-local information. Second, a mapping from load mass to model parameters of torque error model is learned in order to generalize the inverse dynamics to new load masses. The modular approach improves the accuracy of an existing KUKA LWR IV+ inverse dynamic model. The results are compared with a single step IJL approach. The results show the excellent generalization for new load masses using regression in the model space.