Ambra Bisio

Measuring Human-Robot Interaction Through Motor Resonance

In the last decades, the introduction of robotic devices in fields such as industries, dangerous environments, and medicine has notably improved working practices. The availability of a new generation of humanoid robots for everyday’s activities in human populated environments can entail an even wider revolution. Indeed, not only domestic activities but also social behaviors will adapt to a continuous interaction with a completely new kind of social agents.In the light of this scenario, it becomes crucial to design robots suited to natural cooperation with humans, and contextually to develop quantitative methods to measure human-robot interaction (HRI). Motor resonance, i.e. the activation of the observer’s motor control system during action perception, has been suggested to be a key component of human social behavior, and as such is thought to play a central role for HRI.In the literature there are reports of robots that have been used as tools to understand the human brain. The aim of this review is to offer a different perspective in suggesting that human responses can become a tool to measure and improve robot interactional attitudes. In the first part of the paper the notion of motor resonance and its neurophysiological correlates are introduced. Subsequently we describe motor resonance studies on the perception of robotic agents’ behavior. Finally we introduce proactive gaze and automatic imitation, two techniques adopted in human motor resonance studies, and we present the advantages which would follow their application to HRI.

Motor cortical plasticity induced by motor learning through mental practice.

Several investigations suggest that actual and mental actions trigger similar neural substrates. Motor learning via physical practice results in long-term potentiation (LTP)-like plasticity processes, namely potentiation of M1 and a temporary occlusion of additional LTP-like plasticity. However, whether this neuroplasticity process contributes to improve motor performance through mental practice remains to be determined. Here, we tested skill learning-dependent changes in primary motor cortex (M1) excitability and plasticity by means of transcranial magnetic stimulation (TMS) in subjects trained to physically execute or mentally perform a sequence of finger opposition movements. Before and after physical practice and motor-imagery practice, M1 excitability was evaluated by measuring the input-output (IO) curve of motor evoked potentials. M1 LTP and long-term depression (LTD)-like plasticity was assessed with paired-associative stimulation (PAS) of the median nerve and motor cortex using an interstimulus interval of 25 ms (PAS25) or 10 ms (PAS10), respectively. We found that even if after both practice sessions subjects significantly improved their movement speed, M1 excitability and plasticity were differentially influenced by the two practice sessions. First, we observed an increase in the slope of IO curve after physical but not after MI practice. Second, there was a reversal of the PAS25 effect from LTP-like plasticity to LTD-like plasticity following physical and MI practice. Third, LTD-like plasticity (PAS10 protocol) increased after physical practice, whilst it was occluded after MI practice. In conclusion, we demonstrated that MI practice lead to the development of neuroplasticity, as it affected the PAS25- and PAS10- induced plasticity in M1. These results, expanding the current knowledge on how MI training shapes M1 plasticity, might have a potential impact in rehabilitation.

Dynamic Shaping of the Defensive Peripersonal Space through Predictive Motor Mechanisms: When the "Near" Becomes "Far".

The hand blink reflex is a subcortical defensive response, known to dramatically increase when the stimulated hand is statically positioned inside the defensive peripersonal space (DPPS) of the face. Here, we tested in a group of healthy human subjects the hand blink reflex in dynamic conditions, investigating whether the direction of the hand movements (up-to/down-from the face) could modulate it. We found that, on equal hand position, the response enhancement was present only when the hand approached to (and not receded from) the DPPS of the face. This means that, when the hand is close to the face but the subject is planning to move the hand down, the predictive motor system can anticipate the consequence of the movement: the “near” becomes “far.” We found similar results both in passive movement condition, when only afferent (visual and proprioceptive) information can be used to estimate the final state of the system, and in motor imagery task, when only efferent (intentional) information is available to predict the consequences of the movement. All these findings provide evidence that the DPPS is dynamically shaped by predictive mechanisms run by the motor system and based on the integration of feedforward and sensory feedback signals. SIGNIFICANCE STATEMENT The defensive peripersonal space (DPPS) has a crucial role for survival, and its modulation is fundamental when we interact with the environment, as when we move our arms. Here, we focused on a defensive response, the hand blink reflex, known to increase when a static hand is stimulated inside the DPPS of the face. We tested the hand blink reflex in dynamic conditions (voluntary, passive, and imagined movements) and we found that, on equal hand position, the response enhancement was present only when the hand approached to (and not receded from) the DPPS of the face. This suggests that, through the integration of efferent and afferent signals, the safety boundary around the body is continuously shaped by the predictive motor system.

Spontaneous movement tempo can be influenced by combining action observation and somatosensory stimulation

Spontaneous movement tempo (SMT) was a popular field of study of the Gestalt psychologists It can be determined from subjects freely tapping out a rhythm with their finger, and it has been found to average about 2 Hz. A previous study showed that SMT changed after the observation of rhythmical movements performed at frequency different from the SMT. This effect was long-lasting only when movement execution immediately followed action observation (AO). We recently demonstrated that only when AO was combined with peripheral nerve stimulation (AO-PNS) was it possible to induce plastic changes in the excitability of the motor cortex, whereas AO and PNS alone did not evoke any changes. Here we investigated whether the observation of rhythmical actions at a frequency higher than the SMT combined with PNS induced lasting changes in SMT even in absence of immediate movement execution. Forty-eight participants were assigned to four groups. In AO-PNS group they observed a video showing a right hand performing a finger opposition movement sequence at 3 Hz and contemporarily received an electrical stimulation at the median nerve; in AO group and PNS group participants either observed the same video or received the same electrical stimulation of the AO-PNS group, respectively; in LANDSCAPE group subjects observed a neutral video. Participants performed a finger opposition movement sequence at spontaneous movement rate before and 30 min after the conditioning protocols. Results showed that SMT significantly changed only after AO-PNS. This result suggested that the AO-PNS protocol was able to induce lasting changes in SMT due to neuroplasticity mechanisms, indicating possible application of AO-PNS in rehabilitative treatments.

Action observation: mirroring across our spontaneous movement tempo

During action observation (AO), the activity of the “mirror system” is influenced by the viewer’s expertise in the observed action. A question that remains open is whether the temporal aspects of the subjective motor repertoire can influence the “mirror system” activation.

The tool as the last piece of the athlete's gesture imagery puzzle.

The present study tested whether and how motor experience with a specific tool affects motor representation of a specific movement. To this aim, we considered a group of expert tennis players and a control group of athletic individuals without tennis experience. Participants were asked to execute 20 single forehands into the wall with a tennis racket (movement execution - ME) and, afterward, to produce a kinesthetic image of themselves while executing the same movements (motor imagery - MI). During MI participants handled one of the following tools: a tennis racket, a tennis-like racket and an umbrella. Results showed that the duration of the real and the imagined movements were almost similar when participants of both groups held the tennis rackets. In contrast, when tennis players handled the tools not specific for tennis the duration of the imagined movements increased significantly compared to the MI duration with a tennis racket. On the opposite, the handled tool did not modulate MI performances of the control group. In conclusion, this study showed that motor representation of subjects who developed motor skills associated to tool-use is reliant on the object used to practice movements. This finding suggests that, although MI mainly relies on the activity of cortical motor regions, non-motor information - as the use of the tool to practice movement - strongly affects the MI performance.

Learning by observing: the effect of multiple sessions of action-observation training on the spontaneous movement tempo and motor resonance

ABSTRACT The present study was designed to explore the changes in motor performance and motor resonance after multiple sessions of action observation (AO) training. Subjects were exposed to the observation of a video showing finger tapping movements executed at 3 Hz, a frequency higher than the spontaneous one (2 Hz) for four consecutive days. Motor performance and motor resonance were tested before the AO training on the first day, and on the last day. Results showed that multiple sessions of AO training induced a shift of the speed of execution of finger tapping movements toward the observed one and a change in motor resonance. Before the 3 Hz‐AO training cortical excitability was highest during the observation of the 2 Hz video. This motor resonance effect was lost after one single session of 3 Hz‐AO training whereas after multiple sessions of 3 Hz‐AO training cortical excitability was highest during the observation of the 3 Hz video. Our study shows for the first time that multiple sessions of AO training are able not only to induce performance gains but also to change the way by which the observers motor system recognizes a certain movement as belonging to the individual motor repertoire. These results may encourage the development of novel rehabilitative protocols based on multiple sessions of action observation aimed to regain a correct movement when its spontaneous speed is modified by pathologies or to modify the innate temporal properties of certain movements. HIGHLIGHTSM1 excitability resonated with the spontaneous tempo (SMT) of the observed movement.Multiple sessions of action observation (AO‐training) induced SMT changes.After AO‐training M1 excitability resonated with the tempo of the trained motor act.This work adds new evidence on the neurophysiological basis of AO‐training.

Reach endpoint formation during the visuomotor planning of free arm pointing

Volitional motor control generally involves deciding ‘where to go’ and ‘how to go there’. Understanding how these two constituent pieces of motor decision coordinate is an important issue in neuroscience. Although the two processes could be intertwined, they are generally thought to occur in series, whereby visuomotor planning begins with the knowledge of a final hand position to attain. However, daily activities are often compatible with an infinity of final hand positions. The purpose of the present study was to test whether the reach endpoint (‘where’) is an input of arm motor planning (‘how’) in such ecological settings. To this end, we considered a free pointing task, namely arm pointing to a long horizontal line, and investigated the formation of the reach endpoint through eye–hand coordination. Although eye movement always preceded hand movement, our results showed that the saccade initiation was delayed by ~ 120 ms on average when the line was being pointed to as compared with a single target dot; the hand reaction time was identical in the two conditions. When the latency of saccade initiation was relatively brief, subjects often performed double, or even triple, saccades before hand movement onset. The number of saccades triggered was found to significantly increase as a function of the primary saccade latency and accuracy. These results suggest that knowledge about the reach endpoint built up gradually along with the arm motor planning process, and that the oculomotor system delayed the primary reach‐related saccade in order to gain more information about the final hand position.

Motor training and the combination of action observation and peripheral nerve stimulation reciprocally interfere with the plastic changes induced in primary motor cortex excitability

AO-PNS is a stimulation protocol combining action observation (AO) and peripheral nerve stimulation (PNS) to induce plasticity in the primary motor cortex (M1) (increased excitability). Another method to increase M1 excitability is motor training. The combination of two protocols, which individually induce long-term potentiation (LTP)-like plasticity in overlapping neural circuits, results in a transitory occlusion or reverse of this phenomenon. This study aimed to understand the neurophysiological mechanisms underlying AO-PNS by testing whether AO-PNS and motor training induced LTP-like plasticity in, at least partially, overlapping neural networks. One group of participants practiced a motor training (finger opposition movements) followed by AO-PNS, whereas another group performed the two protocols in reverse order. Motor performance was evaluated by means of a sensor-engineered glove and transcranial magnetic stimulation was used to assess M1 excitability before and after each conditioning protocol. Motor training increased movement frequency, suggesting the occurrence of motor learning in both groups. When applied on first, both motor training and AO-PNS significantly increased the motor-evoked potential (MEP), but occluded the increase of cortical excitability expected after the following protocol, leading to a significant decrease of MEP amplitude. These results suggest that motor training and AO-PNS act on partially overlapping neuronal networks, which include M1, and that AO-PNS might be able to induce LTP-like plasticity in a similar way to overt movement execution. This candidates AO-PNS as methodology potentially useful when planning rehabilitative interventions on patients who cannot voluntarily move.

The ITALK project : A developmental robotics approach to the study of individual, social, and linguistic learning

This article presents results from a multidisciplinary research project on the integration and transfer of language knowledge into robots as an empirical paradigm for the study of language development in both humans and humanoid robots. Within the framework of human linguistic and cognitive development, we focus on how three central types of learning interact and co-develop: individual learning about ones own embodiment and the environment, social learning (learning from others), and learning of linguistic capability. Our primary concern is how these capabilities can scaffold each others development in a continuous feedback cycle as their interactions yield increasingly sophisticated competencies in the agents capacity to interact with others and manipulate its world. Experimental results are summarized in relation to milestones in human linguistic and cognitive development and show that the mutual scaffolding of social learning, individual learning, and linguistic capabilities creates the context, conditions, and requisites for learning in each domain. Challenges and insights identified as a result of this research program are discussed with regard to possible and actual contributions to cognitive science and language ontogeny. In conclusion, directions for future work are suggested that continue to develop this approach toward an integrated framework for understanding these mutually scaffolding processes as a basis for language development in humans and robots.