A sad thumbs up: incongruent gestures and disrupted sensorimotor activity both slow processing of facial expressions

Abstract

ABSTRACT Recognising a facial expression is more difficult when the expresser s body conveys incongruent affect. Existing research has documented such interference for universally recognisable bodily expressions. However, it remains unknown whether learned, conventional gestures can interfere with facial expression processing. Study 1 participants (N\u2009=\u200962) viewed videos of people simultaneously producing facial expressions and hand gestures and reported the valence of either the face or hand. Responses were slower and less accurate when the face-hand pairing was incongruent compared to congruent. We hypothesised that hand gestures might exert an even stronger influence on facial expression processing when other routes to understanding the meaning of a facial expression, such as with sensorimotor simulation, are disrupted. Participants in Study 2 (N\u2009=\u2009127) completed the same task, but the facial mobility of some participants was restricted, which disrupted face processing in prior work. The hand-face congruency effect from Study 1 was replicated. The facial mobility manipulation affected males only, and it did not moderate the congruency effect. The present work suggests the affective meaning of conventional gestures is processed automatically and can interfere with face perception, but does not suggest that perceivers rely more on gestures when sensorimotor face processing is disrupted.