Karina Johansson

Cognition as a Bridge Between Signal and Dialogue: Communication in the Hearing Impaired and Deaf

This paper is an overview of the research we have carried out at Linköping University on cognitive aspects of visual language processing and related communicative forms. In the first section, a cognitive individual difference perspective on speechreading is introduced. In the second, compensatory perceptual mechanisms are discussed on the basis of behavioral and neurophysiological data. The third section addresses further the issue of perceptual versus cognitive compensation, and the fourth and fifth sections apply a set of cognitive tests based on the concept of working memory for poorly specified language input to data on the successes and failures with cochlear implants, tactile aids and hearing aids.

The role of facial approach signals in speechreading

Facial approach signals in speechreading were studied in three experiments. Experiment 1 investigated effects on speechreading performance of different emotional content (sad and happy) in sentences combined with facial expression. The experiment also investigated the effect of script information on speechreading of emotionally charged sentences. In Experiment 2 the effect of emotional sentences on speechreading performance was investigated when no or very limited script information was given. Happy sentences were superior to sad sentences, except in the sad script where no difference between sad and happy was found. Experient 3 investigated the effect of articulation (overarticulation vs. underarticulation) on speechreading of emotional sentences. Happy sentences were again generally superior to sad sentences. As happy sentences were superior to sad sentences both when overarticulated and underarticulated the superiority of the happy sentences was explained in light of the approach hypothesis.