Elizabeth S. Redden

Comparing the effects of visual-auditory and visual-tactile feedback on user performance: a meta-analysis

In a meta-analysis of 43 studies, we examined the effects of multimodal feedback on user performance, comparing visual-auditory and visual-tactile feedback to visual feedback alone. Results indicate that adding an additional modality to visual feedback improves performance overall. Both visual-auditory feedback and visual-tactile feedback provided advantages in reducing reaction times and improving performance scores, but were not effective in reducing error rates. Effects are moderated by task type, workload, and number of tasks. Visual-auditory feedback is most effective when a single task is being performed (g = .87), and under normal workload conditions (g = .71). Visual-tactile feedback is more effective when multiple tasks are begin performed (g = .77) and workload conditions are high (g = .84). Both types of multimodal feedback are effective for target acquisition tasks; but vary in effectiveness for other task types. Implications for practice and research are discussed.

The benefits of multimodal information: a meta-analysis comparing visual and visual-tactile feedback

Information display systems have become increasingly complex and more difficult for human cognition to process effectively. Based upon Wickens Multiple Resource Theory (MRT), information delivered using multiple modalities (i.e., visual and tactile) could be more effective than communicating the same information through a single modality. The purpose of this meta-analysis is to compare user effectiveness when using visual-tactile task feedback (a multimodality) to using only visual task feedback (a single modality). Results indicate that using visual-tactile feedback enhances task effectiveness more so than visual feedback (g = .38). When assessing different criteria, visual-tactile feedback is particularly effective at reducing reaction time (g = .631) and increasing performance (g = .618). Follow up moderator analyses indicate that visual-tactile feedback is more effective when workload is high (g = .844) and multiple tasks are being performed (g = .767). Implications of results are discussed in the paper.

Tactile Cuing to Augment Multisensory Human–Machine Interaction.

Tactile displays promise to improve the information-processing capacity of operators, especially when used in conjunction with visual and auditory displays. In this article, we describe current applications and future directions in tactile cuing.

A Tool for the Accumulation and Evaluation of Multimodal Research

A surge of interest exists in multimodal research and interfaces. This is due, at least in part, to an exponential increase in the amount and type of information that can be presented to a user. When a great deal of information is presented via a single sensory modality, it can exceed the operators capacity to manage the information efficiently, generating cognitive overload. As a consequence, the users performance becomes susceptible to slower response times, loss of situational awareness, faulty decision making, and execution errors. Researchers and designers have responded to these issues with the development and application of multimodal information displays. The cross-disciplinary flavor of multimodal applications presents a challenge to the accumulation, evaluation, and dissemination of relevant research. We describe the development of a taxonomy for the evaluation and comparison of multimodal display research studies, and the implementation of the taxonomy into a database: the Multimodal Query System (MQueS).