Bradley A. Smith

Analysing multimodality in an interactive digital environment: software as a meta-semiotic tool

The present paper discusses issues arising from the use of an interactive digital software tool to analyse multimodal communication. The focus is on the ways in which such technical resources and associated techniques enable the analyst of social semiosis to apply different types of analysis, and provide the site for critical reflection upon the results of such analyses. The aim is to present an argument for the use of an interactive digital software application as a meta-semiotic tool. Three major challenges for scholars engaged in multimodal social semiotics are addressed in pursuing this aim, specifically with reference to the development of digital interactive analytical resources. Firstly, scholars are faced with the task of not only accounting for an increasing range of semiotic resources, but also for the way different semiotic phenomena interact to produce meaning. Secondly, the multimodal semiotician has a broad range of analytical approaches potentially relevant to any task to draw upon. Thirdly, the nature of contemporary media forms offer challenges themselves in terms of access, analysis and presentation of analysis.

Developing Novel Multimodal and Linguistic Annotation Software

In this paper we present a collaborative work between computer and social scientists resulting in the development of annotation software for conducting research and analysis in social semiotics in both multimodal and linguistic aspects. The paper describes the proposed software and discusses how this tool can contribute for development of social semiotic theory and practice.