Elisabetta Adami

‘We/YouTube’: exploring sign-making in video-interaction

On the video-sharing website YouTube, the ‘video response’ option triggers a new interaction practice, i.e. communication threads started by an initial video, built up by video responses and resumed by a video-summary. This article examines a video-thread that starts from one of YouTube’s ‘most responded’ videos; by using a social semiotic multimodal analysis, the author investigates how video responses relate to the initial video and how the video-summary selectively transforms the resources of the responses while presenting itself as a resume of the video-thread. This analysis helps to explore the notion of ‘interest’, which shapes sign-making in a chain of semiosis in video-interaction, thus creating an approach to communication in which traditional notions of coherence and relevance are reshaped in terms of an interest-driven prompt—response relation.

What’s in a click? A social semiotic framework for the multimodal analysis of website interactivity

This article presents a social semiotic framework for the multimodal analysis of website interactivity. Distinguishing it from interaction, it defines interactivity as the affordance of a text of being acted (up)on, thus including hypertextuality. The author introduces the notion of ‘interactive sites/signs’ as the loci of interactivity in digital texts; these have a two-fold nature and a two-dimensional functioning. In their two-fold nature, they are both places enabling actions producing effects and forms endowed with meanings. Notwithstanding the non-direct correspondence between forms, actions and effects (which makes any specific association between the three significant within a webpage design), and in spite of their many possible forms (encompassing still and dynamic images, shapes and writing), a small range of actions can activate them (click/click+type/hover), producing a restricted set of textual effects (access/provide/transfer text). In their two-dimensional functioning, interactive sites/signs function both syntagmatically, on the page where they are displayed, in their relation with other co-occurring elements, and paradigmatically, opening to optional text realizations, hence in their relation with these. The framework adapts Halliday’s three metafunctions to the analysis of the two-fold nature and two-dimensional functioning of interactive sites/signs. It provides a fine-grained account of the interactive meaning potentials of digital texts, distinguishing between a text’s aesthetics of interactivity – as visually communicated before it is activated, performed and experienced – and its functionality, in the configuration of interactive possibilities offered by a page. Designed to complement the extant practices of text analysis of webpages, the framework can be used comparatively, as exemplified in its application to the analysis of two blog pages, and can provide a more refined assessment of the interactive meaning potential of a webpage than traditional methodologies such as content analysis.

Special Issue: Social media and the visual

Social media are a significant part of contemporary communication. It is estimated that, by the end of 2016, over 2 billion people worldwide will be using social media platforms such as Facebook and Twitter, to communicate, interact, and undertake a range of formal and informal activities and practices (Mccarthy et al., 2014). It is therefore vital that we understand social media platforms and their usage. With a shared interest in the social significance of visual communication in social media (and beyond), the five articles in this special issue contribute to a small body of empirical research on visual meaning making resources in the context of social media environments, including blogs, Tumblr and Instagram. Existing work on social media within the field of multimodal studies includes genre analysis of Facebook publishing (Eisenlauer, 2010) and text compositional practices on Facebook (Bezemer and Kress, 2014), children’s blogging (Abas, 2011), the aesthetic meaning potential of web design (Adami, 2015), the expression of style in Tumblr and Pinterest (Jewitt and Henriksen, 2017, forthcoming) and identity work through repurposing resources (Leppänen et al., 2013), as well as trans-media production across different social media (Adami, 2014). Work focused more specifically on image includes Seko’s (2013) multimodal analysis of self-injury photographs on Flikr, Aiello and Woodhouse’s (2016) multimodal critical discourse analysis of gendered identity in stock photography and Manovich’s (2016a) latest work on style in Instagram photos. 644153 VCJ0010.1177/1470357216644153Visual CommunicationAdami and Jewitt research-article2016

Styling the self online: semiotic technologization in weblog publishing

ABSTRACT Testing on digital semiotic production the concepts of (self-)styling and technologization of discourse, developed for offline linguistic phenomena, the article investigates the role of digital platforms in shaping the relation between self-expression online, semiotic regulation and the social construction of taste. By focusing on the use of semiotic resources of webdesign for identity expression, the study analyses the semiotic regimes emerging from regulatory practices and webdesign styles on WordPress, and their influence on the changes in the projected identity of a personal blogger. In spite of the participatory character of WordPress, results show the role of the platform in objectifying/technologizing hegemonic semiotic preferences, with consequent normalising effects in bloggers’ self-styling practices. The conclusions relate the findings to broader power dynamics in the social construction of taste and their implications for both online and offline forms of self-expression.

Special Issue: Analysing Digitized Visual Culture

T h e D I G I T A l A G e As is well known, Communication and media studies have been challenged by the advent and consolidation of the digital age. The digital revolution with its continuing advances in technologies and innovations has greatly increased the resources and modes that are available for production, distribution and reception of communication in various forms. Whether it is the replacement of analog photography and photochemical films by arrays of photo sensors and computer software, their constant availability on phones and other mobile devices to document our daily life or the evolving new channels of communication on the internet, including the expansion of social media both for private and commercial use, we are surrounded by digitized media in which digitized photos and other (audio)visuals play a major role. In some communities, in fact, multimodal artefacts and performances composed of various representations have become the dominant way of 770584 VCJ Visual CommunicationWildfeuer et al.

Visual communication: mobilizing perspectives:

Since the launch of Visual Communication in 2002, much has changed in the lived experience of contemporary communication practices, as well as in the academic study of it. The founding editors of the journal – Carey Jewitt, Theo van Leeuwen, Ron Scollon and Teal Triggs – set themselves the ambitious task of creating a forum for introducing the visual as a (somewhat) new focus for research and critique. As they said at the time:

Shaping Public Spaces from Below: The Vernacular Semiotics of Leeds Kirkgate Market

ABSTRACT The paper presents a social semiotic approach to vernacular sign-making in place, by examining the visual landscape of Leeds Kirkgate Market, as an example of a semiotically-unregulated place. Traders have ample freedom of self-expression and agency in shaping their stalls through all visual-material resources (beyond mere signage, as analysed in linguistic landscape research). There derives a composite non-cohesive aesthetic of the markets visual landscape, driven by the situated needs and tastes of the socio-culturally diverse population inhabiting the place. The resulting semiotic diversity is remarkable when compared to its neighbouring areas, as well as to recently renovated UK city markets. Against an increased institutional regulation of urban landscapes, semiotically-unregulated places enable vernacular practices from below that defy current aesthetic tenets in professional design and help revealing and questioning the power dynamics underlying a social construction of taste. Research into vernacular semiotics shows the enhanced cultural richness resulting from peoples agency in shaping the public spaces in which they live.