Mark Evan Nelson

Locating the Semiotic Power of Multimodality

This article reports research that attempts to characterize what is powerful about digital multimodal texts. Building from recent theoretical work on understanding the workings and implications of multimodal communication, the authors call for a continuing empirical investigation into the roles that digital multimodal texts play in real-world contexts, and they offer one example of how such investigations might be approached. Drawing on data from the practice of multimedia digital storytelling, specifically a piece titled “Lyfe-N-Rhyme,” created by Oakland, California, artist Randy Young (accessible at http://www.oaklanddusty.org/videos.php), the authors detail the method and results of a fine-grained multimodal analysis, revealing semiotic relationships between and among different, copresent modes. It is in these relationships, the authors argue, that the expressive power of multimodality resides.

Challenges of Multimedia Self-Presentation Taking, and Mistaking, the Show on the Road

One privilege enjoyed by new-media authors is the opportunity to realize representations of Self that are rich textual worlds in themselves and also to engage the wider world, with a voice, a smile, imagery, and sound. Still, closer investigation of multimedia composition practices reveals levels of complexity with which the verbal virtuoso is unconcerned. This article argues that while technology-afforded multimedia tools make it comparatively easy to author a vivid text, it is a multiplicatively more complicated matter to vividly realize and publicize an authorial intention. Based on analysis of the digital story creation process of a youth named “Steven,” the authors attempt to demonstrate the operation of two forces upon which the successful multimodal realization of the authors intention may hinge: “fixity” and “fluidity.” The authors show how, within the process of digital self-representation, these forces can intersect to influence multimodal meaning making, and an authors life, in consequential ways.

Literacy, Media, and Morality: Making the Case for an Aesthetic Turn

Media scholar Roger Silverstone (2007) opens his provocative meditation on the role of media in a global world by recounting a story that we find powerfully emblematic. It is emblematic of his book, which theorizes the potential of media to constitute a moral public space, and of our chapter, which proposes aesthetics, or one’s sense of what is beautiful or right,1 as an organizing principle for future Literacy Studies. Silverstone recounts a brief interview that was broadcast on BBC Radio in the midst of the US war in Afghanistan not long after 9/11 and the World Trade Center attack. This interview featured an Afghani blacksmith, who had his own take on why so many bombs were falling on his village. It was because, his translated voice proposed, ‘Al Qaeda had killed many Americans and their donkeys and had destroyed some of their castles’ (p. 1). What interested Silverstone about the blacksmith’s account was that, for a brief moment, it reversed the ‘customary polarities of interpretation’ (p. 4) ‘in which we in the West do the defining, and in which you are, and I am not, the other’ (p. 3). Silverstone believes that the quintessential characteristic of media in our global and digital world is its potential to link strangers to each other, across geographic, social, and historical space. Indeed, he argued that the images of strangers, mediated by television, computers, cell phones, radio, and the like, largely constitute our understanding of the world.

Youth Breaking New “Ground”: Iconicity and Meaning Making in Social Media

This chapter argues for a semiotic approach to understanding and productively facilitating the interaction and learning that young people do via social media. As an organizing analogy for what follows, we first call forth the enduring Indian parable of The Blind Men and the Elephant. This story, many will know, describes the initial encounter between six unsighted sages and a pachyderm, an exotic creature of which none of the men have any prior knowledge. The essential premise is that each of the men differently identifies the beast, based only upon his particular tactile experience of it. Touching the elephant’s flank, one man explains it as “very like a wall,” as nineteenth-century poet John Godfrey Saxe’s version has it. The man who feels the trunk recognizes the elephant’s snake-like quality; a tusk is perceived to be a “spear”; and so forth. The important lessons, for present purposes, are that 1) what an unknown something seems like can fundamentally shape a person’s interpretation of what that thing itself is, and that 2) a thing and that which it resembles are really more subjectively and partially related than one might otherwise believe. These are ideas that resonate closely with C. S. Peirce’s (1935, 1940/1955) influential theory of “iconicity” in human meaning making, which this chapter engages and operationalizes as a critical lens through which to look afresh at learning, literacies, and global digital media.