Tuomo Hiippala

The Multimodality of Digital Longform Journalism

Digital longform journalism has recently attracted increased attention among both academics and professionals. This study contributes to the growing body of research by dissecting the multimodal structure of digital longform journalism, that is, how the emerging genre combines written language, photography, short videos, maps and other graphical elements, and joins them together into a seamless narrative using subtle transitions. The data consist of 12 longform articles published in 2012–2013, which have been annotated for their visual and verbal content, their underlying principle of organization and the transitions that hold between them. The annotation is stored into a digital corpus, which is then analyzed to examine the multimodal structures that enable the longform genre to establish a narrative, and to explicate how the longform attempts to captivate its audience by creating a distraction-free environment.

The interface between rhetoric and layout in multimodal artefacts

This research-in-progress report describes ongoing work on a doctoral dissertation, which attempts to model the prototypical structure of the tourist brochure as a multimodal artefact. By using a multimodal corpus based on the Genre and Multimodality model, the dissertation investigates how the brochures use both language and image to fulfil their communicative function. This paper focuses on a specific aspect of the prototypical structure, that is, how the brochures organize the content in the layout and signal its

The localisation of advertising print media as a multimodal process

The concept of localisation is typically associated with the field of information technology and software development, and involves describing the process of translating a product and its documentation, verifying the translation, and accounting for any cultural factors that might be related to the use of the product. Localisation has emerged as a result of globalised markets and business, where products are first developed with international markets in mind, and then adapted to country specific markets where they are intended to be sold. Esselink (2000, p. 3) quotes a definition from the Localisation Industry Standards Association (LISA), which describes the process of localisation and introduces the concept of a locale:

Machine learning for tracking illegal wildlife trade on social media

To the Editor — Illegal trade in wildlife is booming on e-commerce platforms1, the ‘dark web’2 and social media1,3. The ease of access and high number of users of social media make it a particularly concerning venue1,3,4. Wildlife dealers use social media to release photos and information about products to attract customers and to market their products to networks of contacts. We currently lack the tools for automated monitoring of high-volume data that are needed to investigate and prevent this illegal trade, but machine-learning algorithms offer a way forward. Operating within the broader field of artificial intelligence, the concept of machine learning refers to algorithms that learn from data without human guidance. Deep-learning algorithms5 are a family of these algorithms that are highly successful in classifying image contents and locating individual objects within them, and in processing natural language. Applying these techniques to social media data allows human behaviour to be investigated on an unprecedented scale. Yet these techniques and data sources are still rarely used to address drivers of the biodiversity crisis6. Many social media platforms provide an application programming interface that allows access to user-generated text, images and videos, as well as to accompanying metadata, such as where and when the content was uploaded, and connections between users. Processing such data manually is inefficient and time consuming, but machine-learning algorithms can be trained to filter this content to identify relevant information (see Fig. 1 for an example). These algorithms can be trained to detect which species or wildlife products, such as horns or scales, appear in an image or video, while also classifying their setting, such as a natural habitat or a marketplace. When processing video, algorithms can use audio clues, such as identifying bird species by their songs and calls, as well as interrogating the image stream. Natural language processing can be used to infer the meaning of a verbal description, for example, whether an animal or plant is for sale or observed in nature, and to classify the sentiment and preferences of social media users. To unlock this potential, social media platforms must be engaged to share their data and actively collaborate in the development of real-time monitoring tools that can be used to automatically identify content pertaining to the illegal wildlife trade, and to report this content to enforcers. Furthermore, machine-learning algorithms need humanverified training data. Such training datasets may be generated through crowd-sourcing initiatives, and collaborations between scientists and enforcers may further improve the algorithms’ performance. Together with advances in artificial intelligence that will refine the algorithms themselves, such efforts

A framework for investigating illegal wildlife trade on social media with machine learning

Abstract Article impact statement: Machine learning can be used to automatically monitor and assess illegal wildlife trade on social media platforms.

Recognizing military vehicles in social media images using deep learning

This paper presents a system that uses machine learning to recognize military vehicles in social media images. To do so, the system draws on recent advances in applying deep neural networks to computer vision tasks, while also making extensive use of openly available libraries, models and data. Training a vehicle recognition system over three classes, the paper reports on two experiments that use different architectures and strategies to overcome the challenges of working with limited training data: data augmentation and transfer learning. The results show that transfer learning outperforms data augmentation, achieving an average accuracy of 95.18% using 10-fold cross-validation, while also generalizing well on a separate testing set consisting of social media content.