Joel Ross

Who are the crowdworkers?: shifting demographics in mechanical turk

Amazon Mechanical Turk (MTurk) is a crowdsourcing system in which tasks are distributed to a population of thousands of anonymous workers for completion. This system is increasingly popular with researchers and developers. Here we extend previous studies of the demographics and usage behaviors of MTurk workers. We describe how the worker population has changed over time, shifting from a primarily moderate-income, U.S.-based workforce towards an increasingly international group with a significant population of young, well-educated Indian workers. This change in population points to how workers may treat Turking as a full-time job, which they rely on to make ends meet.

Ethics and tactics of professional crowdwork

Paid crowd workers are not just an API call---but all too often, they are treated like one.

Massively distributed authorship of academic papers

Wiki-like or crowdsourcing models of collaboration can provide a number of benefits to academic work. These techniques may engage expertise from different disciplines, and potentially increase productivity. This paper presents a model of massively distributed collaborative authorship of academic papers. This model, developed by a collective of thirty authors, identifies key tools and techniques that would be necessary or useful to the writing process. The process of collaboratively writing this paper was used to discover, negotiate, and document issues in massively authored scholarship. Our work provides the first extensive discussion of the experiential aspects of large-scale collaborative research.

Collaborative filtering and carbon footprint calculation

An increasingly common way for people to measure and understand their individual environmental impact is by using an online carbon footprint calculator. Although a wide variety of these calculators are available, the vast majority shares the same form of user interaction and calculation methods, limiting the reach of these tools and introducing a number of problems for informing users of their environmental impact. We find that the interaction afforded by popular calculators—in addition to considering only a limited number of impact factors at a single instant in time—treats a persons carbon footprint as an individual matter, rather than drawing attention to the interconnectivity and wider impacts of footprints across the persons broader community. To address these problems, we present the Better Carbon calculator, which uses collaborative filtering and location-based calculation to provide an individual footprint estimate while simultaneously affecting and improving the estimates for other people in a users community.

How games can redirect humanity's cognitive surplus for social good

Permission to make digital or hard copies of part or all of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies show this notice on the first page or initial screen of a display along with the full citation. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, to republish, to post on servers, to redistribute to lists, or to use any component of this work in other works requires prior specific permission and/or a fee. Permissions may be requested from Publications Dept., ACM, Inc., 2 Penn Plaza, Suite 701, New York, NY 10121-0701 USA, fax +1 (212) 869-0481, or permissions@acm.org. ©2010 ACM 1544-3574/2010/12-ART25

Richly connected systems and multi-device worlds

10.00 DOI: 10.1145/1921141.1921145 http://doi.acm.org/10.1145/1921141.1921145

Media richness, interactivity and retargeting to mobile devices: a survey

Many human activities now take place in settings that include several computational devicessuch as desktop computers, laptops, and mobile phonesin the same physical space. However, we lack interaction paradigms that support a coherent experience across these collocated technologies and enable them to work effectively as systems. This article presents a conceptual framework for building richly connected systems of collocated devices, and offers two implemented examples of interactive virtual worlds built on this framework. Aspects of this framework include multiple channels of real and apparent connectivity among devices: for example, multiple kinds of data networking, cross-device graphics and sound, and embodied mobile agents that inhabit the multi-device system. In addition, integration of the system with the physical world helps bridge the gap between devices. We evaluate the framework in terms of the types of user experiences afforded and enabled by the implemented systems. We also present a number of lessons learned from this evaluation regarding how to develop richly connected systems using heterogeneous devices, as well as the expectations that users bring to this kind of system. The core contribution of this paper is a novel framework for collocated multi-device systems; by presenting this framework, this paper lays the groundwork for a wide range of potential applications.

Pervasive Human Computing

Mobile devices, such as mobile phones, are becoming more ubiquitous and gaining more capabilities, leading to their increasing use as portable media players. However, the great majority of produced visual media is targeted towards devices with larger displays and greater hardware capabilities, making it difficult for these media to be presented on mobile devices. These difficulties will only increase as multimedia becomes more and more interactive, such as with increasingly popular mobile games. In this paper, we survey past methods for retargeting visual media to mobile devices, considering how such methods have been applied to text, still images and video content. We identify a shift from interactive retargeting techniques towards more automated methods as media richness increases. We discuss implications of this trend for the retargeting of 3D games and virtual worlds to mobile devices, particularly the need to focus on the retargeting of experience rather than solely visual content.

Who are the Turkers? Worker Demographics in Amazon Mechanical Turk

Systems involving human computation often rely on computation being distributed spatially and temporally, enabling large-scale human-driven information processing. This distribution suggests that human computation systems may be effectively supported with pervasive computing technologies, which aim to invisibly embed networked computation in everyday life. In this chapter, I consider previous work at the intersection of human computing and pervasive computing, focusing on how human computation has been deployed through mobile platforms and how localized humans can act as computer-based sensors. I suggest a number of questions for guiding future research, framed around the question of: “in what ways does the situatedness enabled by pervasive systems influence human computation?” In addition, I discuss how the pervasive computing lens of “seamless” interaction highlights issues in human computation systems of rendering both computers and computation users invisible; this lens suggests further considerations in developing pervasive human computation systems.