Jim Laredo

An Assessment of Intrinsic and Extrinsic Motivation on Task Performance in Crowdsourcing Markets

Crowdsourced labor markets represent a powerful new paradigm for accomplishing work. Understanding the motivating factors that lead to high quality work could have significant benefits. However, researchers have so far found that motivating factors such as increased monetary reward generally increase workers’ willingness to accept a task or the speed at which a task is completed, but do not improve the quality of the work. We hypothesize that factors that increase the intrinsic motivation of a task – such as framing a task as helping others – may succeed in improving output quality where extrinsic motivators such as increased pay do not. In this paper we present an experiment testing this hypothesis along with a novel experimental design that enables controlled experimentation with intrinsic and extrinsic motivators in Amazon’s Mechanical Turk, a popular crowdsourcing task market. Results suggest that intrinsic motivation can indeed improve the quality of workers’ output, confirming our hypothesis. Furthermore, we find a synergistic interaction between intrinsic and extrinsic motivators that runs contrary to previous literature suggesting “crowding out” effects. Our results have significant practical and theoretical implications for crowd work.

CrisisTracker: crowdsourced social media curation for disaster awareness

Victims, volunteers, and relief organizations are increasingly using social media to report and act on large-scale events, as witnessed in the extensive coverage of the 2010-2012 Arab Spring uprisings and 2011 Japanese tsunami and nuclear disasters. Twitter® feeds consist of short messages, often in a nonstandard local language, requiring novel techniques to extract relevant situation awareness data. Existing approaches to mining social media are aimed at searching for specific information, or identifying aggregate trends, rather than providing narratives. We present CrisisTracker, an online system that in real time efficiently captures distributed situation awareness reports based on social media activity during large-scale events, such as natural disasters. CrisisTracker automatically tracks sets of keywords on Twitter and constructs stories by clustering related tweets on the basis of their lexical similarity. It integrates crowdsourcing techniques, enabling users to verify and analyze stories. We report our experiences from an 8-day CrisisTracker pilot deployment during 2012 focused on the Syrian civil war, which processed, on average, 446,000 tweets daily and reduced them to consumable stories through analytics and crowdsourcing. We discuss the effectiveness of CrisisTracker based on the usage and feedback from 48 domain experts and volunteer curators.

Peoplecloud for the globally integrated enterprise

Crowdsourcing has emerged as the new on-line distributed production model in which people collaborate and may be awarded to complete a task. While many existing services enable enterprises to employ the wisdom of crowd, there is no existing practice defined for integration of crowdsourcing with the business processes. We propose PeopleCloud, as the (1) mechanism to enable access to scalable workforce on-line, connecting it to the enterprise and (2) an interface to services required for crowdsourcing tasks. We define requirements for PeopleCloud, based on our experiences in employing wisdom of crowd to source business and IT information within the enterprise.

Challenges and experiences in deploying enterprise crowdsourcing service

The value of crowdsourcing, arising from an instant access to a scalable expert network on-line, has been demonstrated by many success stories, such as GoldCorp, Netflix, and TopCoder. For enterprises, crowdsourcing promises significant cost-savings, quicker task completion times, and formation of expert communities (both within and outside the enterprise). Many aspects of the vision of enterprise crowdsourcing are under vigorous refinement. The reasons for this lack of progress, beyond the isolated and purpose-specific crowdsourcing efforts, are manifold. In this paper, we present our experience in deploying an enterprise crowdsourcing service in the IT Inventory Management domain. We focus on the technical and sociological challenges of creating enterprise crowdsourcing service that are generalpurpose, and that extend beyond mere specific-purpose, run-once prototypes. Such systems are deployed to the extent that they become an integrated part of business processes. Only when such degree of integration is achieved, the enteprises can fully adopt crowdsourcing and reap its benefits. We discuss the challenges in creating and deploying the enterprise crowdsourcing platform, and articulate current technical, governance and sociological issues towards defining a research agenda.

PeopleCloud Service for Enterprise Crowdsourcing

Web 2.0 provides the technological foundations upon which the crowdsourcing paradigm evolves and operates, enabling enterprises, universities and eGovernments, to access scalable networks of knowledge experts on-line. However, there is no existing practice allowing for coordination of crowdsourcing tasks, and their integration with existing business processes and embedding these services into the Web fabric. In this paper, we examine two applications of enterprise crowdsourcing service in the domain of IT Service Delivery: 1) IT Inventory Management and 2) End-User Support. We illustrate how a) expert discovery mechanism, b) virtual team building capabilities, c) task management and d) provisioning of task-based services, enable enterprises to effectively build knowledge networks, which are able to execute complex and transformative knowledge-intensive tasks. Finally, based on the application analysis, we propose PeopleCloud, an on-demand service system, which spawns and manages scalable virtual teams of knowledge workers by either (1) building on the wisdom of crowds within an enterprise or across a value chain or (2) creating a marketplace for accessing specialists on-line.

Towards Real-time Emergency Response using Crowd Supported Analysis of Social Media

This position paper outlines an ongoing research project that aims to incorporate crowdsourcing as part of an emergency response system. The proposed systems novelty is that it integrates crowdsourcing into its architecture to analyze and structure social media content posted by microbloggers and service users, including emergency response coordinators and victims, during the event or disaster. An important challenge in this approach is identifying appropriate tasks to crowdsource, and adopting effective motivation strategies. Author Keywords Emergency response, social media, crowdsourcing, text mining.

A Graph-Based Data Model for API Ecosystem Insights

APIs are increasingly important for companies to enable partners and consumers to access their services and resources. API ecosystems deal with related challenges like publication, promotion and provision of APIs by providers and identification, selection and consumption of APIs by consumers. To address these challenges, to match consumers with relevant APIs, and to support API providers and thus ultimately the ecosystem to evolve, API ecosystems rely on information about APIs, their usage and characteristics, and the social environment around them. We present an extensible, graph-based data model to capture the entities in an API ecosystem and their relations. The data model includes temporal information to capture the evolution of API ecosystems. Analysis operations on top of the data model provide insights for consumers, providers and the ecosystem provider to address the introduced challenges. We present a system implementing the conceptualized data model. We integrate this system with an API ecosystem used in the context of a hackathon event to continuously collect data. We furthermore show the data models capabilities to represent a well-known dataset about ProgrammableWeb and to drive analysis operations on both datasets.

Continuous Improvement through Iterative Development in a Multi-Geography

With a very short time frame in mind we were commissioned to build a service solution that encompassed over 430 requirements. Given expertise, skills, time and budget constraints we had to search for resources around the world and assembled a team of more than 40 people in 7 locations across 6 different time zones. We chose state of the art architecture and development paradigms such as Service Oriented Architecture (SOA) and iterative development to facilitate the service capabilities between components and teams and to continuously refine our approach from iteration to iteration. In this paper we identify some of the challenges we faced and describe how we addressed them in subsequent iterations. The aggregate of our improvements constitutes a set of best practices that we recommend for future engagements of this type and suggestions for new tooling to support these activities.

The Future of Service Marketplaces in the Cloud

For as long as there have been services there has been a desire to have a convenient medium to expose and discover service offerings. Since early on, various efforts have attempted various approaches at the exchange of computational services, prompting the question of whether there is a market for Web services. We believe that a services marketplace should fulfill the promise of an electronic emporium where third party service providers are able to offer their services in a ubiquitous ecosystem, and where service consumers are able to acquire service solutions that are tailored to their requirements. This paper explores the landscape of cloud services marketplaces, where we are, what enablers are needed to realize the vision, and it presents a prospective architecture to that end.

Distributed Cross-Domain Change Management

Distributed systems increasingly span organizational boundaries and, with this, system and service management domains. Web services are the primary means of exposing services to clients, be it in electronic commerce, Software-as-a-Service (SaaS) or on cloud platforms and are being used and integrated with customer-managed applications as well as in complex mashups. Maturing cross-domain relationships and an increase in loose coupling and ad-hocness makes managing configuration changes, e.g., changes in interfaces or endpoints, increasingly relevant. Traditional service management processes within organizations, in particular change management, relies on a central configuration management database (CMDB) to assess the impact a change has on other components of the system. However, this approach does not work in a cross-domain environment, due to the lack of a central CMDB, centralized management processes, and knowledge by service providers which clients depends on their respective services. This paper proposes the Change 2.0 approach to cross-domain change management based on an inversion of responsibility for impact assessment and the facilitation of cross-domain service process integration. We present the requirements imposed by cross-domain change management, the Change 2.0 architecture, and a brief evaluation of its benefits.