David Geiger

Personalized Task Recommendation in the Field

This chapter evaluates the potential of the developed prototype to realize and study personalized task recommendation on existing crowdsourcing platforms and thereby improve the match between contributors and available tasks. To this intent, the following sections present a sequence of three studies that were conducted along the process of field testing and deploying the metacrowd service on the Mechanical Turk platform. Section 5.1 first describes a pilot study that gathered initial feedback from a small group of experienced contributors. Section 5.2 then lays out the details of a large-scale survey on general task search behavior and on the perceived utility of the metacrowd service. Section 5.3, finally, analyzes a dataset that was gathered during an extended period of productive use and discusses the challenges identified during the evaluation.

Current State of Personalized Task Recommendation

In order to build a foundation for the design of mechanisms that assist individuals in finding suitable tasks, this chapter describes a review of the related academic literature. By channeling any existing efforts into a systematic perspective, this review generates a comprehensive picture of the current state of personalized task recommendation and provides insights that exceed the boundaries of individual crowdsourcing systems.

Design of a Third-Party Task Recommendation Service

Building on the insights that were gained during the literature review, this chapter describes the prototypical design of a third-party task recommendation service called metacrowd, which integrates with existing crowdsourcing systems that do not feature built-in recommendation mechanisms. In line with the identified need for large-scale field research, metacrowd provides a platform for studies in real-world, productive contexts. The setup permits complete control over the recommender system design and, consequently, facilitates the extensive collection of related research data and the systematic study of specific design aspects.