Robert Kern

Managing Quality of Human-Based eServices

Modern business strategies consider Web-based outsourcing of microjobs to the masses. Respective business activities are however difficult to manage. Traditional approaches of covering human tasks in business processes build on assumptions of limited process scale and closed organizational models that are not valid in crowdsourcing scenarios. Web services have been proposed as a means to represent human tasks that allow leveraging interaction, brokerage and composition capabilities of SOC for human interaction management. In this paper we argue that crowdsourcing requires considering qualitative constraints and sketch a platform for managing the quality of human-based eServices.

Statistical Quality Control for Human-Based Electronic Services

Crowdsourcing in form of human-based electronic services (people services) provides a powerful way of outsourcing tasks to a large crowd of remote workers over the Internet. Research has shown that multiple redundant results delivered by different workers can be aggregated in order to achieve a reliable result. However, existing implementations of this approach are rather inefficient as they multiply the effort for task execution and are not able to guarantee a certain quality level. As a starting point towards an integrated approach for quality management of people services we have developed a quality management model that combines elements of statistical quality control (SQC) with group decision theory. The contributions of the workers are tracked and weighted individually in order to minimize the quality management effort while guaranteing a well-defined level of overall result quality. A quantitative analysis of the approach based on an optical character recognition (OCR) scenario confirms the efficiency and reach of the approach.

Quality assurance for human-based electronic services: a decision matrix for choosing the right approach

Crowdsourcing in the form of human-based electronic services provides a powerful way of outsourcing so called human intelligence tasks (HITs) to a large workforce of people over the Internet. Because of the limited control over that workforce, it is challenging to ensure the quality of the work results. Several approaches have been proposed that can be applied to specific types of HITs. However, it is difficult to identify a suitable quality management approach for any given type of HIT. This paper aims to provide a first sketch of a decision matrix.

Dynamic and goal-based quality management for human-based electronic services

Crowdsourcing in the form of human-based electronic services (people services) provides a powerful way of outsourcing tasks to a large crowd of remote workers over the Internet. Research has shown that multiple redundant results delivered by different workers can be aggregated in order to achieve a reliable result. However, basic implementations of this approach are rather inefficient as they multiply the effort for task execution and are not able to guarantee a certain quality level. In this paper, we are addressing these challenges by elaborating on a statistical approach for quality management of people services which we had previously proposed. The approach combines elements of statistical quality management with dynamic group decisions. We present a comprehensive statistical model that enhances our original work and makes it more transparent. We also provide an extendible toolkit that implements our model and facilitates its application to real-time experiments as well as to simulations. A quantitative analysis based on an optical character recognition (OCR) scenario confirms the efficiency and reach of our model.

Dynamic Quality Management for Cloud Labor Services

Cloud labor services extend the cloud computing paradigm by providing human workforce as a scalable resource over the Internet. The approach can help labor-intensive industries to react more flexibly to varying demands but at the same time introduces severe challenges regarding quality management. This thesis addresses the challenges by using a combination of statistical quality control and newly developed dynamic voting mechanisms. Several case studies illustrate the benefits of the approach.

Evaluation of Model Variations

The model variations developed in Chap. 6 have been evaluated in two separate case studies. While the DVM for multi-labeling was applied to a medical coding business scenario, the model extension for non-deterministic tasks was applied to a product research scenario. In order to investigate the managerial implications of the models from the different perspectives of the basic cloud labor scenario, industry partners from all three sides have been included: A requester, a cloud labor platform and a workforce provider. Section 9.1 covers the medical coding case study while Sect. 9.2 addresses the product research scenario.

Statistical Quality Control

The general objective of statistical quality control (SQC) is to make a statement about the quality of multiple items in an efficient way by only assessing a sample of them. This chapter covers aspects of QM and SQC that are needed as a foundation for the actual contribution of the book.

Evaluation of Core Model

The evaluation of the core model has been performed in two complementary ways. First, it has been applied to a model scenario in order to validate its effectiveness and efficiency. Second, the applicability of the model has been provys. First, it has been applied to a moderio in order to validate its effectiveness and efficiency. Second, the applicability of the model has been proven in a business scenario in cooperation with industry partners.

Cloud Labor Services

Despite all endeavors of rationalizing their business processes, organizations still face many types of tasks which cannot be fully automated but which require human intelligence or action to be completed. Cloud labor services provide human workforce on demand as a scalable service in order to meet the needs of organizations that deal with huge but varying workloads of such tasks.

Quality Management for Cloud Labor Services

According to a 2012 analyst report on cloud labor platforms, the quality of the work results represents a “key disruptive opportunity that crowd labor platforms can exploit and improve” (Turian 2012). This chapter provides a comprehensive overview of the state-of-the-art of quality management for cloud labor services and provides a gap analysis with regard to the objectives of the book.