M. Brian Blake

Self-Generating a Labor Force for Crowdsourcing: Is Worker Confidence a Predictor of Quality?

When leveraging the crowd to perform complex tasks, it is imperative to identify the most effective worker for a particular job. Demographic profiles provided by workers, skill self-assessments by workers, and past performance as captured by employers all represent viable data points available within labor markets. Employers often question the validity of a workers self-assessment of skills and expertise level when selecting workers in context of other information. More specifically, employers would like to answer the question, Is worker confidence a predictor of quality? In this paper, we discuss the state-of-the-art in recommending crowd workers based on assessment information. A major contribution of our work is an architecture, platform, and push/pull process for categorizing and recommending workers based on available self-assessment information. We present a study exploring the validity of skills input by workers in light of their actual performance and other metrics captured by employers. A further contribution of this approach is the extrapolation of a body of workers to describe the nature of the community more broadly. Through experimentation, within the language-processing domain, we demonstrate a new capability of deriving trends that might help future employers to select appropriate workers.

Using Collaborative Filtering to Automate Worker-Job Recommendations for Crowdsourcing Services

Generally, in crowdsourcing, providers advertise their task offerings (i.e. the open call model) largely to crowdworkers who subscribe their interest in working (i.e. subscription model). The combined open call and subscription model represent significant bottlenecks for recruitment in the paradigm of crowdsourcing. Consequently, attracting and retaining a crowd are the major challenges to the success of a crowdsourcing platform and forming a labor market. To address this problem, we introduce a worker-job matching model for crowdsourcing supported by a service-oriented architecture. The service-oriented architecture implements a push-pull mechanism and an underlying algorithm based on collaborative filtering techniques. Preliminary studies show that the infrastructure can effectively infer the levels of expertise of potential crowdworkers based on their profile and past performance history.

Crowdsourcing, Mixed Elastic Systems and Human-Enhanced Computing–A Survey

State-of-the-art practices have recognized the utility of leveraging human intervention as a crucial aspect of modern computing systems. The emerging crowdsourcing paradigm is based on harnessing human intelligence, effort and rational behaviors to augment computation and analysis. In addition to the crowdsourcing paradigm, new techniques have emerged that incorporate machine and human computational resources together forming a hybrid intelligence when addressing complex problems and tasks. This combined technique is particularly impactful if human and machine contributions can scale automatically in response to their respective efficiency and effectiveness when addressing subsets of a bigger problem – an approach that we have named mixed elastic systems. In this survey, we highlight state-of-the-art projects that investigate crowdsourcing, hybrid intelligence systems and mixed elastic systems. We also present a taxonomy and classification of the broader domain of human-enhanced computing systems as it assimilates crowdsourcing, hybrid intelligence, and mixed elastic systems.

Towards a Distributed Worker-Job Matching Architecture for Crowdsourcing

While the crowd sourcing paradigm facilitates the use of human-enacted resources from large groups of individuals, matching workers with jobs is limited by the need for these potential workers to proactively subscribe to various networks. This subscription phase is part of an open call model that reduces the ability for crowd sourcing platforms to scale or retain crowd-oriented workers. Leveraging collaborative filtering techniques, in this paper, we propose an alternative model that seeks to address the issue through a recommendation technique and system that exploits a push-pull model.

Reusable Meta-Models for Crowdsourcing Driven Elastic Systems (Invited Paper)

Elastic systems utilize both human and machine working units to accomplish tasks that are eligible for crowdsourcing. The quality in the results of work completed by either type of computing unit is tantamount on the characteristics they bear. In this paper we draw parallels from our previous work into looking at the suitability of working units in completing viable tasks in crowdsourcing. We seek to understand characteristics for modeling tasks and workers within these types of systems. Based on our experiments and lessons learned in related literature, we propose a dynamic worker-task information meta-model with a corresponding operational workflow model that can be used in a variety of problem domains involving crowdsourced tasks to provide support in making this decision.

Next-Generation Mobile Services

What might comic book heroes, crowdsensing, and Internet computing have in common? In this column, they share the possibility of futuristic techniques for mobile devices.

Monetizing Autonomous Control

Somewhere in the intersection between networked software (the Internet of Things) and artificial intelligence lies an impressive opportunity for Internet Computing professionals. Here, Editor-in-Chief Brian Blake explores how the culmination of the two will create a new economy for monetized control software.

Twenty Years in the Making

In honor of IEEE Internet Computings 20th anniversary, Editor-in-Chief Brian Blake discusses the articles showcased in this issue to celebrate this special milestone.

Customizing and Sizing the Internet for IoT Devices

The Internet began as something much smaller; in certain ways less user-friendly but far more manageable. As the Internet evolves and the Internet of Things comes into focus, how can researchers and engineers customize diverse and open information to handle the Internets rapid growth while ensuring a certain level of manageability?

Reflecting on Software Engineering Research for Internet Computing

Now that 2015 has drawn to a close and 2016 is underway, what were the prominent Internet computing-related themes from last years conferences? Here, Editor-in-Chief Brian Blake investigates.