Florian Teschner

How (not) to Incent Crowd Workers - Payment Schemes and Feedback in Crowdsourcing

Crowdsourcing gains momentum: In digital work places such as Amazon Mechanical Turk, oDesk, Clickworker, 99designs, or InnoCentive it is easy to distribute human work to hundreds or thousands of freelancers. In these crowdsourcing settings, one challenge is to properly incent worker effort to create value. Common incentive schemes are piece rate payments and rank-order tournaments among workers. Tournaments might or might not disclose a worker’s current competitive position via a leaderboard. Following an exploratory approach, we derive a model on worker performance in rank-order tournaments and present a series of real effort studies using experimental techniques on an online labor market to test the model and to compare dyadic tournaments to piece rate payments. Data suggests that on average dyadic tournaments do not improve performance compared to a simple piece rate for simple and short crowdsourcing tasks. Furthermore, giving feedback on the competitive position in such tournaments tends to be negatively related to workers’ performance. This relation is partially mediated by task completion and moderated by the provision of feedback: When playing against strong competitors, feedback is associated with workers quitting the task altogether and, thus, showing lower performance. When the competitors are weak, workers tend to complete the task but with reduced effort. Overall, individual piece rate payments are most simple to communicate and implement while incenting performance is on par with more complex dyadic tournaments.

Evaluating Hidden Market Design

Electronic markets are increasingly gaining importance in the coordination of complex allocation problems. One reason for market failure is the inherent complexity excluding non-sophisticated users. Recently researchers proposed the idea of hidden market design which merges the fields of market design with user interface design in order to make complex markets accessible to a broader audience. One way to accomplish that is to simplify the market interface. Thus far it remains empirically unclear how using such interfaces affects market efficiency and individual trading performance. In a prediction market for economic variables, traders can choose between a standard trading interface, and one that hides most market complexities. We find that market participants using a simplified trading -hidden market- interface are more likely to submit profitable orders.

AKX – An Exchange for Predicting Water Dam Levels in Australia

The Australian population in rural and urban areas is heavily influenced and affected by such water shortages, either economically or in their life style. Managing water resources is therefore seen as a critical environmental, social and economic issue. Good forecasts can provide better understanding for the current situation (e.g. drought severity) and consequently improve decision making.

The Impact of customizable Market Interfaces on Trading Performance

One reason for market failure is the inherent complexity that excludes non-sophisticated users. Market complexity can be reduced by adapting the market rules or by simplifying the user interface. Just recently researchers started to address this topic and identified the need to merge market and interface design. Thus far it remains unclear how to design user-centric market interfaces. In a prediction market for economic variables, traders can customize their trading interface according to their informational needs. Surprisingly, we show that on average an increase in information reduces trading performance. An explanation for this effect might lie in cognitive theory. Displaying more information increases the participants’ cognitive load and hence might reduce trading performance. We are able to distinguish between trading behaviour and performance and thereby provide insight into the interplay between information and decision making. Finally, we also track the influence of individual information elements and identify those that improve or decrease trading performance.

Beware of Performance Indicators

Online trading interfaces are important instruments for retail investors. For sound reasons, regulators obligate online brokers to inform customers about certain trade related risks. Research has shown that different behavioral biases can decrease traders’ performance and hence lead to pecuniary losses. The disposition to hold losing stocks too long and sell winning stocks too early (‘disposition effect’) is such a deviation from rational behavior. The disposition effect is analyzed for the prediction market ‘Kurspiloten’ which predicts selected stock prices and counts nearly 2000 active traders and more than 200,000 orders. We show that the disposition effect can be aggravated by visual feedback on a trader’s performance via colored trend direction arrows and percentages. However, we find no evidence that such an interface modification leads to higher activity. Furthermore, we can not confirm that creating awareness of the disposition effect with textual information is suited to decreasing its strength.

Decision Support Services Based on Dynamic Digital Analyses - Quality Metrics for Financial Planning Processes

Decision making in corporate financial controlling is typically based on the aggregation of huge data sets of financial planning items stemming from a multitude of companies with heterogeneous financial planning processes and planning quality. Quality of financial planning is usually quantified by its outcome using accepted ex-post metrics such as planning accuracy or alternative derivatives of plan versus actual distances (planning errors). However, additional metrics for measuring the quality of the planning processes themselves are mandatory. First, controllers want to determine suspicious planning data and revisions that will likely result in huge planning errors. Second, the determination of flawed planning processes allows for more profound root cause analysis of poor planning accuracy. Unfortunately, nowadays controllers have little guidance on how to assess running planning processes. This is particularly true because of the complex data structure in financial planning processes often underlying unknown assumptions and dynamics. This papers discusses two ex-ante candidate-metrics for measuring the quality of financial planning, namely Benfords Law and weak planning data efficiency. Both measures are applied to multi-year financial planning data from set of over hundred enterprises. The outcomes of numerical analysis are presented and first managerial implications regarding decision support are drawn.