Stanko Dimitrov

MEAD - A Platform for Multidocument Multilingual Text Summarization

Abstract This paper describes the functionality of MEAD, a comprehensive, public domain, open source, multidocument multilingual summarization environment that has been thus far downloaded by more than 500 organizations. MEAD has been used in a variety of summarization applications ranging from summarization for mobile devices to Web page summarization within a search engine and to novelty detection.

Gaming Prediction Markets: Equilibrium Strategies with a Market Maker

We study the equilibrium behavior of informed traders interacting with market scoring rule (MSR) market makers. One attractive feature of MSR is that it is myopically incentive compatible: it is optimal for traders to report their true beliefs about the likelihood of an event outcome provided that they ignore the impact of their reports on the profit they might garner from future trades. In this paper, we analyze non-myopic strategies and examine what information structures lead to truthful betting by traders. Specifically, we analyze the behavior of risk-neutral traders with incomplete information playing in a dynamic game. We consider finite-stage and infinite-stage game models. For each model, we study the logarithmic market scoring rule (LMSR) with two different information structures: conditionally independent signals and (unconditionally) independent signals. In the finite-stage model, when signals of traders are independent conditional on the state of the world, truthful betting is a Perfect Bayesian Equilibrium (PBE). Moreover, it is the unique Weak Perfect Bayesian Equilibrium (WPBE) of the game. In contrast, when signals of traders are unconditionally independent, truthful betting is not a WPBE. In the infinite-stage model with unconditionally independent signals, there does not exist an equilibrium in which all information is revealed in a finite amount of time. We propose a simple discounted market scoring rule that reduces the opportunity for bluffing strategies. We show that in any WPBE for the infinite-stage market with discounting, the market price converges to the fully-revealing price, and the rate of convergence can be bounded in terms of the discounting parameter. When signals are conditionally independent, truthful betting is the unique WPBE for the infinite-stage market with and without discounting.

Non-myopic strategies in prediction markets

One attractive feature of market scoring rules [Hanson, Information Systems Frontiers, 2003] is that they are myopically strategyproof: It is optimal for a trader to report her true belief about the likelihood of an event provided that we ignore the impact of her report on the profit she might garner from future trades. This does not rule out the possibility that traders may profit by first misleading other traders through dishonest trades and then correcting the errors made by other traders. In this paper, we describe a new approach to analyzing non-myopic strategies and the existence of myopic equilibria. We first use a simple model with two partially informed traders in a single information market to gain insight into the conditions under which different equilibrium behavior emerges. We prove that, under generic conditions, the myopically optimal strategy profile is not a weak Perfect Bayesian Equilibrium (PBE) strategy for the logarithmic market scoring rule. We show that our results extend to multiple traders and signals. We propose a simple discounted market scoring rule that reduces the opportunity for bluffing strategies. We show that in any weak PBE, myopic or otherwise, the market price converges to the optimal price, and the rate of convergence can be bounded in terms of the discounting parameter.

Promoting symmetric weight selection in data envelopment analysis: A penalty function approach

Traditionally, data envelopment analysis models assume total flexibility in weight selection, though this assumption can lead to several variables being ignored in determining the efficiency score. Existing methods constrain weight selection to a predefined range, thus removing possible feasible solutions. As such, in this paper we propose the symmetric weight assignment technique (SWAT) that does not affect feasibility and rewards decision making units (DMUs) that make a symmetric selection of weights. This allows for a method of weight restrictions that does not require preference constraints on the variables. Moreover, we show that the SWAT method may be used to differentiate among efficient DMUs.

Composition of markets with conflicting incentives

We study information revelation in scoring rule and prediction market mechanisms in settings in which traders have conflicting incentives due to opportunities to profit from the market operators subsequent actions. In our canonical model, an agent Alice is offered an incentive-compatible scoring rule to reveal her beliefs about a future event, but can also profit from misleading another trader Bob about her information and then making money off Bobs error in a subsequent market. We show that, in any weak Perfect Bayesian Equilibrium of this sequence of two markets, Alice and Bob earn payoffs that are consistent with a minimax strategy of a related game. We can then characterize the equilibria in terms of an information channel: the outcome of the first scoring rule is as if Alice had only observed a noisy version of her initial signal, with the degree of noise indicating the adverse effect of the second market on the first. We provide a partial constructive characterization of when this channel will be noiseless. We show that our results on the canonical model yield insights into other settings of information extraction with conflicting incentives.

Mobile commerce and device specific perceived risk

This study examines the role of perceived risk and access device type on consumers’ on-line purchase decisions. We use a two-step hurdle approach to estimate consumer behavior. In the first step, the decision of whether to engage in eCommerce is estimated and in the second step, how many orders to place is estimated. We use a large multi-year survey sample of households from Canada’s national statistical agency—Statistics Canada. The sample size is such that we are able to conduct sub-sample analysis of PC-only users, mobile-only users, and other-users. We show that online security and price significantly influence mobile eCommerce. We also show that there is a statistically significant difference in the intensity of eCommerce engagement across device type and consumer risk type (high/low). One of our main findings is that perceived risk affects purchase decisions for mobile users more than PC users, however additional comparisons are carried out with our analysis.

How many crowdsourced workers should a requester hire

Recent years have seen an increased interest in crowdsourcing as a way of obtaining information from a potentially large group of workers at a reduced cost. The crowdsourcing process, as we consider in this paper, is as follows: a requester hires a number of workers to work on a set of similar tasks. After completing the tasks, each worker reports back outputs. The requester then aggregates the reported outputs to obtain aggregate outputs. A crucial question that arises during this process is: how many crowd workers should a requester hire? In this paper, we investigate from an empirical perspective the optimal number of workers a requester should hire when crowdsourcing tasks, with a particular focus on the crowdsourcing platform Amazon Mechanical Turk. Specifically, we report the results of three studies involving different tasks and payment schemes. We find that both the expected error in the aggregate outputs as well as the risk of a poor combination of workers decrease as the number of workers increases. Surprisingly, we find that the optimal number of workers a requester should hire for each task is around 10 to 11, no matter the underlying task and payment scheme. To derive such a result, we employ a principled analysis based on bootstrapping and segmented linear regression. Besides the above result, we also find that overall top-performing workers are more consistent across multiple tasks than other workers. Our results thus contribute to a better understanding of, and provide new insights into, how to design more effective crowdsourcing processes.

The U.S. Navy explores detailing cost reduction via Data Envelopment Analysis

In this paper we show how a variation of Data Envelopment Analysis, the Generalized Symmetric Weight Assignment Technique, is used to assign sailors to jobs for the U.S. Navy. This method differs from others as the assignment is a multi-objective problem where the importance of each objective, called a metric, is determined by the decision-maker and promoted within the assignment problem. We explore how the method performs as the importance of particular metrics increases. Finally, we show that the proposed method leads to substantial cost savings for the U.S. Navy without degrading the resulting assignments’ performance on other metrics.

Multi-document summarization using off the shelf compression software

This study examines the usefulness of common off the shelf compression software such as gzip in enhancing already existing summaries and producing summaries from scratch. Since the gzip algorithm works by removing repetitive data from a file in order to compress it, we should be able to determine which sentences in a summary contain the least repetitive data by judging the gzipped size of the summary with the sentence compared to the gzipped size of the summary without the sentence. By picking the sentence that increased the size of the summary the most, we hypothesized that the summary will gain the sentence with the most new information. This hypothesis was found to be true in many cases and to varying degrees in this study.

A Study on the Influence of the Number of MTurkers on the Quality of the Aggregate Output

Recent years have seen an increased interest in crowdsourcing as a way of obtaining information from a large group of workers at a reduced cost. In general, there are arguments for and against using multiple workers to perform a task. On the positive side, multiple workers bring different perspectives to the process, which may result in a more accurate aggregate output since biases of individual judgments might offset each other. On the other hand, a larger population of workers is more likely to have a higher concentration of poor workers, which might bring down the quality of the aggregate output.