Shaili Jain

Learning about Social Learning in MOOCs: From Statistical Analysis to Generative Model

We study user behavior in the courses offered by a major massive online open course (MOOC) provider during the summer of 2013. Since social learning is a key element of scalable education on MOOC and is done via online discussion forums, our main focus is on understanding forum activities. Two salient features of these activities drive our research: (1) high decline rate: for each course studied, the volume of discussion declined continuously throughout the duration of the course; (2) high-volume, noisy discussions: at least 30 percent of the courses produced new threads at rates that are infeasible for students or teaching staff to read through. Further, a substantial portion of these discussions are not directly course-related. In our analysis, we investigate factors that are associated with the decline of activity on MOOC forums, and we find effective strategies to classify threads and rank their relevance. Specifically, we first use linear regression models to analyze the forum activity count data over time, and make a number of observations; for instance, the teaching staffs active participation in the discussions is correlated with an increase in the discussion volume but does not slow down the decline rate. We then propose a unified generative model for the discussion threads, which allows us both to choose efficient thread classifiers and to design an effective algorithm for ranking thread relevance. Further, our algorithm is compared against two baselines using human evaluation from Amazon Mechanical Turk.

Designing incentives for online question and answer forums

In this paper, we provide a simple game-theoretic model of an online question and answer forum. We focus on factual questions in which user responses aggregate while a question remains open. Each user has a unique piece of information and can decide when to report this information. The asker prefers to receive information sooner rather than later, and will stop the process when satisfied with the cumulative value of the posted information. We consider two distinct cases: a complements case, in which each successive piece of information is worth more to the asker than the previous one; and a substitutes case, in which each successive piece of information is worth less than the previous one. A best-answer scoring rule is adopted to model Yahoo! Answers, and is effective for substitutes information, where it isolates an equilibrium in which all users respond in the first round. But we find that this rule is ineffective for complements information, isolating instead an equilibrium in which all users respond in the final round. In addressing this, we demonstrate that an approval-voting scoring rule and a proportional-share scoring rule can enable the most efficient equilibrium with complements information, under certain conditions, by providing incentives for early responders as well as the user who submits the final answer.

A Game-Theoretic Analysis of Games with a Purpose

We present a simple game-theoretic model for the ESP game, an interactive game devised to label images on the web, and characterize the equilibrium behavior of the model. We show that a simple change in the incentive structure can lead to different equilibrium structure and suggest the possibility of formal incentive design in achieving desirable system-wide outcomes, complementing existing considerations of robustness against cheating and human factors.

An Economically-Principled Generative Model of AS Graph Connectivity

End-to-end packet delivery in the Internet is achieved through a system of interconnections between the network domains of independent entities called Autonomous Systems (ASes). Inter-domain connections are the result of a complex, dynamic process of negotiated business relationships between pairs of ASes. We present an economically-principled generative model for Autonomous System graph connectivity. While there is already a large literature devoted to understanding Internet connectivity at the AS level, many of these models are either static or based on generalized stochastics. In a thoughtful critique of such models, Li, Alderson, Doyle and Willinger [10] show that while many generative models reproduce certain statistical features of the AS graph, they fail to capture the good performance of realistic networks [10]. In a study of the AS’s intra-domain graph, Li, Alderson, Willinger and Doyle [11] define performance instead in terms of network throughput and show that it is very unlikely that randomized generative models will yield graphs that have the highly-optimized structure of real-world networks. The goal of this paper is to provide insight into the economic drivers that yield, over time, the rich and complex AS interconnection patterns that constitute today’s Internet. Notable features of our model include the assignment of AS business models with an asymmetric gravity model of interdomain traffic demand [3], an explicit representation of AS utility that incorporates benefits for traffic routed, congestion costs, and payments between ASes, and a deterministic process for link revision that can cascade throughout the network. This is the first attempt at AS graph modeling that incorporates a diffusion process to capture how ASes respond to direct and indirect externalities from changes in the network structure, which brings it closer to an equilibrium model. We validate our model against other generative models. To do this, we define the social planner’s problem which is parameterized by the business models of the graph and provide a method to compare earlier generative models with our model by optimizing the placement of business models on the network. We find that our model yields graphs that are better performing as compared to other dynamic generative models. We also show that our model yields a structured placement of nodes endogenously, where this placement of nodes generally reflects ASes’ business models. This is some of the first evidence of the significance of the business competitive landscape in determining the structure of the AS graph.

A game-theoretic analysis of the ESP game

“Games with a Purpose” are interactive games that users play because they are fun, with the added benefit that the outcome of play is useful work. The ESP game, developed byy von Ahn and Dabbish [2004], is an example of such a game devised to label images on the web. Since labeling images is a hard problem for computer vision algorithms and can be tedious and time-consuming for humans, the ESP game provides humans with incentive to do useful work by being enjoyable to play. We present a simple game-theoretic model of the ESP game and characterize the equilibrium behavior in our model. Our equilibrium analysis supports the fact that users appear to coordinate on low effort words. We provide an alternate model of user preferences, modeling a change that could be induced through a different scoring method, and show that equilibrium behavior in this model coordinates on high-effort words. We also give sufficient conditions for coordinating on high-effort words to be a Bayesian-Nash equilibrium. Our results suggest the possibility of formal incentive design in achieving desirable system-wide outcomes for the purpose of human computation, complementing existing considerations of robustness against cheating and human factors.

An approximation to the greedy algorithm for differential compression of very large files

This paper presents a new differential compression algorithm that combines the hash value and suffix array technique. In this algorithm, hash values for every block of the reference file is computed. Next, suffix arrays on these block hash values are computed. This algorithm finds the longest matches for every offset of the version file. This algorithm depends upon the utilization of three new data structures, the block hash table, the quick index array, and the pointer array, which improves the run-time of the algorithm, and compress very large files.

Combinatorial agency of threshold functions

We study the combinatorial agency problem introduced by Babaioff, Feldman and Nisan [5] and resolve some open questions posed in their original paper. Our results include a characterization of the transition behavior for the class of threshold functions. This result confirms a conjecture of [5], and generalizes their results for the transition behavior for the OR technology and the AND technology. In addition to establishing a (tight) bound of 2 on the Price of Unaccountability (POU) for the OR technology for the general case of n > 2 agents (the initial paper established this for n = 2, an extended version establishes a bound of 2.5 for the general case), we establish that the POU is unbounded for all other threshold functions (the initial paper established this only for the case of the AND technology). We also obtain characterization results for certain compositions of anonymous technologies and establish an unbounded POU for these cases.