Brent Harrison

Using sequential observations to model and predict player behavior

In this paper, we present a data-driven technique for designing models of user behavior. Previously, player models were designed using user surveys, small-scale observation experiments, or knowledge engineering. These methods generally produced semantically meaningful models that were limited in their applicability. To address this, we have developed a purely data-driven methodology for generating player models based on past observations of other players. Our underlying assumption is that we can accurately predict what a player will do in a given situation if we examine enough data from former players that were in similar situations. We have chosen to test our method on achievement data from the MMORPG World of Warcraft. Experiments show that our method greatly outperforms a baseline algorithm in both precision and recall, proving that this method can create accurate player models based solely on observation data.

Learning From Explanations Using Sentiment and Advice in RL

In order for robots to learn from people with no machine learning expertise, robots should learn from natural human instruction. Most machine learning techniques that incorporate explanations require people to use a limited vocabulary and provide state information, even if it is not intuitive. This paper discusses a software agent that learned to play the Mario Bros. game using explanations. Our goals to improve learning from explanations were twofold: 1) to filter explanations into advice and warnings and 2) to learn policies from sentences without state information. We used sentiment analysis to filter explanations into advice of what to do and warnings of what to avoid. We developed object-focused advice to represent what actions the agent should take when dealing with objects. A reinforcement learning agent used object-focused advice to learn policies that maximized its reward. After mitigating false negatives, using sentiment as a filter was approximately 85% accurate. object-focused advice performed better than when no advice was given, the agent learned where to apply the advice, and the agent could recover from adversarial advice. We also found the method of interaction should be designed to ease the cognitive load of the human teacher or the advice may be of poor quality.

A Computational Model of Plan-Based Narrative Conflict at the Fabula Level

Conflict is an essential element of interesting stories. In this paper, we operationalize a narratological definition of conflict and extend established narrative planning techniques to incorporate this definition. The conflict partial order causal link planning algorithm (CPOCL) allows narrative conflict to arise in a plan while maintaining causal soundness and character believability. We also define seven dimensions of conflict in terms of this algorithms knowledge representation. The first three-participants, reason, and duration-are discrete values which answer the “who?” “why?” and “when?” questions, respectively. The last four-balance, directness, stakes, and resolution-are continuous values which describe important narrative properties that can be used to select conflicts based on the authors purpose. We also present the results of two empirical studies which validate our operationalizations of these narrative phenomena. Finally, we demonstrate the different kinds of stories which CPOCL can produce based on constraints on the seven dimensions.

Four quantitative metrics describing narrative conflict

Conflict is an essential element of interesting stories. In previous work, we proposed a formal model of narrative conflict along with 4 quantitative dimensions which can be used to distinguish one conflict from another based on context: balance, directness, intensity, and resolution. This paper presents the results of an experiment designed to measure how well these metrics predict the responses of human readers when asked to measure these same values in a set of four stories. We conclude that our metrics are able to rank stories similarly to human readers for each of these four dimensions.

Analytics-driven dynamic game adaption for player retention in Scrabble

This paper shows how game analytics can be used in conjunction with an adaptive system in order to increase player retention at the level of individual game sessions in Scrabblesque, a Flash game based on the popular board game Scrabble. In this paper, we use game analytic knowledge to create a simplified search space (called the game analytic space) of board states. We then target a distribution of game analytic states that are predictive of players playing a complete game session of Scrabblesque in order to increase player retention. Our adaptive system then has a computer-controlled AI opponent take moves that will help realize this distribution of game analytic states with the ultimate goal of reducing the quitting rate. We test this system by performing a user study in which we compare how many people quit playing the adaptive version of Scrabblesque early and how many people quit playing a nonadaptive version of Scrabblesque early. We also compare how well the adaptive version of Scrabblesque was able to influence player behavior as described by game analytics. Our results show that our adaptive system is able to produce a significant reduction in the quitting rate (p = 0.03) when compared to the non-adaptive version. In addition, the adaptive version of Scrabblesque is able to better fit a target distribution of game analytic states when compared to the non-adaptive version.

An Analytic and Psychometric Evaluation of Dynamic Game Adaption for Increasing Session-Level Retention in Casual Games

This paper shows how game analytics can be used to dynamically adapt casual game environments in order to increase session-level retention. Our technique involves using game analytics to create an abstracted game analytic space to make the problem tractable. We then model player retention in this space and use these models to make guided changes to game analytics in order to bring about a targeted distribution of game states that will, in turn, influence player behavior. Experiments performed showed that the adaptive versions of two different casual games, Scrabblesque and Sidequest: The Game, were able to better fit a target distribution of game states while also significantly reducing the quitting rate compared to the nonadaptive version of the games. We showed that these gains were not coming at the cost of player experience by performing a psychometric evaluation in which we measured player intrinsic motivation and engagement with the game environments. In both cases, we showed that players playing the adaptive version of the games reported higher intrinsic motivation and engagement scores than players playing the nonadaptive version of the games.

Biclustering-driven ensemble of Bayesian belief network classifiers for underdetermined problems

In this paper, we present BENCH (Biclustering-driven ENsemble of Classifiers), an algorithm to construct an ensemble of classifiers through concurrent feature and data point selection guided by unsupervised knowledge obtained from biclustering. BENCH is designed for underdetermined problems. In our experiments, we use Bayesian Belief Network (BBN) classifiers as base classifiers in the ensemble; however, BENCH can be applied to other classification models as well. We show that BENCH is able to increase prediction accuracy of a single classifier and traditional ensemble of classifiers by up to 15% on three microarray datasets using various weighting schemes for combining individual predictions in the ensemble.

Improvisational Computational Storytelling in Open Worlds

Improvisational storytelling involves one or more people interacting in real-time to create a story without advanced notice of topic or theme. Human improvisation occurs in an open-world that can be in any state and characters can perform any behaviors expressible through natural language. We propose the grand challenge of computational improvisational storytelling in open-world domains. The goal is to develop an intelligent agent that can sensibly co-create a story with one or more humans through natural language. We lay out some of the research challenges and propose two agent architectures that can provide the basis for exploring the research issues surrounding open-world human-agent interactions.

A Survey and Analysis of Techniques for Player Behavior Prediction in Massively Multiplayer Online Role-Playing Games

While there has been much research done on player modeling in single-player games, player modeling in massively multiplayer online role-playing games (MMORPGs) has remained relatively unstudied. In this paper, we survey and evaluate three classes of player modeling techniques: 1) manual tagging; 2) collaborative filtering; and 3) goal recognition. We discuss the strengths and weaknesses that each technique provides in the MMORPG environment using desiderata that outline the traits an algorithm should posses in an MMORPG. We hope that this discussion as well as the desiderata help future research done in this area. We also discuss how each of these classes of techniques could be applied to the MMORPG genre. In order to demonstrate the value of our analysis, we present a case study from our own work that uses a model-based collaborative filtering algorithm to predict achievements in World of Warcraft. We analyze our results in light of the particular challenges faced by MMORPGs and show how our desiderata can be used to evaluate our technique.

Exploring key-level analytics for computational modeling of typing behavior

Typing is a human activity that can be affected by a number of situational and task-specific factors. Changes in typing behavior resulting from the manipulation of such factors can be predictably observed through key-level input analytics. Here we present a study designed to explore these relationships. Participants play a typing game in which letter composition, word length and number of words appearing together are varied across levels. Inter-keystroke timings and other higher order statistics (such as bursts and pauses), as well as typing strategies, are analyzed from game logs to find the best set of metrics that quantify the effect that different experimental factors have on observable metrics. Beyond task-specific factors, we also study the effects of habituation by recording changes in performance with practice. Currently a work in progress, this research aims at developing a predictive model of human typing. We believe this insight can lead to the development of novel security proofs for interactive systems that can be deployed on existing infrastructure with minimal overhead. Possible applications of such predictive capabilities include anomalous behavior detection, authentication using typing signatures, bot detection using word challenges etc.