Joseph C. Osborn

Refining operational logics

This paper expands on and refines the theoretical framework of operational logics, which simultaneously addresses how games operate at a procedural level and how games communicate these operations to players. In the years since their introduction, operational logics have been applied in domains ranging from game studies to game generation and game modeling languages. To support these uses and to enable new ones, we resolve some standing ambiguities and provide a catalog of key, fundamental operational logics. Concretely, we provide an explicit and detailed definition of operational logics; specify a set of logics which seems fundamental and suffices to interpret a broad variety of games across several genres; give the first detailed explanation of how exactly operational logics combine; and suggest application domains for which operational logics-based analysis and knowledge representation are especially appropriate.

Automatic mapping of NES games with mappy

Game maps are useful for human players, general-game-playing agents, and data-driven procedural content generation. These maps are generally made by hand-assembling manually-created screen-shots of game levels. Besides being tedious and error-prone, this approach requires additional effort for each new game and level to be mapped. The results can still be hard for humans or computational systems to make use of, privileging visual appearance over semantic information. We describe a software system, Mappy, that produces a good approximation of a linked map of rooms given a Nintendo Entertainment System game program and a sequence of button inputs exploring its world. In addition to visual maps, Mappy outputs grids of tiles (and how they change over time), positions of non-tile objects, clusters of similar rooms that might in fact be the same room, and a set of links between these rooms. We believe this is a necessary step towards developing larger corpora of high-quality semantically-annotated maps for PCG via machine learning and other applications.

CHARDA: Causal Hybrid Automata Recovery via Dynamic Analysis

We propose and evaluate a new technique for learning hybrid automata automatically by observing the runtime behavior of a dynamical system. Working from a sequence of continuous state values and predicates about the environment, CHARDA recovers the distinct dynamic modes, learns a model for each mode from a given set of templates, and postulates causal guard conditions which trigger transitions between modes. Our main contribution is the use of information-theoretic measures (1)~as a cost function for data segmentation and model selection to penalize over-fitting and (2)~to determine the likely causes of each transition. CHARDA is easily extended with different classes of model templates, fitting methods, or predicates. In our experiments on a complex videogame character, CHARDA successfully discovers a reasonable over-approximation of the characters true behaviors. Our results also compare favorably against recent work in automatically learning probabilistic timed automata in an aircraft domain: CHARDA exactly learns the modes of these simpler automata.

Mechanics automatically recognized via interactive observation: jumping

Jumping has been an important mechanic since its introduction in Donkey Kong. It has taken a variety of forms and shown up in numerous games, with each jump having a different feel. In this paper, we use a modified Nintendo Entertainment System (NES) emulator to semi-automatically run experiments on a large subset (~30%) of NES platform games. We use these experiments to build models of jumps from different developers, series, and games across the history of the console. We then examine these models to gain insights into different forms of jumping and their associated feel.

Crowdsourcing program preconditions via a classification game

Invariant discovery is one of the central problems in software verification. This paper reports on an approach that addresses this problem in a novel way; it crowdsources logical expressions for likely invariants by turning invariant discovery into a computer game. The game, called Binary Fission, employs a classification model. In it, players compose preconditions by separating program states that preserve or violate program assertions. The players have no special expertise in formal methods or programming, and are not specifically aware they are solving verification tasks. We show that Binary Fission players discover concise, general, novel, and human readable program preconditions. Our proof of concept suggests that crowdsourcing offers a feasible and promising path towards the practical application of verification technology.

From Mechanics to Meaning

While generative approaches to game design offer great promise, systems can only reliably generate what they can “understand,” which is often represented in a limited, implicit form in hand-crafted evaluation functions or constructive rules. Proceduralist readings, a semiformal approach for interpreting the meaning of a game based on its underlying processes and interactions in conjunction with aesthetic and cultural cues, offer a novel, systematic approach to game understanding. We formalize proceduralist argumentation as a logic program that performs static reasoning over game specifications to derive higher level meanings, as part of Gemini, a bidirectional game analysis and generation system.

Visualizing loops and data structures in Xylem: the code of plants

The visual representation of software data structures is especially relevant to the creation of games which crowd source science problems to a gaming community and to educational games which seek to teach computer science concepts within the context of computer games. Xylem: The Code of Plants is a game designed to crowd source formal software verification via loop invariant specifications. Due to the nature of this project, it was important that the game 1) appeal to a large audience, 2) support a wide variety of data structures and 3) hide any actual data from the source code that was generating the game levels. To these ends, a method of visualizing data structures was developed that features a consistent plant-based narrative metaphor, is flexible enough to accommodate multiple types of data structures while maintaining narrative integrity, and obscures all real data from the target source code.

Getting the GISST: a toolkit for the creation, analysis and reference of game studies resources

Scholars who study games lack many resources and abilities taken for granted by scholars of other forms such as literature. To begin to address this deficiency, this paper presents a basic formalization for different classes of resources associated with game critical practices, and ties those resources to a prototypical system for their reference, generation, and use in game studies arguments. The formalization is focused on three forms of reference: objective reference to game objects, enacted reference to game performances, and internal reference to sub-structures within both. We operationalize our formalization with the Game and Interactive Software Scholarship Toolkit (GISST), a custom suite of tools that manage the reference and creation of game executable data, game performance video recordings, and game executable state from emulated programs. GISST provides tools for ingesting each class of reference and for the easy creation and citation of game play videos and emulated system states. For the last case, GISST allows for embedding citations to running game programs --- in emulation --- inside web browsers, and for their retrieval through URLs from a stable citation database. The goal is to point toward a new class of scholarly support tool for arguments about games and other interactive software that would benefit from the inclusion of audiovisual and executable references.

Visualizing the strategic landscape of arbitrary games

We present Gamalyzer, a game-independent and efficient visualization of sets of play traces. Unlike previous work on game-independent visualization, we focus on sequences of game actions as opposed to sequences of game states. Action sequences directly represent players’ strategic decisions. Moreover, since game actions may already be recorded as part of games’ telemetry and metrics, Gamalyzer is easier to integrate into existing analysis toolchains than state-sequence-based visualizations. Gamalyzer displays each play trace as a vertical line, with symbols along the line indicating game events. Similar play traces (according to the Gamalyzer metric, a specialization of edit distance) are arranged together along the horizontal axis, and as traces become more and less similar to each other over time, they bend towards and away from each other. The Gamalyzer metric is also used to present only the most interestingly different traces in the visualization, with the rest grouped together under their most similar cousins. We position Gamalyzer as an ideal trace filtering and selection tool to be used in concert with a state-centric (and possibly game-specific) visualization for context. This article also provides a detailed account of the Gamalyzer metric and new advice for defining game action schema to maximize the benefits obtained from the tool, along with two detailed case studies of the Gamalyzer visualization in practice.

Operationalizing Operational Logics: Semiotic Knowledge Representations for Interactive Systems.

All projects in AI begin by selecting or devising knowledge representations suitable for the project’s functional requirements. Interactive systems (including games) have semiotic considerations on top of their functional requirements: they must be legible to users, players, and their own designers. AI working within or around interactive systems must acknowledge and support the concerns of human users. These concerns are generally phrased as inductive bias or domain knowledge and handled in an ad hoc way; I argue that it is possible and useful to represent them explicitly within a unifying approach. This work refines and extends operational logics, an interpretive framework describing how interactive systems communicate their own mechanisms to users. Making this move yields formal notations for interactive systems that are useful for humans and machines, with applications in modeling, verification, general game-playing, reverseengineering, and automatic self-documentation.