Pantelis Vlachos

A simulation-intensive approach for checking hierarchical models

Recent computational advances have made it feasible to fit hierarchical models in a wide range of serious applications. In the process, the question of model adequacy arises. While model checking usually addresses the entire model specification, model failures can occur at each hierarchical stage. Such failures include outliers, mean structures errors, dispersion misspecification, and inappropriate exchangeabilities. We propose an approach which is entirely simulation based. Given a model specification and a dataset, we need only be able to simulate draws from the resultant posterior. By replicating a posterior of interest using data obtained under the model we can “see” the extent of variability in such a posterior. Then, we can compare the posterior obtained under the observed data with this medley of posterior replicates to ascertain whether the former is in agreement with them and accordingly, whether it is plausible that the observed data came from the proposed model. Many such comparisons can be run, each focusing on a different potential model failure. Focusing on generalized linear mixed models, we explore the questions of when hierarchical model stages are separable and checkable and illustrate the approach with both real and simulated data.

On the calibration of Bayesian model choice criteria

Abstract Model choice is a fundamental problem in data analysis. With interest in hierarchical models which typically arise as Bayesian specifications, we confine ourselves to Bayesian model choice criteria. If Y denotes the observed data and T( Y ) is the criterion, our goal is to calibrate T( Y ) in order to assess how large or small it is under a given model. In particular, if we have the distribution of T( Y ) , then we can compute any probabilities or determine any quantities of interest. Apart from very special cases, analytic development of such distributions is intractable. Standard analytic approximations may be inapplicable if usual random effects are introduced at the various modeling levels. Indeed, calculation of T( Y ) itself is often difficult enough. We suggest a generic simulation-intensive approach for obtaining the distribution of T( Y ) to arbitrary accuracy. We focus on various Bayes factors, e.g., the usual Bayes factor, the posterior Bayes factor and the pseudo-Bayes factor. We illustrate with a binomial regression example.

Teaching Language Awareness in Rhetorical Choice Using IText and Visualization in Classroom Genre Assignments

This article introduces an IText system the authors built to enhance student practice in language awareness within commonly taught written genres (e.g., self-portraits, profiles, scenic writing, narratives, instructions, and arguments). The system provides text visualization and analysis that seek to increase students’ sensitivity to the rhetorical and whole-text implications of the small runs of language they read and write. The authors describe the way the system can create possibilities for classroom discourse and discussion about student writing that seem harder to reproduce in traditional writing classrooms. They also describe the limitations of the current system for wide-scale use and its future prospects.

Hybrid methods for calculating optimal few-stage sequential strategies: Data monitoring for a clinical trial

Optimal batch-sequential designs are difficult to compute, even when sufficient statistics and relatively uncomplicated loss functions simplify the calculations required. While backward induction applies, its difficulty grows exponentially in the number of stages, while a recently developed forward algorithm grows only linearly, but involves a maximization over a rather flat surface. This paper explores a hybrid algorithm, partially backward induction, partially forward, that has some of the advantages of each.

Textual genre analysis and identification

This chapter reports on a research program that investigates language and text from a rhetorical point of view. By rhetorical, we mean an approach that features the relationship between the speaker and the audience or between the writer and the reader. Fundamental to a rhetorical approach to language is an interest in linguistic and textual agency, how speakers and writers manage to use language strategically to affect audiences; and how audiences and readers, agents in their own right, manage, or not, to pick up on, register, and respond to a speaker or writers bids. Historical and cultural factors play a central role in how speakers and writer settle into agent roles vis-a-vis listeners and readers. It is therefore no surprise that rhetorical approaches to language treat language, culture, and history as deeply permeable with one another. Rhetorical approaches to language have, since ancient Greece, been the dominant approach for educating language-users in the western educational curriculum [1].

Chapter 9: Mining Textual Knowledge for Writing Education and Research: The DocuScope Project

This chapter reviews progress on a new corpus-based text analysis technology developed at Carnegie Mellon, called DocuScope. It describes a new technology used to support research and education involving digitized text, especially corpus-based rhetorical analysis and on-line writing education. The first half of the chapter lays out an overview of analytic choices we have made for conducting textual research. The second half of the chapter explores how the knowledge derived from the DocuScope tool can benefit writing education and the evaluation of writing curricula. The chapter overviews different frameworks and describes a specific system as a result of the choices that confronted us within the space of these frameworks. It overviews some applications of our system, notably using student samples both to evaluate and improve writing instruction and writing curricula itself. Keywords: analytic choices; corpus-based rhetorical analysis; digitized text; DocuScope; on-line writing education; student samples; textual research; writing curricula

Nonparametric Bayesian Group Sequential Design

The customary objective of a group sequential design is the comparison of several treatments or populations. The evolutionary nature of such trials encourages the use of the Bayesian paradigm in the design, monitoring and analysis of these trials. Here we focus on the design issue for the case of continuous, possibly multivariate response at each trial. Our approach is descriptive. Given a model specification, a design is characterized by a number of interim evaluations, the group size for each interim look, and a set of stopping criteria which determine our decision at a given look. By simulating replications of the design we can summarize design performance in terms of when the trial was stopped and reason for stopping. Such simulation and evaluation requires a fully Bayesian model specification for each treatment. We take a nonparametric perspective for the likelihood specification by assuming that the data is drawn from a distribution which arises through Dirichlet process mixing. However, we distinguish a sampling or “what if” prior, reflecting illustrative differences between populations, from a fitting or skeptical prior assuming no differences. By drawing trials under the sampling model, while fitting the model under the fitting prior, Bayesian learning moves us from prior indifference to detection of differences. We illustrate with two examples, one having univariate response, the other bivariate response.

Decision Analysis for a Data Monitoring Committee of a Clinical Trial

A data monitoring committee holds a position of great trust within the structure of a US clinical trial, as it alone receives the accumulating data and decides whether to continue the trial. The committee generally meets at regularly scheduled intervals, e.g. six months. Recent advances in computation allow us to find optimal group-sequential strategies for each member of such a committee. This paper reviews how we plan to use the newly available computational ability to advise members of a data monitoring committee.