Eric T. Bradlow

A General Bayesian Model for Testlets: Theory and Applications

The need for more realistic and richer forms of assessment in educational tests has led to the inclusion (in many tests) of polytomously scored items, multiple items based on a single stimulus (a “testlet”), and the increased use of a generalized mixture of binary and polytomous item formats. In this paper, the authors extend earlier work on the modeling of testlet-based response data to include the situation in which a test is composed, partially or completely, of polytomously scored items and/or testlets. The model they propose, a modified version of commonly employed item response models, is embedded within a fully Bayesian framework, and inferences under the model are obtained using Markov chain Monte Carlo techniques. The authors demonstrate its use within a designed series of simulations and by analyzing operational data from the North Carolina Test of Computer Skills and the Educational Testing Service’s Test of Spoken English. Their empirical findings suggest that the North Carolina Test of Computer Skills exhibits significant testlet effects, indicating significant dependence of item scores obtained from common stimuli, whereas the Test of Spoken English does not.

An Integrated Model for Bidding Behavior in Internet Auctions: Whether, Who, When, and How Much

The authors develop a general parametric modeling framework for bidding behavior in Internet auctions. Toward this end, they incorporate four key components of the bidding process under their framework: whether people bid on an auction, (if so) who bids, when they bid, and how much they bid over the entire sequence of bids in an auction. This integrated framework is based on a single, latent, time-varying construct of consumer willingness to bid, which bidders have and update for a particular auction item over the course of the auction duration. Using a database of notebook auctions from one of the largest Internet auction sites in Korea, the authors demonstrate that this general (yet parsimonious) model captures the key behavioral aspects of bidding behavior. Furthermore, the authors provide a valuable tool for managers at auction sites to conduct their customer relationship management efforts, which require them to evaluate the ”goodness” of the listed auction items (whether people bid) and the goodness of the potential bidders (who bids, when they bid, and how much they bid).

Testlet Response Theory: An Analog for the 3PL Model Useful in Testlet-Based Adaptive Testing

The invention of short multiple choice test items provided an enormous technical and practical advantage for test developers; certainly the items could be scored easily, but that was just one of the reasons for their popular adoption in the early part of the 20th century. A more important reason was the increase in validity offered because of the speed with which such items could be answered. This meant that a broad range of content specifications could be addressed, and hence an examinee need no longer be penalized because of an unfortunate choice of constructed response (e.g., essay) question. These advantages, as well as many others (see Anastasi, 1976, 415-417) led the multiple choice format to become, by far, the dominant form used in large-scale standardized mental testing throughout this century. Nevertheless, this breakthrough in test construction, dominant at least since the days of Army is currently being reconsidered. Critics of tests that are made up of large numbers of short questions suggest that decontextualized items yield a task that is abstracted too far from the domain of inference for many potential uses. For several reasons, only one of them as a response to this criticism, variations in test theory were considered that would allow the retention of the shortanswer format while at the same time eliminating the shortcomings expressed by those critics. One of these variations was the development of item response theory (IRT), an analytic breakthrough in test scoring. A key feature of IRT is that examinee responses are conceived of as reflecting evidence of a particular location on a single underlying latent

Structural Estimation of the Effect of Out-of-Stocks

We develop a structural demand model that endogenously captures the effect of out-of-stocks on customer choice by simulating a time-varying set of available alternatives. Our estimation method uses store-level data on sales and partial information on product availability. Our model allows for flexible substitution patterns, which are based on utility maximization principles and can accommodate categorical and continuous product characteristics. The methodology can be applied to data from multiple markets and in categories with a relatively large number of alternatives, slow-moving products, and frequent out-of-stocks (unlike many existing approaches). In addition, we illustrate how the model can be used to assist the decisions of a store manager in two ways. First, we show how to quantify the lost sales induced by out-of-stock products. Second, we provide insights on the financial consequences of out-of-stocks and suggest price promotion policies that can be used to help mitigate their negative economic impact, which run counter to simple commonly used heuristics.

Public Reporting On Hospital Process Improvements Is Linked To Better Patient Outcomes

The Centers for Medicare and Medicaid Services publicly reports so-called process performance at all U.S. hospitals, such as whether certain recommended treatments are given to specific types of patients. We examined whether hospital performance on key process indicators improved during the three years since this reporting began. We also studied whether or not these changes improved patient outcomes or yielded other quality improvements, such as reduced hospital readmission rates. We found that, from 2004 to 2006, hospital process performance improved and was associated with better patient and quality outcomes. Most notably, for acute myocardial infarction, performance improvements were associated with declines in mortality rates, lengths-of-stay, and readmission rates. Although we cannot conclude that public reporting caused the improvement in processes or outcomes, these results are encouraging, since improving process performance may improve quality more broadly.

Research Note---The Traveling Salesman Goes Shopping: The Systematic Deviations of Grocery Paths from TSP Optimality

We examine grocery shopping paths using the traveling salesman problem (TSP) as a normative frame of reference. We define the TSP-path for each shopper as the shortest path that connects all of his purchases. We then decompose the length of each observed path into three components: the length of the TSP-path, the additional distance because of order deviation (i.e., not following the TSP-order of category purchases), and the additional distance because of travel deviation (i.e., not following the shortest point-to-point route). We explore the relationship between these deviations and different aspects of in-store shopping/purchase behavior. Among other things, our results suggest that (1) a large proportion of trip length is because of travel deviation; (2) paths that deviate substantially from the TSP solution are associated with larger shopping baskets; (3) order deviation is strongly associated with purchase behavior, while travel deviation is not; and (4) shoppers with paths closer to the TSP solution tend to buy more from frequently purchased product categories.

A Bivariate Timing Model of Customer Acquisition and Retention

Two widely recognized components, central to the calculation of customer value, are acquisition and retention propensities. However, while extant research has incorporated such components into different types of models, limited work has investigated the kinds of associations that may exist between them. In this research, we focus on the relationship between a prospective customers time until acquisition of a particular service and the subsequent duration for which he retains it, and examine the implications of this relationship on the value of prospects and customers. To accomplish these tasks, we use a bivariate timing model to capture the relationship between acquisition and retention. Using a split-hazard model, we link the acquisition and retention processes in two distinct yet complementary ways. First, we use the Sarmonov family of bivariate distributions to allow for correlations in the observed acquisition and retention times within a customer; next, we allow for differences across customers using latent classes for the parameters that govern the two processes. We then demonstrate how the proposed methodology can be used to calculate the discounted expected value of a subscription based on the time of acquisition, and discuss possible applications of the modeling framework to problems such as customer targeting and resource allocation.

A Unified Approach to Conjoint Analysis Models

We present a unified approach to conjoint analysis models using a Bayesian framework. One data source is used to form a prior distribution for the partworths, whereas full-profile evaluations under a rating scale, ranking, discrete choice, or constant-sum scale constitute the likelihood data (“one model fits all”). Standard existing models for conjoint analysis, considered in the literature, become particular cases of the proposed specification, and explicit formulas for the gains of using multiple sources of data are presented. We demonstrate our method on a conjoint analysis dataset containing both self-explicated evaluations and constant-sum profile data on new automobiles originally collected and described by Krieger, Green, and Umesh. Our empirical findings are “mixed” in that for some out-of-sample predictive measures our Bayesian approach is superior to using profile-only or self-explicated–only data, and for other measures it is not. Our findings suggest that the primary determinant as to whether self-explicated data add information above and beyond the profile data is the degree of incongruity between the calibration and validation data formats. Specifically, when the same type of data are collected for both sources, self-explicated data add less, and vice versa. A further contribution of our work (and one that is easily implemented, given the general nature of our approach) is that we take our data and fit constant sum, ranking, and binary choice models to it, allowing us to infer the “change” in information when taking data and transforming its scale (a common practice). A simulation study indicates the viability of this approach. A simple Gibbs sampler simulation scheme adapted to the form of the outcome measure, using data augmentation and Metropolis sampling, is considered for inference under the model.

A Bayesian lifetime model for the Hot 100 Billboard songs

People have long been enamored by ranked lists of celebrities (e.g., “best-dressed” lists), places (e.g., best cities to live in), things (e.g., most popular songs, books, and movies), and countless other entities. Likewise, people are equally interested in watching these rankings evolve over time and speculating about possible future changes (e.g., who will be ##1 next week?) We focus on a popular, but fairly typical ranked list (the Billboard “Hot 100” songs) to explain and model the simultaneous movement of multiple items (songs) up and down the chart over time. Although our interest in Billboard data partly reflects the glamour of the music industry, these charts provide a very rich and general data structure. Surprisingly little research has been done on time-series models for ranked objects. We further enrich the dataset by adding covariates (e.g., artist history) to capture additional sources of variation across songs and over time. We posit a model for the time series of charts based on a latent lifetime (worth) process. Specifically, the latent popularity of each song is assumed to follow a generalized gamma (GG) lifetime curve with double exponentially (DE) distributed errors. The immense flexibility in the GG family allows the mean of a songs latent worth process to follow an exponential, Weibull, lognormal, or gamma curve (among others), reflecting the many possible paths that it might take through the chart. The DE error structure is used for convenience, as it leads to a well-established “exploding” multinomial-logit likelihood. This framework is embedded in a Bayesian structure in which parameters of the GG curve are song specific, with means related to observed covariates, and assumed to come from a multivariate lognormal prior. Inferences from the model are obtained from posterior samples using Markov chain Monte Carlo techniques.

A Hierarchical Bayes Model for Assortment Choice

In this research, the authors merge an established methodology—hierarchical Bayesian modeling—and an existing utility model—Farquhar and Raos (1976) balance model—to describe individual choices among assortments of multiattributed items. This approach facilitates addressing three managerial questions of direct importance: (1) Which assortment of a given size has the most customers for whom it is the preferred assortment (and what fraction of customers)? (2) Which products should be added to a given assortment to maximize the fraction of customers who can find their most preferred assortment? and (3) For a given assortment, what type of customer is likely to purchase it? The model is applied to assortment choices constructed from a set of eight popular magazines with five measured attributes: business, current events, human interest stories, sports, and technology. The analysis indicates that consumers are heterogeneous: Some customers are price sensitive and unresponsive to the magazine attributes, and others are sensitive to the magazine features but do not necessarily want more of them. In addition, the authors observe that many subjects do not prefer varied assortments; rather, consumers focus on purchasing magazines with high levels of the attribute they want.