David Poole

Inference for Deterministic Simulation Models: The Bayesian Melding Approach

Abstract Deterministic simulation models are used in many areas of science, engineering, and policy making. Typically, these are complex models that attempt to capture underlying mechanisms in considerable detail, and they have many user-specified inputs. The inputs are often specified by some form of trial-and-error approach in which plausible values are postulated, the corresponding outputs inspected, and the inputs modified until plausible outputs are obtained. Here we address the issue of more formal inference for such models. A probabilistic approach, called Bayesian synthesis, was shown to suffer from the Borel paradox, according to which the results can depend on the parameterization of the model. We propose a modified approach, called Bayesian melding which takes into full account information and uncertainty about both inputs and outputs to the model, while avoiding the Borel paradox. This is done by recognizing the existence of two priors, one implicit and one explicit, on each input and output; these are combined via logarithmic pooling. Bayesian melding is then standard Bayesian inference with the pooled prior on inputs, and is implemented here by posterior simulation using the sampling-importance-resampling (SIR) algorithm. We develop this initially for invertible models, and then extend it to the more difficult and more common case of noninvertible models. We illustrate the methodology using a number of examples. Simulation studies show that the method outperforms a simpler Bayesian approach in terms of mean squared error. A number of open research problems are discussed.

A Versatile Model for Packet Loss Visibility and its Application to Packet Prioritization

In this paper, we propose a generalized linear model for video packet loss visibility that is applicable to different group-of-picture structures. We develop the model using three subjective experiment data sets that span various encoding standards (H.264 and MPEG-2), group-of-picture structures, and decoder error concealment choices. We consider factors not only within a packet, but also in its vicinity, to account for possible temporal and spatial masking effects. We discover that the factors of scene cuts, camera motion, and reference distance are highly significant to the packet loss visibility. We apply our visibility model to packet prioritization for a video stream; when the network gets congested at an intermediate router, the router is able to decide which packets to drop such that visual quality of the video is minimally impacted. To show the effectiveness of our visibility model and its corresponding packet prioritization method, experiments are done to compare our perceptual-quality-based packet prioritization approach with existing Drop-Tail and Hint-Track-inspired cumulative-MSE-based prioritization methods. The result shows that our prioritization method produces videos of higher perceptual quality for different network conditions and group-of-picture structures. Our model was developed using data from high encoding-rate videos, and designed for high-quality video transported over a mostly reliable network; however, the experiments show the model is applicable to different encoding rates.

Predicting packet-loss visibility using scene characteristics

We examine the influence of scene-level content on the visibility of packet loss impairments in MPEG-2 and H.264 compressed video. We consider both global camera motion and proximity to a scene cut. We use Patient Rule Induction Method (PRIM) to pick out both highly visible and very invisible packet losses. We show that global camera motion significantly increases visibility relative to a still camera. Further, while packet losses that are concealed using a prior scene’s image are strongly visible, all other packet losses near a scene change are much less likely to be visible, all other factors being equal.

Characterizing packet-loss impairments in compressed video

We examine metrics to predict the visibility of packet losses in MPEG-2 and H.264 compressed video. We use subjective data that has a wide range of parameters, including different error concealment strategies and different compression standards. We evaluate SSIM, MSE, and a slice-boundary mismatch (SBM) metric for their effectiveness at characterizing packet-loss impairments.

Bayesian estimation of survival from mark-recapture data

An understanding of survival patterns is a fundamental component of animal population biology. Mark-recapture models are often used in the estimation of animal survival rates. Maximum likelihood estimation, via either analytic solution or numerical approximation, has typically been used for inference in these models throughout the literature. In this article, a Bayesian approach is outlined and an easily applicable implementation via Markov chain Monte Carlo is described. The method is illustrated using 13 years of mark-recapture data for fulmar petrels on an island in Orkney. Point estimates of survival are similar to the maximum likelihood estimates (MLEs), but the posterior variances are smaller than the corresponding asymptotic variances of the MLEs. The Bayesian approach yields point estimates of 0.9328 for the average annual survival probability and 14.37 years for the expected lifetime of the fulmar petrels. A simple modification that accounts for missing data is also described. The approach is easier to apply than augmentation methods in this case, and simulations indicate that the performance of the estimators is not significantly diminished by the missing data.

Estimating the size of the telephone universe: a Bayesian Mark-recapture approach

Mark-recapture models have for many years been used to estimate the unknown sizes of animal and bird populations. In this article we adapt a finite mixture mark-recapture model in order to estimate the number of active telephone lines in the USA. The idea is to use the calling patterns of lines that are observed on the long distance network to estimate the number of lines that do not appear on the network. We present a Bayesian approach and use Markov chain Monte Carlo methods to obtain inference from the posterior distributions of the model parameters. At the state level, our results are in fairly good agreement with recent published reports on line counts. For lines that are easily classified as business or residence, the estimates have low variance. When the classification is unknown, the variability increases considerably. Results are insensitive to changes in the prior distributions. We discuss the significant computational and data mining challenges caused by the scale of the data, approximately 350 million call-detail records per day observed over a number of weeks.

Ensuring Access to Emergency Services in the Presence of Long Internet Dial-Up Calls

Telephone availability is critical, particularly in emergency situations when people need immediate help. We used statistical data analysis and queueing models to identify the root cause of dial-tone unavailability in parts of the AT&T network and to develop remedies. Our solutions restored quality service, protecting the AT&T brand name and ensuring the safety of our customers. This work also gave AT&T opportunities to reduce transit charges paid to other carriers by

Problematic likelihood functions from sensible population dynamics models: a case study

15 million per year. In addition, we have filed five patent requests, of which two have been granted and the rest are pending (Chaudhury et al. 2004, Kaplan and Ramaswami 2004). Furthermore, our findings have important implications for several current areas of research related to Internet and broadband technologies, call-center engineering, and network security.

Connected cars in cellular network: a measurement study

All International Whaling Commission assessments of the Bering-Chukchi-Beaufort Seas stock of bowhead whales (Balaena mysticetus) rely on likelihood functions derived from nearly the same data and a particular family of population dynamics models. Eleven such past bowhead assessments are compared. We note that the type of dynamics model used in all of these assessments has a strong influence on the features of the likelihood surface. We examine how the likelihood surface created by such models exhibits a narrow, cusp-shaped, flat-topped, steep-edged ridge of strong nonlinear dependency between key model parameters. We discuss how such features are generally troublesome for statistical inference and interpretation. Through examples we examine some of the implications for maximum likelihood estimation and the parametric bootstrap. Although such a dynamics model is very useful for producing realistic population trajectories, it is a poor model from which to generate likelihood functions in this case because it and the data together establish a nearly chaotic dynamical nonlinear system.

Containing the effects of long holding times due to Internet dial-up connections

Connected cars are a rapidly growing segment of Internet of Things (IoT). While they already use cellular networks to support emergency response, in-car WiFi hotspots and infotainment, there is also a push towards updating their firmware over-the-air (FOTA). With millions of connected cars expected to be deployed over the next several years, and more importantly persist in the network for a long time, it is important to understand their behavior, usage patterns, and impact --- both in terms of their experience, as well as other users. Using one million connected cars on a production cellular network, we conduct network-scale measurements of over one billion radio connections to understand various aspects including their spatial and temporal connectivity patterns, the network conditions they face, use and handovers across various radio frequencies and mobility patterns. Our measurement study reveals that connected cars have distinct sets of characteristics, including those similar to regular smartphones (e.g. overall diurnal pattern), those similar to IoT devices (e.g. mostly short network sessions), but also some that belong to neither type (e.g. high mobility). These insights are invaluable in understanding and modeling connected cars in a cellular network and in designing strategies to manage their data demand.