John A. Rice

Smoothing Spline Models for the Analysis of Nested and Crossed Samples of Curves

Abstract We introduce a class of models for an additive decomposition of groups of curves stratified by crossed and nested factors, generalizing smoothing splines to such samples by associating them with a corresponding mixed-effects model. The models are also useful for imputation of missing data and exploratory analysis of variance. We prove that the best linear unbiased predictors (BLUPs) from the extended mixed-effects model correspond to solutions of a generalized penalized regression where smoothing parameters are directly related to variance components, and we show that these solutions are natural cubic splines. The model parameters are estimated using a highly efficient implementation of the EM algorithm for restricted maximum likelihood (REML) estimation based on a preliminary eigenvector decomposition. Variability of computed estimates can be assessed with asymptotic techniques or with a novel hierarchical bootstrap resampling scheme for nested mixed-effects models. Our methods are applied to me...

A simple and effective method for predicting travel times on freeways

We present a method to predict the time that will be needed to traverse a given section of a freeway when the departure is at a given time in the future. The prediction is done on the basis of the current traffic situation in combination with historical data. We argue that, for our purposes, the current traffic situation of a section of a freeway is well summarized by the current status travel time. This is the travel time that would result if one were to depart immediately and no significant changes in traffic would occur. This current status travel time can be estimated from single- or double-loop detectors, video data, probe vehicles, or any other means. Our prediction method arises from the empirical observation that there exists a linear relationship between any future travel time and the current status travel time. The slope and intercept of this relationship may change subject to the time of day and the time until departure, but linearity persists. This observation leads to a prediction scheme by means of linear regression with time-varying coefficients.

SHORT-TERM TRAVEL TIME PREDICTION

Abstract Effective prediction of travel times is central to many advanced traveler information and transportation management systems. In this paper we propose a method to predict freeway travel times using a linear model in which the coefficients vary as smooth functions of the departure time. The method is straightforward to implement, computationally efficient and applicable to widely available freeway sensor data. We demonstrate the effectiveness of the proposed method by applying the method to two real-life loop detector data sets. The first data set––on I-880––is relatively small in scale, but very high in quality, containing information from probe vehicles and double loop detectors. On this data set the prediction error ranges from 5% for a trip leaving immediately to 10% for a trip leaving 30 min or more in the future. Having obtained encouraging results from the small data set, we move on to apply the method to a data set on a much larger spatial scale, from Caltrans District 12 in Los Angeles. On this data set, our errors range from about 8% at zero lag to 13% at a time lag of 30 min or more. We also investigate several extensions to the original method in the context of this larger data set.

Detecting Errors and Imputing Missing Data for Single-Loop Surveillance Systems

Single-loop detectors provide the most abundant source of traffic data in California, but loop data samples are often missing or invalid. A method is described that detects bad data samples and imputes missing or bad samples to form a complete grid of clean data, in real time. The diagnostics algorithm and the imputation algorithm that implement this method are operational on 14,871 loops in six districts of the California Department of Transportation. The diagnostics algorithm detects bad (malfunctioning) single-loop detectors from their volume and occupancy measurements. Its novelty is its use of time series of many samples, instead of basing decisions on single samples, as in previous approaches. The imputation algorithm models the relationship between neighboring loops as linear and uses linear regression to estimate the value of missing or bad samples. This gives a better estimate than previous methods because it uses historical data to learn how pairs of neighboring loops behave. Detection of bad loops and imputation of loop data are important because they allow algorithms that use loop data to perform analysis without requiring them to compensate for missing or incorrect data samples.

Smoothing Spline Estimation for Varying Coefficient Models With Repeatedly Measured Dependent Variables

Longitudinal samples, i.e., datasets with repeated measurements over time, are common in biomedical and epidemiological studies such as clinical trials and cohort observational studies. An exploratory tool for the analyses of such data is the varying coefficient model Y(t)=XT(t)β(t) + ϵ(t), where Y(t) and X(t) = (X(0)(t),…,X(k)(t))T are the response and covariates at time t, β(t) = (β0(t),…,βk(t))T are smooth coefficient curves of t and ϵ(t) is a mean zero stochastic process. A special case that is of particular interest in many situations is data with time-dependent response and time-independent covariates. We propose in this article a componentwise smoothing spline method for estimating β0(t),…,βk(t) nonparametrically based on the previous varying coefficient model and a longitudinal sample of (t,Y(t),X) with time-independent covariates X = (X(0),…,X(k))T from n independent subjects. A “leave-one-subject-out” cross-validation is suggested to choose the smoothing parameters. Asymptotic properties of our spline estimators are developed through the explicit expressions of their asymptotic normality and risk representations, which provide useful insights for inferences. Applications and finite sample properties of our procedures are demonstrated through a longitudinal sample of opioid detoxification and a simulation study.

Monotone smoothing with application to dose-response curves and the assessment of synergism.

A spline-based procedure for monotone curve smoothing is proposed and is illustrated by application to dose-response curves. It is then shown how such smoothing can be applied to assess possible synergism or antagonism of two drugs.

Displaying the Important Features of Large Collections of Similar Curves

Abstract Naively displaying a large collection of curves by superimposing them one on another all on the same graph is largely uninformative and aesthetically unappealing. We propose that a simple principal component analysis be used to identify important modes of variation among the curves and that principal component scores be used to identify particular curves which clearly demonstrate the form and extent of that variation. As a result, we obtain a small number of figures on which are plotted a very few “representative” curves from the original collection; these successfully convey the major information present in sets of “similar” curves in a clear and attractive manner. Useful adjunct displays, including the plotting of principal component scores against covariates, are also described. Two examples—one concerning a data-based bandwidth selection procedure for kernel density estimation, the other involving ozone level curve data—illustrate the ideas.

Estimating the proportion of false null hypotheses among a large number of independently tested hypotheses

We consider the problem of estimating the number of false null hypotheses among a very large number of independently tested hypotheses, focusing on the situation in which the proportion of false null hypotheses is very small. We propose a family of methods for establishing lower 100(1 - a)% confidence bounds for this proportion, based on the empirical distribution of the p-values of the tests. Methods in this family are then compared in terms of ability to consistently estimate the proportion by letting α → 0 as the number of hypothesis tests increases and the proportion decreases. This work is motivated by a signal detection problem that occurs in astronomy.

Accurate estimation of travel times from single-loop detectors

As advanced traveler information systems become increasingly prevalent the importance of accurately estimating link travel times grows. Unfortunately, the predominant source of highway traffic information comes from single-loop loop detectors which do not directly measure vehicle speed. The conventional method of estimating speed, and hence travel time, from the single-loop data is to make a common vehicle length assumption and to use a resulting identity relating density, flow, and speed. Hall and Persaud (Transportation Research Record 1232, 9-16, 1989) and Pushkar et al. (Transportation Research Record 1457, 149-157, 1994) show that these speed estimates are flawed. In this paper we present a methodology to estimate link travl times directly from the single-loop loop detector flow and occupancy data without heavy reliance on the flawed speed calculations. Our methods arise naturally from an intuitive stochastic model of traffic flow. We demonstrate by example on data collected on I-880 data (Skabardonis et al. Technical Report UCB-ITS-PRR-95-S, Institute of Transportation Studies, University of California, 1994) that when the loop detector data has a fine resolution (about one second), the single-loop based estimates of travel time can accurately track the true travel time through many degrees of congestion. Probe vehicle data and double-loop based travel time estimates corroborate the accuracy of our methods in our examples.

Maximum Likelihood Estimation and Identification Directly from Single-Channel Recordings

We present a method for analysis of noisy sampled data from a single-channel patch clamp which bypasses restoration of an idealized quantal signal. We show that, even in the absence of a specific model, the conductance levels and mean dwell times within those levels can be estimated. Estimation of the rate constants of a hypothesized kinetic scheme is more difficult. We present examples in which the rate constants can be effectively estimated and examples in which they cannot.