J. T. Gene Hwang

Prediction Intervals for Artificial Neural Networks

Abstract The artificial neural network (ANN) is becoming a very popular model for engineering and scientific applications. Inspired by brain architecture, artificial neural networks represent a class of nonlinear models capable of learning from data. Neural networks have been applied in many areas, including pattern matching, classification, prediction, and process control. This article focuses on the construction of prediction intervals. Previous statistical theory for constructing confidence intervals for the parameters (or the weights in an ANN), is inappropriate, because the parameters are unidentifiable. We show in this article that the problem disappears in prediction. We then construct asymptotically valid prediction intervals and also show how to use the prediction intervals to choose the number of nodes in the network. We then apply the theory to an example for predicting the electrical load.

Estimating the number of true null hypotheses from a histogram of p values

In an earlier article, an intuitively appealing method for estimating the number of true null hypotheses in a multiple test situation was proposed. That article presented an iterative algorithm that relies on a histogram of observed p values to obtain the estimator. We characterize the limit of that iterative algorithm and show that the estimator can be computed directly without iteration. We compare the performance of the histogram-based estimator with other procedures for estimating the number of true null hypotheses from a collection of observed p values and find that the histogram-based estimator performs well in settings similar to those encountered in microarray data analysis. We demonstrate the approach using p values from a large microarray experiment aimed at uncovering molecular mechanisms of barley resistance to a fungal pathogen.

Quick calculation for sample size while controlling false discovery rate with application to microarray analysis

MOTIVATION Sample size calculation is important in experimental design and is even more so in microarray or proteomic experiments since only a few repetitions can be afforded. In the multiple testing problems involving these experiments, it is more powerful and more reasonable to control false discovery rate (FDR) or positive FDR (pFDR) instead of type I error, e.g. family-wise error rate (FWER). When controlling FDR, the traditional approach of estimating sample size by controlling type I error is no longer applicable. RESULTS Our proposed method applies to controlling FDR. The sample size calculation is straightforward and requires minimal computation, as illustrated with two sample t-tests and F-tests. Based on simulation with the resultant sample size, the power is shown to be achievable by the q-value procedure. AVAILABILITY A Matlab code implementing the described methods is available upon request.

Prediction Intervals, Factor Analysis Models, and High-Dimensional Empirical Linear Prediction

Abstract We discuss a technique that provides prediction intervals based on a model called an empirical linear model. The technique, high-dimensional empirical linear prediction (HELP), involves principal component analysis, factor analysis and model selection. In fact, a special case of the empirical model is the factor analysis model. A factor analysis model does not generally aim at prediction, however. Therefore, HELP can be viewed as a technique that provides prediction (and confidence) intervals based on a factor analysis model or a more generalized model, possibly with unknown dimension to be estimated. Although factor analysis models do not typically have justifiable theory due to nonidentifiability, we show that our intervals are justifiable asymptotically. An interval for a future response is called a prediction interval; an interval for the mean of the future response is called a confidence interval. These intervals were compared to the intervals of Hwang and Liu, which were derived using stand...

Optimal Confidence Sets, Bioequivalence, and the Limaçon of Pascal

Abstract We begin with a decision-theoretic investigation into confidence sets that minimize expected volume at a given parameter value. Such sets are constructed by inverting a family of uniformly most powerful tests, and hence they also enjoy the optimality property of being uniformly most accurate. In addition, these sets possess Bayesian optimal volume properties and represent the first case (to our knowledge) of a frequentist 1 – α confidence set that possesses a Bayesian optimality property. The hypothesis testing problem that generates these sets is similar to that encountered in bioequivalence testing. Our sets are optimal for testing bioequivalence in certain settings; in the case of the normal distribution, the optimal set is a curve known as the limacon of Pascal. We illustrate the use of these curves with a biopharmaceutical example.

Principal Components Regression With Data Chosen Components and Related Methods

Multiple regression with correlated explanatory variables is relevant to a broad range of problems in the physical, chemical, and engineering sciences. Chemometricians in particular have made heavy use of principal components regression and related procedures for predicting a response variable from a large number of highly correlated variables. In this article we develop a general theory for selecting principal components that yield estimates of regression coefficients with low mean squared error. Our numerical results suggest that the theory also can be used to improve partial least squares regression estimators and regression estimators based on rotated principal components. Although our work has been motivated by the statistical genetics problem of mapping quantitative trait loci, the results are applicable to any problem in which estimation of regression coefficients for correlated explanatory variables is of interest.

Is Pitman Closeness a Reasonable Criterion

The criterion of Pitman closeness has been proposed as an alternative comparison criterion to quadratic losses and, more generally, to decision theory. However, it may lead to quite paradoxical phenomena, the most dramatic being a possible nontransitivity. The criterion takes into consideration the joint distribution of the compared estimators, but this consideration may be misleading in the selection of the “best” estimator. We show through examples that this criterion is not consistent with a decision theoretic analysis and that it should be used very cautiously, if ever.

How to Approximate a Histogram by a Normal Density

Abstract Which normal density curve best approximates the sample histogram? The answer suggested here is the normal curve that minimizes the integrated squared deviation between the histogram and the normal curve. A simple computational procedure is described to produce this best-fitting normal density. A few examples are presented to illustrate that this normal curve does indeed provide a visually satisfying fit, one that is better than the traditional , s answer. Some further aspects of this procedure are investigated. In particular it is shown that there is a satisfactory answer that is independent of the bar width of the histogram. It is also noted that this graphical procedure provides highly robust estimates of the sample mean and standard deviation. We demonstrate our technique by using data including Newcombs data of passage time of light and Fishers iris data.

Maximum Likelihood Estimation under Order Restrictions by the Prior Feedback Method

Abstract Algorithms for deriving isotonic regression estimators in order-restricted linear models and more generally restricted maximum likelihood estimators are usually quite dependent on the particular problem considered. We propose here an optimization method based on a sequence of formal Bayes estimates whose variances converge to zero. This method, akin to simulated annealing, can be applied “universally”; that is, as long as these Bayes estimators can be derived by exact computation or Markov chain Monte Carlo sampling approximation. We then give an illustration of our method for two real-life examples.

Testing average equivalence : Finding a compromise between theory and practice

Recently, Brown, Hwang, and Munk (1998) proposed and unbiased test for the average equivalence problem which improves noticeably in power on the standard two one-sided tests procedure. Nevertheless, from a practical point of view there are some objections against the use of this test which are mainly adressed to the ‘unusual’ shape of the critical region. We show that every unbiased test has a critical region with such an ‘unusual’ shape. Therefore, we discuss three (biased) modifications of the unbiased test. We come to the conclusion that a suitable modification represents a good compromise between a most powerful test and a test with an appealing shape of its critical region. In order to perform these tests figures are given containing the rejection region. Finally, we compare all tests in an example from neurophysiology. This shows that it is beneficial to use these improved tests instead of the two one-sided tests procedure.