William H. Swallow

Monte Carlo Comparison of ANOVA, MIVQUE, REML, and ML Estimators of Variance Components

For the one-way classification random model with unbalanced data, we compare five estimators of σ2 a and σ2 e , the among- and within-treatments variance components: analysis of variance (ANOVA), maximum likelihood (ML), restricted maximum likelihood (REML), and two minimum variance quadratic unbiased (MIVQUE) estimators. MIVQUE(0) is MIVQUE with a priori values = 0 and = 1; MIVQUE(A) is MIVQUE with the ANOVA estimates used as a prioris, We enforce nonnegativity for all estimators, setting any negative estimate to zero in accord with usual practice. The estimators are compared through their biases and MSEs, estimated by Monte Carlo simulation. Our results indicate that the ANOVA estimators perform well, except with seriously unbalanced data when σ2 a /σ2 e > 1; ML is excellent when σ2 a /σ2 e < 0.5, and MIVQUE(A) is adequate; further iteration to the REML estimates is unnecessary. When σ2 a /σ2 e ≥ 1, MIVQUE(0) (the default for SASS PROCEDURE VARCOMP) is poor for estimating σ2 a and very poor for σ2 e ,...

Using Group Testing to Estimate a Proportion, and to Test the Binomial Model

Group testing has been extensively studied as an efficient way to classify units as defective or satisfactory when the proportion (p) of defectives is small. It can also be used to estimate p, often substantially reducing the mean squared error (MSE) of p and cost per unit information. Group testing is useful for larger p in the estimation problem than in the classification problem, but for larger p more care must be taken in choosing the group size (k); k being too large not only increases MSE (p), but adversely affects the robustness of p to both errors in testing (misclassification) and errors in the assumed binomial model. Procedures that retest units from defective groups, if even feasible, are shown to reduce cost per unit information very little in the estimation problem, but can provide useful information for testing the model. Methods are given for using data from tests of unequal-sized groups to estimate p and for testing the validity of the binomial model.

A Review of the Development and Application of Recursive Residuals in Linear Models

Abstract Recursive residuals have been shown to be useful in a variety of applications in linear models. Unlike the more familiar ordinary least squares residuals or studentized residuals, recursive residuals are independent as well as homoscedastic under the model. Their independence is particularly appealing for use in developing test statistics. They are not uniquely defined; their values depend on the order in which they are calculated, although their properties do not. In some applications one can exploit this order dependence, coupled with the fact that they are in clear one-to-one correspondence with the observations for which they are calculated. Uses for recursive residuals have been suggested in almost all areas of regression model validation. Regression diagnostics have been constructed from recursive residuals for detecting serial correlation, heteroscedasticity, functional misspecification, and structural change. Other statistics based on recursive residuals have focused on detection of outli...

A two-stage adaptive group-testing procedure for estimating small proportions

Abstract A method for adaptively estimating a proportion p using group-testing procedures is presented and analyzed, with emphasis placed on a two-stage procedure. This estimator is compared to the usual group-testing estimator via asymptotic and small-sample relative efficiency. The adaptive estimator is generally recommended with the restriction that adaptations (i.e., adjustments of group size) are based on at least 10 measurements/responses. But if one is confident that one has a “very good” a priori estimate of p, then the usual estimator is recommended. Monte Carlo simulations show that a normal approximation is adequate for generating confidence intervals with confidence coefficients as high as .95.

Using recursive residuals, calculated on adaptively-ordered observations, to identify outliers in linear regression

A new procedure for identifying outliers or influential observations is proposed. The procedure uses recursive residuals, calculated on observations that have been ordered according to their Studentized residuals, values of Cooks D, or another regression diagnostic of the users choice. Under the model, these recursive residuals, appropriately standardized, have approximate Students t distributions. Thus, convenient critical values are available for deciding which observations merit scrutiny and, perhaps, special treatment. The power of the test procedure to identify one or more outliers is investigated through simulation, and its dependence on the number and configuration of the outliers, that is, their placement with respect to the main body of the data, explored. The proposed procedure and two variations of it, also based on these recursive residuals, are compared with alternatives based on internally or externally Studentized residuals. The use of recursive residuals, calculated on adaptively-ordered observations, increases power and helps to combat the masking of one outlier by another when multiple outliers are present in configurations that create masking.

A Monte Carlo comparison of five procedures for identifying outliers in linear regression

Five procedures for detecting outliers in linear regression are compared: sequential testing of the maximum internally studentized residual or maximum externally studentized (cross-validatory) residual, Marasinghes multistage procedure, and two procedures based on recursive residuals, calculated on adaptively-ordered observations. All of these procedures initially test a no-outliers hypothesis, and they have an underlying unity in their general approach to the outlier identification problem. Which procedure is most effective depends on the number and placement of outliers in the data. The multistage procedure is very effective in some cases, but requires prespecifying a value k, the maximum number of outliers one can then detect; the procedure can suffer severely if the chosen value for k is either larger or smaller than the number of outliers actually in the data.

OPTIMUM PLOT SIZE DETERMINATION AND ITS APPLICATION TO CUCUMBER YIELD TRIALS

SummaryMethods of estimating Smiths b and, thereby, optimum plot size are compared from a theoretical viewpoint. For estimating b, generalized least squares is recommended over Smiths (1938) original method and other methods because the points used to fit the required regression are correlated and have unequal variances.Optimum plot size for once-over-harvest trials measuring yield (as number of fruits per plot) of pickling and frest-market cucumbers (Cucumis sativus L.) was estimated to be 0.7 to 3.8 m2 (0.5 to 2.5 m of row for rows 1.5 m apart) for conventional harvesting, and 1.0 to 5.6 m2 (0.7 to 3.7 m of row) for simulated harvesting using paraquat to defoliate plots before evaluation. Estimates of optimum plot size were calculated from a number of uniformity trials differing in year (1982 or 1983), planting date (early or late), and field. The estimates were sufficiently stable to suggest that they have useful generality.For multiple-harvest yield trials, optimum plot sizes for determining yield of pickling (expressed in

Using Robust Scale Estimates in Detecting Multiple Outliers in Linear Regression

/ha or q/ha) or fresh-market cucumbers (i.e. USDA Fancy and No. 1 grade fruit combined or USDA Fancy, No. 1, and No. 2 grade fruit combined, in q/ha) were estimated from experimental data to be 6.4 to 10.3 m2 (4.3 to 6.8 m of row).

Optimum allocation of plots to years, seasons, locations, and replications, and its application to once-over-harvest cucumber trials

SUMMARY Statistics for detecting outliers generally suffer from masking when multiple outliers are present. One aspect of this masking is inflation by the outliers of estimates of scale. This shrinks test statistics and results in loss of power to identify the outliers. Two familiar robust scale estimators are considered: the interquartile range (IR) and the median absolute deviation from the median (MAD). They are used here to scale statistics both for testing individual observations and for testing a no-outliers hypothesis. Some of these statistics use ordinary least squares residuals, others use recursive residuals calculated on adaptively ordered observations. The more severe the masking problem, the more advantageous robust scale estimation was found to be. IR and MAD worked equally well. Test statistics based on the recursive residuals were more powerful than those based on ordinary residuals.

Variability of flowering and 2n pollen production in diploid potatoes under high temperatures

SummaryLarge experiments and breeding trials are often conducted over years, seasons (or planting dates), and locations, and with replication (blocks). This is costly and time-consuming, but it is usually deemed necessary to sample a range of environments. In this paper, we describe a general approach to optimum allocation of sampling effort, and apply it to once-over-harvest cucumber trials. Two criteria for optimality are considered: minimizing the variance of a genotype (or treatment) mean, and minimizing cost per unit information. Costs could include penalties for delaying a breeding program. Thus, costs may depend on the goal, as well as the size, of the experiment or breeding trial.We found that efficient allocation of resources favors using more years and/or season, with fewer locations and/or replications. Using more years with fewer locations and/or replications is suggested when genotypes are to be evaluated by yield alone. When both yield and quality variables are of interest, as is likely, using more seasons with fewer locations and/or replications is recommended.