Zhengyuan Zhu

Spatial sampling design for prediction with estimated parameters

We study spatial sampling design for prediction of stationary isotropic Gaussian processes with estimated parameters of the covariance function. The key issue is how to incorporate the parameter uncertainty into design criteria to correctly represent the uncertainty in prediction. Several possible design criteria are discussed that incorporate the parameter uncertainty. A simulated annealing algorithm is employed to search for the optimal design of small sample size and a two-step algorithm is proposed for moderately large sample sizes. Simulation results are presented for the Matérn class of covariance functions. An example of redesigning the air monitoring network in EPA Region 5 for monitoring sulfur dioxide is given to illustrate the possible differences our proposed design criterion can make in practice.

Long-range dependence analysis of Internet traffic

Long-range-dependent time series are endemic in the statistical analysis of Internet traffic. The Hurst parameter provides a good summary of important self-similar scaling properties. We compare a number of different Hurst parameter estimation methods and some important variations. This is done in the context of a wide range of simulated, laboratory-generated, and real data sets. Important differences between the methods are highlighted. Deep insights are revealed on how well the laboratory data mimic the real data. Non-stationarities, which are local in time, are seen to be central issues and lead to both conceptual and practical recommendations.

Associations between ozone and morbidity using the Spatial Synoptic Classification system

BackgroundSynoptic circulation patterns (large-scale tropospheric motion systems) affect air pollution and, potentially, air-pollution-morbidity associations. We evaluated the effect of synoptic circulation patterns (air masses) on the association between ozone and hospital admissions for asthma and myocardial infarction (MI) among adults in North Carolina.MethodsDaily surface meteorology data (including precipitation, wind speed, and dew point) for five selected cities in North Carolina were obtained from the U.S. EPA Air Quality System (AQS), which were in turn based on data from the National Climatic Data Center of the National Oceanic and Atmospheric Administration. We used the Spatial Synoptic Classification system to classify each day of the 9-year period from 1996 through 2004 into one of seven different air mass types: dry polar, dry moderate, dry tropical, moist polar, moist moderate, moist tropical, or transitional. Daily 24-hour maximum 1-hour ambient concentrations of ozone were obtained from the AQS. Asthma and MI hospital admissions data for the 9-year period were obtained from the North Carolina Department of Health and Human Services. Generalized linear models were used to assess the association of the hospitalizations with ozone concentrations and specific air mass types, using pollutant lags of 0 to 5 days. We examined the effect across cities on days with the same air mass type. In all models we adjusted for dew point and day-of-the-week effects related to hospital admissions.ResultsOzone was associated with asthma under dry tropical (1- to 5-day lags), transitional (3- and 4-day lags), and extreme moist tropical (0-day lag) air masses. Ozone was associated with MI only under the extreme moist tropical (5-day lag) air masses.ConclusionsElevated ozone levels are associated with dry tropical, dry moderate, and moist tropical air masses, with the highest ozone levels being associated with the dry tropical air mass. Certain synoptic circulation patterns/air masses in conjunction with ambient ozone levels were associated with increased asthma and MI hospitalizations.

Accounting for spatial correlation in the scan statistic

The spatial scan statistic is widely used in epidemiology and medical studies as a tool to identify hotspots of diseases. The classical spatial scan statistic assumes the number of disease cases in different locations have independent Poisson distributions, while in practice the data may exhibit overdispersion and spatial correlation. In this work, we examine the behavior of the spatial scan statistic when overdispersion and spatial correlation are present, and propose a modified spatial scan statistic to account for that. Some theoretical results are provided to demonstrate that ignoring the overdispersion and spatial correlation leads to an increased rate of false positives, which is verified through a simulation study. Simulation studies also show that our modified procedure can substantially reduce the rate of false alarms. Two data examples involving brain cancer cases in New Mexico and chickenpox incidence data in France are used to illustrate the practical relevance of the modified procedure.

Robust estimation of the self-similarity parameter in network traffic using wavelet transform

This article studies the problem of estimating the self-similarity parameter of network traffic traces. A robust wavelet-based procedure is proposed for this estimation task of deriving estimates that are less sensitive to some commonly encountered non-stationary traffic conditions, such as sudden level shifts and breaks. Two main ingredients of the proposed procedure are: (i) the application of a robust regression technique for estimating the parameter from the wavelet coefficients of the traces, and (ii) the proposal of an automatic level shift removal algorithm for removing sudden jumps in the traces. Simulation experiments are conducted to compare the proposed estimator with existing wavelet-based estimators. The proposed estimator is also applied to real traces obtained from the Abilene Backbone Network and a university campus network. Both results from simulated experiments and real trace applications suggest that the proposed estimator is superior.

Semiparametric Estimation of Spectral Density With Irregular Observations

We propose a semiparametric method for estimating spectral densities of isotropic Gaussian processes with scattered data. The spectral density function (Fourier transform of the covariance function) is modeled as a linear combination of B-splines up to a cutoff frequency and, from this point, a truncated algebraic tail. We calculate an analytic expression for the covariance function and tackle several numerical issues that arise when calculating the likelihood. The parameters are estimated by maximizing the likelihood using the simulated annealing method. Our method directly estimates the tail behavior of the spectral density, which has the greatest impact on interpolation properties. The use of the likelihood in parameter estimation takes the correlations between observations fully into account. We compare our method with a kernel method proposed by Hall et al. and a parametric method using the Matérn model. Simulation results show that our method outperforms the other two by several criteria. Application to rainfall data shows that our method outperforms the kernel method.

Estimating spatial covariance using penalised likelihood with weighted L1 penalty

In spatial statistics, the estimation of covariance matrices is of great importance because of its role in spatial prediction and design. In this paper, we propose a penalised likelihood approach with weighted L 1 regularisation to estimate the covariance matrix for spatial Gaussian Markov random field models with unspecified neighbourhood structures. A new algorithm for ordering spatial points is proposed such that the corresponding precision matrix can be estimated more effectively. Furthermore, we develop an efficient algorithm to minimise the penalised likelihood via a novel usage of the regularised solution path algorithm, which does not require the use of iterative algorithms. By exploiting the sparsity structure in the precision matrix, we show that the LASSO type of approach gives improved covariance estimators measured by several criteria. Asymptotic properties of our proposed estimator are derived. Both our simulated examples and an application to the rainfall data set show that the proposed method performs competitively.

Singular Value Decomposition and Its Visualization

Singular value decomposition (SVD) is a useful tool in functional data analysis (FDA). Compared to principal component analysis (PCA), SVD is more fundamental, because SVD simultaneously provides the PCAs in both row and column spaces. We compare SVD and PCA from the FDA view point, and extend the usual SVD to variations by considering different centerings. A generalized scree plot is proposed to select an appropriate centering in practice. Several useful matrix views of the SVD components are introduced to explore different features in data, including SVD surface plots, image plots, curve movies, and rotation movies. These methods visualize both column and row information of a two-way matrix simultaneously, relate the matrix to relevant curves, show local variations, and highlight interactions between columns and rows. Several toy examples are designed to compare the different variations of SVD, and real data examples are used to illustrate the usefulness of the visualization methods.

Estimation and Prediction of a Class of Convolution-Based Spatial Nonstationary Models for Large Spatial Data

In this article we address two important issues common to the analysis of large spatial datasets. One is the modeling of nonstationarity, and the other is the computational challenges in doing likelihood-based estimation and kriging prediction. We model the spatial process as a convolution of independent Gaussian processes, with the spatially varying kernel function given by the modified Bessel functions. This is a generalization of the process-convolution approach of Higdon, Swall, and Kern (1999), who used the Gaussian kernel to obtain a closed-form nonstationary covariance function. Our model can produce processes with richer local behavior similar to the processes with the Matérn class of covariance functions. Because the covariance function of our model does not have a closed-form expression, direct estimation and spatial prediction using kriging is infeasible for large datasets. Efficient algorithms for parameter estimation and spatial prediction are proposed and implemented. We compare our method with methods based on stationary model and moving window kriging. Simulation results and application to a rainfall dataset show that our method has better prediction performance. Supplemental materials for the article are available online.

Efficient estimation and prediction for the Bayesian binary spatial model with flexible link functions

Spatial generalized linear mixed models (SGLMMs) are popular models for spatial data with a non-Gaussian response. Binomial SGLMMs with logit or probit link functions are often used to model spatially dependent binomial random variables. It is known that for independent binomial data, the robit regression model provides a more robust (against extreme observations) alternative to the more popular logistic and probit models. In this article, we introduce a Bayesian spatial robit model for spatially dependent binomial data. Since constructing a meaningful prior on the link function parameter as well as the spatial correlation parameters in SGLMMs is difficult, we propose an empirical Bayes (EB) approach for the estimation of these parameters as well as for the prediction of the random effects. The EB methodology is implemented by efficient importance sampling methods based on Markov chain Monte Carlo (MCMC) algorithms. Our simulation study shows that the robit model is robust against model misspecification, and our EB method results in estimates with less bias than full Bayesian (FB) analysis. The methodology is applied to a Celastrus Orbiculatus data, and a Rhizoctonia root data. For the former, which is known to contain outlying observations, the robit model is shown to do better for predicting the spatial distribution of an invasive species. For the latter, our approach is doing as well as the classical models for predicting the disease severity for a root disease, as the probit link is shown to be appropriate. Though this article is written for Binomial SGLMMs for brevity, the EB methodology is more general and can be applied to other types of SGLMMs. In the accompanying R package geoBayes, implementations for other SGLMMs such as Poisson and Gamma SGLMMs are provided.