P.M.T. Broersen

Automatic spectral analysis with time series models

The increased computational speed and developments in the robustness of algorithms have created the possibility to identify automatically a well-fitting time series model for stochastic data. It is possible to compute more than 500 models and to select only one, which certainly is one of the better models, if not the very best. That model characterizes the spectral density of the data. Time series models are excellent for random data if the model type and the model order are known. For unknown data characteristics, a large number of candidate models have to be computed. This necessarily includes too low or too high model orders and models of the wrong types, thus requiring robust estimation methods. The computer selects a model order for each of the three model types. From those three, the model type with the smallest expectation of the prediction error is selected. That unique selected model includes precisely the statistically significant details that are present in the data.

Finite sample criteria for autoregressive order selection

The quality of selected AR models depends on the true process in the finite sample practice, on the number of observations, on the estimation algorithm, and on the order selection criterion. Samples are considered to be finite if the maximum candidate model order for selection is greater than N/10, where N denotes the number of observations. Finite sample formulae give empirical approximations for the statistical average of the residual energy and of the squared error of prediction for several autoregressive estimation algorithms. This leads to finite sample criteria for order selection that depend on the estimation method. The special finite sample information criterion (FSIC) and combined information criterion (CIC) are necessary because of the increase of the variance of the residual energy for higher model orders that has not been accounted for in other criteria. Only the expectation of the logarithm of the residual energy, as a function of the model order, has been the basis for the previous classes of asymptotical and finite sample criteria. However, the behavior of the variance causes an undesirable tendency to select very high model orders without the special precautions of FSIC or CIC.

The quality of models for ARMA processes

The model error (ME) is an objective measure for assessing the quality of different models of a given ARMA process. The expression for ME can be evaluated easily in the time domain. This quality measure for known and given processes is necessary for an objective comparison of the performance of estimation algorithms and of order selection criteria.

Autoregressive spectral estimation by application of the Burg algorithm to irregularly sampled data

Many methods have been developed for spectral analysis of irregularly sampled data. Currently, popular methods such as Lomb-Scargle and resampling tend to be biased at higher frequencies. Slotting methods fail to consistently produce a spectrum that is positive for all frequencies. In this paper, a new estimator is introduced that applies the Burg algorithm for autoregressive spectral estimation to unevenly spaced data. The new estimator can be perceived as searching for sequences of data that are almost equidistant, and then analyzing those sequences using the Burg algorithm for segments. The estimated spectrum is guaranteed to be positive. If a sufficiently large data set is available, results can be accurate up to relatively high frequencies.

Hunting of Evaporators Controlled by a Thermostatic Expansion Valve

Evaporators controlled by a thermostatic expansion valve can exhibit an undesirable oscillating behavior, known as hunting. The equations describing the hunting can be characterized by an open-loop transfer function, the coefficients of which are the physical parameters of a refrigeration system. Several known experimental facts are explained theoretically by means of this transfer function, which also provides a starting point for finding new methods to improve the behavior. A particular improvement was verified experimentally.

Facts and fiction in spectral analysis

This analysis is limited to the spectral analysis of stationary stochastic processes with unknown spectral density. The main spectral estimation methods are: parametric with time series models, or nonparametric with a windowed periodogram. A single time series model will be chosen with a statistical criterion from three previously estimated and selected models: the best autoregressive (AR) model, the best moving average (MA) model, and the best combined ARMA model. The accuracy of the spectrum, computed from this single selected time series model, is compared with the accuracy of some windowed periodogram estimates. The time series model generally gives a spectrum that is better than the best possible windowed periodogram. It is a fact that a single good time series model can be selected automatically for statistical data with unknown spectral density. It is fiction that objective choices between windowed periodograms can be made.

Order selection for vector autoregressive models

Order-selection criteria for vector autoregressive (AR) modeling are discussed. The performance of an order-selection criterion is optimal if the model of the selected order is the most accurate model in the considered set of estimated models: here vector AR models. Suboptimal performance can be a result of underfit or overfit. The Akaike (1969) information criterion (AIC) is an asymptotically unbiased estimator of the Kullback-Leibler discrepancy (KLD) that can be used as an order-selection criterion. AIC is known to suffer from overfit: The selected model order can be greater than the optimal model order. Two causes of overfit are finite sample effects and asymptotic effects. As a consequence of finite sample effects, AIC underestimates the KLD for higher model orders, leading to overfit. Asymptotically, overfit is the result of statistical variations in the order-selection criterion. To derive an accurate order-selection criterion, both causes of overfit have to be addressed. Moreover, the cost of underfit has to be taken into account. The combined information criterion (CIC) for vector signals is robust to finite sample effects and has the optimal asymptotic penalty factor. This penalty factor is the result of a tradeoff of underfit and overfit. The optimal penalty factor depends on the number of estimated parameters per model order. The CIC is compared to other criteria such as the AIC, the corrected Akaike information criterion (AICc), and the consistent minimum description length (MDL).

Autoregressive spectral analysis when observations are missing

A new missing data algorithm ARFIL gives good results in spectral estimation. The log likelihood of a multivariate Gaussian random variable can always be written as a sum of conditional log likelihoods. For a complete set of autoregressive AR(p) data the best predictor in the likelihood requires only p previous observations. If observations are missing, the best AR predictor in the likelihood will in general include all previous observations. Using only those observations that fall within a finite time interval will approximate this likelihood. The resulting non-linear estimation algorithm requires no user provided starting values. In various simulations, the spectral accuracy of robust maximum likelihood methods was much better than the accuracy of other spectral estimates for randomly missing data.

Autoregressive model orders for Durbin's MA and ARMA estimators

Durbins methods (1959, 1960) for moving average (MA) and autoregressive-moving average (ARMA) estimation use the parameters of a long AR model to compute the MA parameters. Linear regression theory is applied to find the best AR order. This yields two different orders: one for the best predicting AR model and another one for the long AR model with the best parameter accuracy, as intermediate for Durbins estimates. Both orders increase with the sample size and have no finite limiting value.

Error measures for resampled irregular data

With resampling, a regularly sampled signal is extracted from observations which are irregularly spaced in time. Resampling methods can be divided into simple and complex methods. Simple methods such as Sample and Hold (S and H) and Nearest Neighbor Resampling (NNR) use only one irregular sample for one resampled observation. A theoretical analysis of the simple methods is given. The various resampling methods are compared using the new error measure SD/sub T/: the spectral distortion at interval T. SD/sub T/ is zero when the time domain properties of the signal are conserved. Using the time domain approach, an antialiasing filter is no longer necessary: the best possible estimates are obtained by using the data themselves. In the frequency domain approach, both allowing aliasing and applying antialiasing leads to distortions in the spectrum. The error measure SD/sub T/ has been compared to the reconstruction error. A small reconstruction error does not necessarily result in an accurate estimate of the statistical signal properties as expressed by SD/sub T/.