Genshiro Kitagawa

Monte Carlo Filter and Smoother for Non-Gaussian Nonlinear State Space Models

Abstract A new algorithm for the prediction, filtering, and smoothing of non-Gaussian nonlinear state space models is shown. The algorithm is based on a Monte Carlo method in which successive prediction, filtering (and subsequently smoothing), conditional probability density functions are approximated by many of their realizations. The particular contribution of this algorithm is that it can be applied to a broad class of nonlinear non-Gaussian higher dimensional state space models on the provision that the dimensions of the system noise and the observation noise are relatively low. Several numerical examples are shown.

Non-Gaussian State—Space Modeling of Nonstationary Time Series

Abstract A non-Gaussian state—space approach to the modeling of nonstationary time series is shown. The model is expressed in state—space form, where the system noise and the observational noise are not necessarily Gaussian. Recursive formulas of prediction, filtering, and smoothing for the state estimation and identification of the non-Gaussian state—space model are given. Also given is a numerical method based on piecewise linear approximation to the density functions for realizing these formulas. Significant merits of non-Gaussian modeling and the wide range of applicability of the method are illustrated by some numerical examples. A typical application of this non-Gaussian modeling is the smoothing of a time series that has mean value function with both abrupt and gradual changes. Simple Gaussian state—space modeling is not adequate for this situation. Here the model with small system noise variance cannot detect jump, whereas the one with large system noise variance yields unfavorable wiggle. To work...

Information Criteria and Statistical Modeling

The Akaike information criterion (AIC) derived as an estimator of the Kullback-Leibler information discrepancy provides a useful tool for evaluating statistical models, and numerous successful applications of the AIC have been reported in various fields of natural sciences, social sciences and engineering. One of the main objectives of this book is to provide comprehensive explanations of the concepts and derivations of the AIC and related criteria, including Schwarzs Bayesian information criterion (BIC), together with a wide range of practical examples of model selection and evaluation criteria. A secondary objective is to provide a theoretical basis for the analysis and extension of information criteria via a statistical functional approach. A generalized information criterion (GIC) and a bootstrap information criterion are presented, which provide unified tools for modeling and model evaluation for a diverse range of models, including various types of nonlinear models and model estimation procedures such as robust estimation, the maximum penalized likelihood method and a Bayesian approach.

A self-organizing state-space model

A self-organizing filter and smoother for the general nonlinear non-Gaussian state-space model is proposed. An expanded state-space model is defined by augmenting the state vector with the unknown parameters of the original state-space model. The state of the augmented state-space model, and hence the state and the parameters of the original state-space model, are estimated simultaneously by either a non-Gaussian filter/smoother or a Monte Carlo filter/smoother. In contrast to maximum likelihood estimation of model parameters in ordinary state-space modeling, for which the recursive filter computation has to be done many times, model parameter estimation in the proposed self-organizing filter/smoother is achieved with only two passes of the recursive filter and smoother operations. Examples such as automatic tuning of dispersion and the shape parameters, adaptation to changes of the amplitude of a signal in seismic data, state estimation for a nonlinear state space model with unknown parameters. and seasonal adjustment with a nonlinear model with changing variance parameters are shown to exemplify the usefulness of the proposed method.

A smoothness priors time-varying AR coefficient modeling of nonstationary covariance time series

A smoothness priors time varying AR coefficient model approach for the modeling of nonstationary in the covariance time series is shown. Smoothness priors in the form of a difference equation constraint excited by an independent white noise are imposed on each AR coefficient. The unknown white noise variances are hyperparameters of the AR coefficient distribution. The critical computation is of the likelihood of the hyperparameters of the Bayesian model. This computation is facilitated by a state-space representation Kalman filter implementation. The best difference equation order-best AR model order-best hyperparameter model locally in time is selected using the minimum AIC method. Also, an instantaneous spectral density is defined in terms of the instantaneous AR model coefficients and a smoothed estimate of the instantaneous time series variance. An earthquake record is analyzed. The changing spectral analysis of the original data and of simulations from a time varying AR coefficient model of that data are shown.

A Smoothness Priors–State Space Modeling of Time Series with Trend and Seasonality

Abstract A smoothness priors modeling of time series with trends and seasonalities is shown. An observed time series is decomposed into local polynomial trend, seasonal, globally stationary autoregressive and observation error components. Each component is characterized by an unknown variance–white noise perturbed difference equation constraint. The constraints or Bayesian smoothness priors are expressed in state space model form. Trading day factors are also incorporated in the model. A Kalman predictor yields the likelihood for the unknown variances (hyperparameters). Likelihoods are computed for different constraint order models in different subsets of constraint equation model classes. Akaikes minimum AIC procedure is used to select the best model fitted to the data within and between the alternative model classes. Smoothing is achieved by using a fixed-interval smoother algorithm. Examples are shown.

A procedure for the modeling of non-stationary time series

SummaryA minimum AIC procedure for the fitting of a locally stationary autoregressive model is proposed. The least squares computation for the procedure is realized by using the Householder transformation which makes the procedure computationally more flexible and efficient than the one originally proposed by Ozaki and Tong.

Bootstrapping Log Likelihood and EIC, an Extension of AIC

Akaike (1973, 2nd International Symposium on Information Theory, 267-281,Akademiai Kiado, Budapest) proposed AIC as an estimate of the expected loglikelihood to evaluate the goodness of models fitted to a given set of data.The introduction of AIC has greatly widened the range of application ofstatistical methods. However, its limit lies in the point that it can beapplied only to the cases where the parameter estimation are performed bythe maximum likelihood method. The derivation of AIC is based on theassessment of the effect of data fluctuation through the asymptoticnormality of MLE. In this paper we propose a new information criterion EICwhich is constructed by employing the bootstrap method to simulate the datafluctuation. The new information criterion, EIC, is regarded as an extensionof AIC. The performance of EIC is demonstrated by some numerical examples.

Theory and Methods

Abstract A self-organizing filter and smoother for the general nonlinear non-Gaussian state-space model is proposed. An expanded state-space model is defined by augmenting the state vector with the unknown parameters of the original state-space model. The state of the augmented state-space model, and hence the state and the parameters of the original state-space model, are estimated simultaneously by either a non-Gaussian filter/smoother or a Monte Carlo filter/smoother. In contrast to maximum likelihood estimation of model parameters in ordinary state-space modeling, for which the recursive filter computation has to be done many times, model parameter estimation in the proposed self-organizing filter/smoother is achieved with only two passes of the recursive filter and smoother operations. Examples such as automatic tuning of dispersion and the shape parameters, adaptation to changes of the amplitude of a signal in seismic data, state estimation for a nonlinear state space model with unknown parameters, ...

The Prediction of Time Series With Trends and Seasonalities

The modeling and prediction of time series with trend and seasonal mean value functions and stationary covariances is approached from a maximization of the expected entropy of the predictive distribution interpretation of Akaikes minimum AIC procedure. The AIC criterion best one-step-ahead and best twelvestep-ahead prediction models are different. They exhibit the relative optimality properties for which they were designed. The results are related to open questions on optimal trend estimation and optimal seasonal adjustment of time series.