Osnat Stramer

Langevin Diffusions and Metropolis-Hastings Algorithms

We consider a class of Langevin diffusions with state-dependent volatility. The volatility of the diffusion is chosen so as to make the stationary distribution of the diffusion with respect to its natural clock, a heated version of the stationary density of interest. The motivation behind this construction is the desire to construct uniformly ergodic diffusions with required stationary densities. Discrete time algorithms constructed by Hastings accept reject mechanisms are constructed from discretisations of the algorithms, and the properties of these algorithms are investigated.

Langevin-Type Models II: Self-Targeting Candidates for MCMC Algorithms*

The Metropolis-Hastings algorithm for estimating a distribution π is based on choosing a candidate Markov chain and then accepting or rejecting moves of the candidate to produce a chain known to have π as the invariant measure. The traditional methods use candidates essentially unconnected to π. We show that the class of candidate distributions, developed in Part I (Stramer and Tweedie 1999), which “self-target” towards the high density areas of π, produce Metropolis-Hastings algorithms with convergence rates that appear to be considerably better than those known for the traditional candidate choices, such as random walk. We illustrate this behavior for examples with exponential and polynomial tails, and for a logistic regression model using a Gibbs sampling algorithm. The detailed results are given in one dimension but we indicate how they may extend successfully to higher dimensions.

Langevin-Type Models I: Diffusions with Given Stationary Distributions and their Discretizations*

We describe algorithms for estimating a given measure π known up to a constant of proportionality, based on a large class of diffusions (extending the Langevin model) for which π is invariant. We show that under weak conditions one can choose from this class in such a way that the diffusions converge at exponential rate to π, and one can even ensure that convergence is independent of the starting point of the algorithm. When convergence is less than exponential we show that it is often polynomial at verifiable rates. We then consider methods of discretizing the diffusion in time, and find methods which inherit the convergence rates of the continuous time process. These contrast with the behavior of the naive or Euler discretization, which can behave badly even in simple cases. Our results are described in detail in one dimension only, although extensions to higher dimensions are also briefly described.

On Simulated Likelihood of Discretely Observed Diffusion Processes and Comparison to Closed-Form Approximation

This article focuses on two methods to approximate the log-likelihood of discretely observed univariate diffusions: (1) the simulation approach using a modified Brownian bridge as the importance sampler, and (2) the closed-form approximation approach. For the case of constant volatility, we give a theoretical justification of the modified Brownian bridge sampler by showing that it is exactly a Brownian bridge. We also discuss computational issues in the simulation approach such as accelerating the numerical variance stabilizing transformation, computing derivatives of the simulated log-likelihood, and choosing initial values of parameter estimates. The two approaches are compared in the context of financial applications under a benchmark model which has an unknown transition density and has no analytical variance stabilizing transformation. The closed-form approximation, particularly the second-order closed-form, is found to be computationally efficient and very accurate when the observation frequency is monthly or higher. It is more accurate in the center than in the tails of the transition density. The simulation approach combined with the variance stabilizing transformation is found to be more reliable than the closed-form approximation when the observation frequency is lower. Both methods perform better when the volatility level is lower, but the simulation method is more robust to the volatility level. When applied to two well-known datasets of daily observations, the two methods yield similar parameter estimates in both datasets but slightly different log-likelihoods in the case of higher volatility.

On the approximation of continuous time threshold ARMA processes

Threshold autoregressive (AR) and autoregressive moving average (ARMA) processes with continuous time parameter have been discussed in several recent papers by Brockwellet al. (1991,Statist. Sinica,1, 401–410), Tong and Yeung (1991,Statist. Sinica,1, 411–430), Brockwell and Hyndman (1992,International Journal Forecasting,8, 157–173) and Brockwell (1994,J. Statist. Plann. Inference,39, 291–304). A threshold ARMA process with boundary width 2δ>0 is easy to define in terms of the unique strong solution of a stochastic differential equation whose coefficients are piecewise linear and Lipschitz. The positive boundary-width is a convenient mathematical device to smooth out the coefficient changes at the boundary and hence to ensure the existence and uniqueness of the strong solution of the stochastic differential equation from which the process is derived. In this paper we give a direct definition of a threshold ARMA processes with δ=0 in the important case when only the autoregressive coefficients change with the level of the process. (This of course includes all threshold AR processes with constant scale parameter.) The idea is to express the distributions of the process in terms of the weak solution of a certain stochastic differential equation. It is shown that the joint distributions of this solution with δ=0 are the weak limits as δ ↓ 0 of the distributions of the solution with δ>0. The sense in which the approximating sequence of processes used by Brockwell and Hyndman (1992,International Journal Forecasting,8, 157–173) converges to this weak solution is also investigated. Some numerical examples illustrate the value of the latter approximation in comparison with the more direct representation of the process obtained from the Cameron-Martin-Girsanov formula. It is used in particular to fit continuous-time threshold models to the sunspot and Canadian lynx series.

Bayesian inference for irreducible diffusion processes using the pseudo-marginal approach

In this article we examine two relatively new MCMC methods which allow for Bayesian inference in diffusion models. First, the Monte Carlo within Metropolis (MCWM) algorithm (O’Neil, Balding, Becker, Serola and Mollison, 2000) uses an importance sampling approximation for the likelihood and yields a limiting stationary distribution that can be made arbitrarily “close” to the posterior distribution (MCWM is not a standard Metropolis-Hastings algorithm, however). The second method, described in Beaumont (2003) and generalized in Andrieu and Roberts (2009), introduces auxiliary variables and utilizes a standard Metropolis-Hastings algorithm on the enlarged space; this method preserves the original posterior distribution. When applied to diffusion models, this approach can be viewed as a generalization of the popular data augmentation schemes that sample jointly from the missing paths and the parameters of the diffusion volatility. We show that increasing the number of auxiliary variables dramatically increases the acceptance rates in the MCMC algorithm (compared to basic data augmentation schemes), allowing for rapid convergence and mixing. The efficacy of ourapproach is demonstrated in a simulation study of the Cox-Ingersoll-Ross (CIR) and Heston models, and is applied to two well known datasets.

Continuous-time threshold AR(1) processes

A threshold AR(1) process with boundary width 28 > 0 was defined by Brockwell and Hyndman [5] in terms of the unique strong solution of a stochastic differential equation whose coefficients are piecewise linear and Lipschitz. The positive boundary-width is a convenient mathematical device to smooth out the coefficient changes at the boundary and hence to ensure the existence and uniqueness of the strong solution of the stochastic differential equation from which the process is derived. In this paper we give a direct definition of a threshold AR(1) process with 8 = 0 in terms of the weak solution of a certain stochastic differential equation. Two characterizations of the distributions of the process are investigated. Both express the characteristic function of the transition probability distribution as an explicit functional of standard Brownian motion. It is shown that the joint distributions of this solution with 6 = 0 are the weak limits as 61° of the distributions of the solution with 6>0. The sense in which an approximating sequence of processes used by Brockwell and Hyndman [5] converges to this weak solution is also investigated. Some numerical examples illustrate the value of the latter approximation in comparison with the more direct representation of the process obtained from the Cameron-Martin-Girsanov formula and results of Engelbert and Schmidt [9]. We also derive the stationary distribution (under appropriate assumptions) and investigate stability of these processes.

On Bayesian Analysis of Nonlinear Continuous-Time Autoregression Models

In a daylight enlarger of the fixed focus type which comprises an enclosed image forming area having two arm-holes which allow an operators hands to enter this area there being present a lamphouse and transparency holder placed above the image framing area but outside the enclosed area and an enlarging lens in the transparency holder there being present in the transparency holder between the transparency and the enlarging lens a movable inclined mirror which in a first position when the lamphouse is operating with a transparency in the holder reflects an erect illuminated image of the transparency away from the enlarger so that it can be viewed by the operator of the enlarger, but which in a second position is moved from beneath the transparency so allowing the illuminated image of the transparency to fall on the image framing area, there is provided means manually controllable from inside the enclosed image forming area of the enlarger to cause the inclined mirror to be moved out of the path of the light which passes through the transparency to the enlarging lens and thence to the image projection area so enabling print material present in the image forming area to be exposed.

The local linearization scheme for nonlinear diffusion models with discontinuous coefficients

The local linearization scheme, defined in Shoji and Ozaki [1997, J. Time Ser. Anal. 18, 485-506] for diffusions with smooth coefficients, is studied under some regularity conditions when the coefficients are not necessarily continuous. It is shown that the local scheme converges weakly to the diffusion itself. These results are applied to continuous-time threshold autoregressive moving-average processes and multi-dimensional continuous-time threshold AR models.

On inference for threshold autoregressive models

We provide a complete Bayesian method for analyzing a threshold autoregressive (TAR) model when the order of the model is unknown. Our approach is based on a version (Godsill (2001)) of the reversible jump algorithm of Green (1995), and the method for estimating marginal likelihood from the Metropolis-Hasting algorithm by Chib and Jeliazkov (2001). We illustrate our results with simulated data and the Wolfe’s sunspot data set.