Richard L. Tweedie

Stability of Markovian processes. III: Foster-Lyapunov criteria for continuous-time processes

In Part I we developed stability concepts for discrete chains, together with Foster-Lyapunov criteria for them to hold. Part II was devoted to developing related stability concepts for continuous-time processes. In this paper we develop criteria for these forms of stability for continuous-parameter Markovian processes on general state spaces, based on Foster-Lyapunov inequalities for the extended generator. Such test function criteria are found for non-explosivity, non-evanescence, Harris recurrence, and positive Harris recurrence. These results are proved by systematic application of Dynkins formula. We also strengthen known ergodic theorems, and especially exponential ergodic results, for continuous-time processes. In particular we are able to show that the test function approach provides a criterion for f-norm convergence, and bounding constants for such convergence in the exponential ergodic case. We apply the criteria to several specific processes, including linear stochastic systems under non-linear feedback, work-modulated queues, general release storage

Stability of Markovian processes II: continuous-time processes and sampled chains

In this paper we extend the results of Meyn and Tweedie (1992b) from discrete-time parameter to continuous-parameter Markovian processes Φ evolving on a topological space. We consider a number of stability concepts for such processes in terms of the topology of the space, and prove connections between these and standard probabilistic recurrence concepts. We show that these structural results hold for a major class of processes (processes with continuous components) in a manner analogous to discrete-time results, and that complex operations research models such as storage models with state-dependent release rules, or diffusion models such as those with hypoelliptic generators, have this property

Criteria for classifying general Markov chains

The aim of this paper is to present a comprehensive set of criteria for classifying as recurrent, transient, null or positive the sets visited by a general state space Markov chain. When the chain is irreducible in some sense, these then provide criteria for classifying the chain itself, provided the sets considered actually reflect the status of the chain as a whole. The first part of the paper is concerned with the connections between various definitions of recurrence, transience, nullity and positivity for sets and for irreducible chains; here we also elaborate the idea of status sets for irreducible chains. In the second part we give our criteria for classifying sets. When the state space is countable, our results for recurrence, transience and positivity reduce to the classical work of Foster (1953); for continuous-valued chains they extend results of Lamperti (1960), (1963); for general spaces the positivity and recurrence criteria strengthen those of Tweedie (1975b). MARKOV CHAIN; RECURRENCE; TRANSIENCE; POSITIVE RECURRENCE; ERGODICITY; NULL RECURRENCE; STATIONARY MEASURE; INVARIANT MEASURE; IRREDUCIBILITY; STATUS SET; MEAN DRIFT

Geometric convergence rates for stochastically ordered Markov chains

Let {Φn} be a Markov chain on the state space [0, ∞ that is stochastically ordered in its initial state; that is, a stochastically larger initial state produces a stochastically larger chain at all other times. Examples of such chains include random walks, the number of customers in various queueing systems, and a plethora of storage processes. A large body of recent literature concentrates on establishing geometric ergodicity of {Φn} in total variation; that is, proving the existence of a limiting probability measure π and a number r > 1 such that

Lung cancer and passive smoking: reconciling the biochemical and epidemiological approaches.

\lim_{n\to \infty} r^n \sup_{A\in {\cal B}[0, \infty} \vert P_x [\Phi_n\in A]-\piA\vert = 0

Exponential decay and ergodicity of general Markov processes and their discrete skeletons

for every deterministic initial state Φ0 ≡ x. We seek to identity the largest r that satisfies this relationship. A dependent sample path coupling and a Foster-Lyapunov drift inequality are used to derive convergence rate bounds; we then show that the bounds obtained are frequently the best possible. Application of the methods to queues and random walks are included.

Linear functionals and Markov chains associated with Dirichlet processes

Meta-analysis enables researchers to combine the results of several studies to assess the information they provide as a whole. It has been used to give a systematic overview of many areas in which data on a possible association between an exposure and an outcome have been collected in a number of studies but where the overall picture remains obscure, both as to the existence or size of the effect. This paper outlines some innovations in meta-analysis, based on using Markov chain Monte Carlo (MCMC) techniques for implementing Bayesian hierarchical models, and compares these with a more well-known random effects (RE) model. The new techniques allow different aspects of variation to be incorporated into descriptions of the association, and in particular enable researchers to better quantify differences between studies. Both the classical and Bayesian methods are applied, in this paper, to the current collection of studies of the association between incidence of lung cancer in female never-smokers and exposure to environmental tobacco smoke (ETS), both in the home through spousal smoking and in the workplace. In this paper it is demonstrated that compared with the RE model, the Bayesian methods: (a) allow more detailed modeling of study heterogeneity to be incorporated; (b) are relatively robust against a wide choice of specifications of such information on heterogeneity; (c) allow for more detailed and satisfactory statements to be made, not only about the overall risk but about the individual studies, on the basis of the combined information. For the workplace exposure data set, the Bayesian methods give a somewhat lower overall estimate of relative risk of lung cancer associated with ETS, indicating the care that needs to be taken in using point estimates based on any one method of analysis. On the larger spousal data set the methods give similar answers. Some of the other concerns with meta-analysis are also considered. These include: consistency between different geographic areas (Asia and the United States), and our studies show that Bayesian methods permit an account of the overall picture to be taken, thus improving the ability to estimate accurately in the subgroups; and publication bias which, as shown with the spousal exposure data, may lead to an inflated excess risk.

Markov chain Monte Carlo estimation of the law of the mean of a Dirichlet process

The accurate determination of exposure to environmental tobacco smoke is notoriously difficult. There have been to date two approaches to determining this exposure in the study of association of passive smoking and lung cancer: the biochemical approach, using cotinine in the main as a marker, and the epidemiological approach. Typically results of the former have yielded much lower relative risk than the latter, and have tended to be ignored in favour of the latter, although there has been considerable debate as to the logical basis for this. We settle this question by showing that, using the epidemiologically based meta-analysis technique of Wald et al. (1986), and misclassification models in the EPA Draft Review (1990), one arrives using all current studies at a result which is virtually identical with the biochemically-based conclusions of Darby and Pike (1988) or Repace and Lowry (1990). The conduct of this meta-analysis itself raises a number of important methodological questions, including the validity of inclusion of studies, the use of estimates adjusted for covariates, and the statistical significance of estimates based on meta-analysis of the epidemiological data. The best estimate of relative risk from spousal smoking is shown to be approximately 1.05-1.10, based on either of these approaches; but it is suggested that considerable extra work is needed to establish whether this is significantly raised.