Zdravko I. Botev

Kernel density estimation via diffusion

We present a new adaptive kernel density estimator based on linear diffusion processes. The proposed estimator builds on existing ideas for adaptive smoothing by incorporating information from a pilot density estimate. In addition, we propose a new plug-in bandwidth selection method that is free from the arbitrary normal reference rules used by existing methods. We present simulation examples in which the proposed approach outperforms existing methods in terms of accuracy and reliability.

Efficient Monte Carlo simulation via the generalized splitting method

We describe a new Monte Carlo algorithm for the consistent and unbiased estimation of multidimensional integrals and the efficient sampling from multidimensional densities. The algorithm is inspired by the classical splitting method and can be applied to general static simulation models. We provide examples from rare-event probability estimation, counting, and sampling, demonstrating that the proposed method can outperform existing Markov chain sampling methods in terms of convergence speed and accuracy.

Global likelihood optimization via the cross-entropy method with an application to mixture models

Global likelihood maximization is an important aspect of many statistical analyses. Often the likelihood function is highly multiextremal. This presents a significant challenge to standard search procedures, which often settle too quickly into an inferior local maximum. We present a new approach based on the cross-entropy (CE) method, and illustrate its use for the analysis of mixture models.

Generalized Cross-entropy Methods with Applications to Rare-event Simulation and Optimization

The cross-entropy and minimum cross-entropy methods are well-known Monte Carlo simulation techniques for rare-event probability estimation and optimization. In this paper, we investigate how these methods can be eXtended to provide a general non-parametric cross-entropy framework based on φ-divergence distance measures. We show how the χ 2 distance, in particular, yields a viable alternative to the Kullback—Leibler distance. The theory is illustrated with various eXamples from density estimation, rare-event simulation and continuous multi-eXtremal optimization.

The normal law under linear restrictions: simulation and estimation via minimax tilting

Simulation from the truncated multivariate normal distribution in high dimensions is a recurrent problem in statistical computing, and is typically only feasible using approximate MCMC sampling. In this article we propose a minimax tilting method for exact iid simulation from the truncated multivariate normal distribution. The new methodology provides both a method for simulation and an efficient estimator to hitherto intractable Gaussian integrals. We prove that the estimator possesses a rare vanishing relative error asymptotic property. Numerical experiments suggest that the proposed scheme is accurate in a wide range of setups for which competing estimation schemes fail. We give an application to exact iid simulation from the Bayesian posterior of the probit regression model.

Static Network Reliability Estimation via Generalized Splitting

We propose a novel simulation-based method that exploits a generalized splitting GS algorithm to estimate the reliability of a graph or network, defined here as the probability that a given set of nodes are connected, when each link of the graph fails with a given small probability. For large graphs, in general, computing the exact reliability is an intractable problem and estimating it by standard Monte Carlo methods poses serious difficulties, because the unreliability one minus the reliability is often a rare-event probability. We show that the proposed GS algorithm can accurately estimate extremely small unreliabilities and we exhibit large examples where it performs much better than existing approaches. It is also flexible enough to dispense with the frequently made assumption of independent edge failures.

Markov chain importance sampling with applications to rare event probability estimation

We present a versatile Monte Carlo method for estimating multidimensional integrals, with applications to rare-event probability estimation. The method fuses two distinct and popular Monte Carlo simulation methods—Markov chain Monte Carlo and importance sampling—into a single algorithm. We show that for some applied numerical examples the proposed Markov Chain importance sampling algorithm performs better than methods based solely on importance sampling or MCMC.

Chapter 3 – The Cross-Entropy Method for Optimization

The cross-entropy method is a versatile heuristic tool for solving difficult estimation and optimization problems, based on Kullback–Leibler (or cross-entropy) minimization. As an optimization method it unifies many existing population-based optimization heuristics. In this chapter we show how the cross-entropy method can be applied to a diverse range of combinatorial, continuous, and noisy optimization problems.

Spatial Process Simulation

The simulation of random spatial data on a computer is an important tool for understanding the behavior of spatial processes. In this chapter we describe how to simulate realizations from the main types of spatial processes, including Gaussian and Markov random fields, point processes, spatial Wiener processes, and Levy fields. Concrete MATLAB code is provided.

Dependent failures in highly reliable static networks

Static network reliability models typically assume that the failures of their components are independent. This assumption allows for the design of efficient Monte Carlo algorithms that can estimate the network reliability in settings where it is a rare-event probability. Despite this computational benefit, independent component failures is frequently not a realistic modeling assumption for real-life networks. In this article we show how the splitting methods for rare-event simulation can be used to estimate the reliability of a network model that incorporates a realistic dependence structure via the Marshal-Olkin copula.