Seksan Kiatsupaibul

An analytically derived cooling schedule for simulated annealing

We present an analytically derived cooling schedule for a simulated annealing algorithm applicable to both continuous and discrete global optimization problems. An adaptive search algorithm is used to model an idealized version of simulated annealing which is viewed as consisting of a series of Boltzmann distributed sample points. Our choice of cooling schedule ensures linearity in the expected number of sample points needed to become arbitrarily close to a global optimum.

Discrete Hit-and-Run for Sampling Points from Arbitrary Distributions Over Subsets of Integer Hyperrectangles

We consider the problem of sampling a point from an arbitrary distribution π over an arbitrary subset S of an integer hyperrectangle. Neither the distribution π nor the support set S are assumed to be available as explicit mathematical equations, but may only be defined through oracles and, in particular, computer programs. This problem commonly occurs in black-box discrete optimization as well as counting and estimation problems. The generality of this setting and high dimensionality of S precludes the application of conventional random variable generation methods. As a result, we turn to Markov chain Monte Carlo (MCMC) sampling, where we execute an ergodic Markov chain that converges to π so that the distribution of the point delivered after sufficiently many steps can be made arbitrarily close to π. Unfortunately, classical Markov chains, such as the nearest-neighbor random walk or the coordinate direction random walk, fail to converge to π because they can get trapped in isolated regions of the support set. To surmount this difficulty, we propose discrete hit-and-run (DHR), a Markov chain motivated by the hit-and-run algorithm known to be the most efficient method for sampling from log-concave distributions over convex bodies in Rn. We prove that the limiting distribution of DHR is π as desired, thus enabling us to sample approximately from π by delivering the last iterate of a sufficiently large number of iterations of DHR. In addition to this asymptotic analysis, we investigate finite-time behavior of DHR and present a variety of examples where DHR exhibits polynomial performance.

An analysis of a variation of hit-and-run for uniform sampling from general regions

Hit-and-run, a class of MCMC samplers that converges to general multivariate distributions, is known to be unique in its ability to mix fast for uniform distributions over convex bodies. In particular, its rate of convergence to a uniform distribution is of a low order polynomial in the dimension. However, when the body of interest is difficult to sample from, typically a hyperrectangle is introduced that encloses the original body, and a one-dimensional acceptance/rejection is performed. The fast mixing analysis of hit-and-run does not account for this one-dimensional sampling that is often needed for implementation of the algorithm. Here we show that the effect of the size of the hyperrectangle on the efficiency of the algorithm is only a linear scaling effect. We also introduce a variation of hit-and-run that accelerates the sampler and demonstrate its capability through a computational study.

Pattern discrete and mixed Hit-and-Run for global optimization

We develop new Markov chain Monte Carlo samplers for neighborhood generation in global optimization algorithms based on Hit-and-Run. The success of Hit-and-Run as a sampler on continuous domains motivated Discrete Hit-and-Run with random biwalk for discrete domains. However, the potential for efficiencies in the implementation, which requires a randomization at each move to create the biwalk, lead us to a different approach that uses fixed patterns in generating the biwalks. We define Sphere and Box Biwalks that are pattern-based and easily implemented for discrete and mixed continuous/discrete domains. The pattern-based Hit-and-Run Markov chains preserve the convergence properties of Hit-and-Run to a target distribution. They also converge to continuous Hit-and-Run as the mesh of the discretized variables becomes finer, approaching a continuum. Moreover, we provide bounds on the finite time performance for the discrete cases of Sphere and Box Biwalks. We embed our samplers in an Improving Hit-and-Run global optimization algorithm and test their performance on a number of global optimization test problems.

Exact Inferences for a Weibull Model

ABSTRACT This article considers the problem of making exact inferences about a standard two parameter Weibull model. Exact inferences are obtained from the construction of a confidence set for the two parameters that has an exact specified confidence level. This exact confidence set is obtained from Kolmogorovs bounds on the empirical cumulative distribution function of the data set. Simple and compact equations are derived that define the confidence set in terms of a set of linear inequalities. The confidence set can be used, for example, to generate confidence bands for the reliability function, confidence intervals for the moments and quantiles of the distribution, and confidence intervals for the parameters themselves. The methodology can be extended to different models such as other two-parameter and three-parameter Weibull models.

Normal probability plots with confidence

Normal probability plots are widely used as a statistical tool for assessing whether an observed simple random sample is drawn from a normally distributed population. The users, however, have to judge subjectively, if no objective rule is provided, whether the plotted points fall close to a straight line. In this paper, we focus on how a normal probability plot can be augmented by intervals for all the points so that, if the population distribution is normal, then all the points should fall into the corresponding intervals simultaneously with probability 1-α. These simultaneous 1-α probability intervals provide therefore an objective mean to judge whether the plotted points fall close to the straight line: the plotted points fall close to the straight line if and only if all the points fall into the corresponding intervals. The powers of several normal probability plot based (graphical) tests and the most popular nongraphical Anderson-Darling and Shapiro-Wilk tests are compared by simulation. Based on this comparison, recommendations are given in Section 3 on which graphical tests should be used in what circumstances. An example is provided to illustrate the methods.

Win-probabilities for comparing two Weibull distributions

This paper considers the problem of comparing two processes or treatments which are each modelled with a Weibull distribution. Win-probabilities are considered, which compare potential single future observations from each of the two treatments. This information can be useful in helping decide which of the two treatments to adopt, and can be combined with other factors relevant to a practitioner such as the availabilities, costs and side-effects of the two treatments. A methodology employing joint confidence sets is developed which not only allows estimation and confidence interval construction for the win-probabilities, but at the same guaranteed confidence level also tests whether Weibull distributions are appropriate for the data, identifies any common Weibull distributions for the two processes and also provides individual inferences for the two Weibull distributions. Examples are given to illustrate the implementation and application of this methodology, for which R computer code is available from the authors. This methodology can be extended to different models such as other two-parameter and three-parameter Weibull models, and to the comparison of three or more Weibull distributions.

Rank constrained distribution and moment computations

Consider a set of independent random variables with specified distributions or a set of multivariate normal random variables with a product correlation structure. This paper shows how the distributions and moments of these random variables can be calculated conditional on a specified ranking of their values. This can be useful when the ordering of the variables can be determined without observing the actual values of the variables, as in ranked set sampling, for example. Thus, prior information on the distributions and moments from their individual specified distributions can be updated to provide improved posterior information using the known ranking. While these calculations ostensibly involve high dimensional integral expressions, it is shown how the previously developed general recursive integration methodology can be applied to this problem so that they can be evaluated in a straightforward manner as a series of one-dimensional or two-dimensional integral calculations. Furthermore, the proposed methodology possesses a self-correction mechanism in the computation that prevents any serious growth of the errors. Examples illustrate how different kinds of ranking information affect the distributions, expectations, variances, and covariances of the variables, and how they can be employed to solve a decision making problem. An integration method is proposed for random variables conditioned on their ranking.High dimensional integration effort is reduced to either one or two dimensional integration.The method possesses a self-correction mechanism supported by numerical results.Reinforcing ranking and opposing ranking are defined and their effects are investigated.

Confidence Intervals for Quantiles of a Two-parameter Exponential Distribution under Progressive Type-II Censoring

Confidence intervals for the pth-quantile Q of a two-parameter exponential distribution provide useful information on the plausible range of Q, and only inefficient equal-tail confidence intervals have been discussed in the statistical literature so far. In this article, the construction of the shortest possible confidence interval within a family of two-sided confidence intervals is addressed. This shortest confidence interval is always shorter, and can be substantially shorter, than the corresponding equal-tail confidence interval. Furthermore, the computational intensity of both methodologies is similar, and therefore it is advantageous to use the shortest confidence interval. It is shown how the results provided in this paper can apply to data obtained from progressive Type II censoring, with standard Type II censoring as a special case. The applications of more complex confidence interval constructions through acceptance set inversions that can employ prior information are also discussed.

Normal probability plots with confidence for the residuals in linear regression

ABSTRACT Normal probability plots for a simple random sample and normal probability plots for residuals from linear regression are not treated differently in statistical text books. In the statistical literature, 1 − α simultaneous probability intervals for augmenting a normal probability plot for a simple random sample are available. The first purpose of this article is to demonstrate that the tests associated with the 1 − α simultaneous probability intervals for a simple random sample may have a size substantially different from α when applied to the residuals from linear regression. This leads to the second purpose of this article: construction of four normal probability plot-based tests for residuals, which have size α exactly. We then compare the powers of these four graphical tests and a non-graphical test for residuals in order to assess the power performances of the graphical tests and to identify the ones that have better power. Finally, an example is provided to illustrate the methods.