Fuchang Gao

Implementing the Nelder-Mead simplex algorithm with adaptive parameters

In this paper, we first prove that the expansion and contraction steps of the Nelder-Mead simplex algorithm possess a descent property when the objective function is uniformly convex. This property provides some new insights on why the standard Nelder-Mead algorithm becomes inefficient in high dimensions. We then propose an implementation of the Nelder-Mead method in which the expansion, contraction, and shrink parameters depend on the dimension of the optimization problem. Our numerical experiments show that the new implementation outperforms the standard Nelder-Mead method for high dimensional problems.

METRIC ENTROPY OF CONVEX HULLS

LetT be a precompact subset of a Hilbert space. The metric entropy of the convex hull ofT is estimated in terms of the metric entropy ofT, when the latter is of order εℒ2. The estimate is best possible. Thus, it answers a question left open in [CKP].

Persistence of iterated partial sums

Let S (2) denote the iterated partial sums. That is, S (2) = S1 + S2 + · · · + Sn, where Si = X1 +X2 +· · ·+Xi. Assuming X1, X2, . . . , Xn are integrable, zero-mean, i.i.d. random variables, we show that the persistence probabilities

Small ball probabilities for the Slepian Gaussian fields

The d-dimensional Slepian Gaussian random field {S(t),t ∈ R d + } is a mean zero Gaussian process with covariance function ES(s)S(t) = I d i =i max(0,a i - s i -t i for a i > 0 and t = (t i ,...,t d ) ∈ R d + . Small ball probabilities for S(t) are obtained under the L 2 -norm on [0,1] d , and under the sup-norm on [0,1] 2 which implies Talagrands result for the Brownian sheet. The method of proof for the sup-norm case is purely probabilistic and analytic, and thus avoids ingenious combinatoric arguments of using decreasing mathematical induction. In particular, Riesz product techniques are new ingredients in our arguments.

ENTROPY OF ABSOLUTE CONVEX HULLS IN HILBERT SPACES

The metric entropy of absolute convex hulls of sets in Hilbert spaces is studied for the general case when the metric entropy of the sets is arbitrary. Under some regularity assumptions, the results are sharp. The Krein–Milman theorem is a powerful tool in analysis. To quantify this theorem, a number of researchers have studied the entropy numbers of the convex hulls of precompact sets in a Banach space or a Hilbert space. The goal is to obtain a sharp upper bound for the entropy of the convex hull conv(T ), knowing the entropy of the set T . The importance of this problem was addressed by Dudley in [7], where some special cases were studied. Dudley’s results were improved by Ball and Pajor [2] and Carl [4], and extended to Banach spaces by Carl, Kyrezi and Pajor [5]. Recall that

METRIC ENTROPY OF HIGH DIMENSIONAL DISTRIBUTIONS

Let F d be the collection of all d-dimensional probability distribution functions on [0, 1] d , d > 2. The metric entropy of F d under the L 2 ([0,1] d ) norm is studied. The exact rate is obtained for d = 1, 2 and bounds are given for d > 3. Connections with small deviation probability for Brownian sheets under the sup-norm are established.

Intrinsic Volumes of the Brownian Motion Body

Motivated from Gaussian processes, we derive the intrinsic volumes of the infinite-dimensional Brownian motion body . The method is by discretization to a class of orthoschemes. Numerical support is offered for a conjecture of Sangwine-Yager, and another conjecture is offered on the rate of decay of intrinsic volume sequences.

How many Laplace transforms of probability measures are there

A bracketing metric entropy bound for the class of Laplace transforms of probability measures on [0, ∞) is obtained through its connection with the small deviation probability of a smooth Gaussian process. Our results for the particular smooth Gaussian process seem to be of independent interest.

Metric entropy and sparse linear approximation of l q -hulls for 0<q≤1

Abstract Consider l q -hulls, 0 q ≤ 1 , from a dictionary of M functions in L p space for 1 ≤ p ∞ . Their precise metric entropy orders are derived. Sparse linear approximation bounds are obtained to characterize the number of terms needed to achieve accurate approximation of the best function in a l q -hull that is closest to a target function. Furthermore, in the special case of p = 2 , it is shown that a weak orthogonal greedy algorithm achieves the optimal approximation under an additional condition.

Bracketing Entropy of High Dimensional Distributions

Let \(\mathcal{F}_{d}\) be the class of probability distribution functions on \([0,\,1]^{d},\,{d}\geq{2}\). The following estimate for the bracketing entropy of \(\mathcal{F}_{d}\) in the \([L]^{p}\) norm, \(1\,\leq\,p\,{<} \infty \), is obtained: