Henryk Woźniakowski

When Are Quasi-Monte Carlo Algorithms Efficient for High Dimensional Integrals?

Recently, quasi-Monte Carlo algorithms have been successfully used for multivariate integration of high dimensiond, and were significantly more efficient than Monte Carlo algorithms. The existing theory of the worst case error bounds of quasi-Monte Carlo algorithms does not explain this phenomenon. This paper presents a partial answer to why quasi-Monte Carlo algorithms can work well for arbitrarily larged. It is done by identifying classes of functions for which the effect of the dimensiondis negligible. These areweightedclasses in which the behavior in the successive dimensions is moderated by a sequence of weights. We prove that the minimalworst caseerror of quasi-Monte Carlo algorithms does not depend on the dimensiondiff the sum of the weights is finite. We also prove that the minimal number of function values in the worst case setting needed to reduce the initial error by ? is bounded byC??p, where the exponentp? 1, 2], andCdepends exponentially on the sum of weights. Hence, the relatively small sum of the weights makes some quasi-Monte Carlo algorithms strongly tractable. We show in a nonconstructive way that many quasi-Monte Carlo algorithms are strongly tractable. Even random selection of sample points (done once for the whole weighted class of functions and then the worst case error is established for that particular selection, in contrast to Monte Carlo where random selection of sample points is carried out for a fixed function) leads to strong tractable quasi-Monte Carlo algorithms. In this case the minimal number of function values in theworst casesetting is of order ??pwith the exponentp= 2. The deterministic construction of strongly tractable quasi-Monte Carlo algorithms as well as the minimal exponentpis open.

Tractability of Multivariate Problems

Multivariate problems occur in many applications. These problems are defined on spaces of d-variate functions and d can be huge – in the hundreds or even in the thousands. Some high-dimensional problems can be solved efficiently to within ε, i.e., the cost increases polynomially in ε−1 and d. However, there are many multivariate problems for which even the minimal cost increases exponentially in d. This exponential dependence on d is called intractability or the curse of dimensionality. This is the first of a three-volume set comprising a comprehensive study of the tractability of multivariate problems. It is devoted to algorithms using linear information consisting of arbitrary linear functionals. The theory for multivariate problems is developed in various settings: worst case, average case, randomized and probabilistic. A problem is tractable if its minimal cost is not exponential in ε−1 and d. There are various notions of tractability, depending on how we measure the lack of exponential dependence. For example, a problem is polynomially tractable if its minimal cost is polynomial in ε−1 and d. The study of tractability was initiated about 15 years ago. This is the first research monograph on this subject. Many multivariate problems suffer from the curse of dimensionality when they are defined over classical (unweighted) spaces. But many practically important problems are solved today for huge d in a reasonable time. One of the most intriguing challenges of theory is to understand why this is possible. Multivariate problems may become tractable if they are defined over weighted spaces with properly decaying weights. In this case, all variables and groups of variables are moderated by weights. The main purpose of this book is to study weighted spaces and to obtain conditions on the weights that are necessary and sufficient to achieve various notions of tractability. The book is of interest for researchers working in computational mathematics, especially in approximation of highdimensional problems. It may be also suitable for graduate courses and seminars. The text concludes with a list of thirty open problems that can be good candidates for future tractability research.

Average Case Complexity of Multivariate Integration

We study the average case complexity of multivariate integration for the class of continuous functions of d variables equipped with the classical Wiener sheet measure. To derive the average case complexity one needs to obtain optimal sample points. We prove that optimal design is closely related to discrepancy theory which has been extensively studied for many years. This relation enables us to show that optimal sample points can be derived from Hammersley points. Extending the result of Roth and using the recent result of Wasilkowski, we conclude that the average case complexity is θ(e -1 (lne -1 ) (d-1)/2 )

Tractability of Multivariate Integration for Weighted Korobov Classes

We study the worst-case error of multivariate integration in weighted Korobov classes of periodic functions of d coordinates. This class is defined in terms of weights ?j which moderate the behavior of functions with respect to successive coordinates. We study two classes of quadrature rules. They are quasi-Monte Carlo rules which use n function values and in which all quadrature weights are 1/n and rules for which all quadrature weights are non-negative. Tractability for these two classes of quadrature rules means that the minimal number of function values needed to guarantee error ? in the worst-case setting is bounded by a polynomial in d and ??1. Strong tractability means that the bound does not depend on d and depends polynomially on ??1. We prove that strong tractability holds iff ?∞j=1?j 0 there exist lattice rules that satisfy an error bound independent of d and of order n??/2+?. This is almost the best possible result, since the order n??/2 cannot be improved upon even for d=1. A corresponding result is deduced for weighted non-periodic Sobolev spaces: if ?∞j=1?1/2j 0 there exist shifted lattice rules that satisfy an error bound independent of d and of order n?1+?. We also check how the randomized error of the (classical) Monte Carlo algorithm depends on d for weighted Korobov classes. It turns out that Monte Carlo is strongly tractable iff ?∞j=1log?j<∞ and tractable iff limsupd?∞?dj=1log?j/logd<∞. Hence, in particular, for ?j=1 we have the usual Korobov space in which integration is intractable for the two classes of quadrature rules in the worst-case setting, whereas Monte Carlo is strongly tractable in the randomized setting.

Integration and Approximation in Arbitrary Dimensions

We study multivariate integration and approximation for various classes of functions of d variables with arbitrary d. We consider algorithms that use function evaluations as the information about the function. We are mainly interested in verifying when integration and approximation are tractable and strongly tractable. Tractability means that the minimal number of function evaluations needed to reduce the initial error by a factor of ɛ is bounded by C(d)ɛ−p for some exponent p independent of d and some function C(d). Strong tractability means that C(d) can be made independent of d. The ‐exponents of tractability and strong tractability are defined as the smallest powers of ɛ{-1} in these bounds.We prove that integration is strongly tractable for some weighted Korobov and Sobolev spaces as well as for the Hilbert space whose reproducing kernel corresponds to the covariance function of the isotropic Wiener measure. We obtain bounds on the ‐exponents, and for some cases we find their exact values. For some weighted Korobov and Sobolev spaces, the strong ‐exponent is the same as the ‐exponent for d=1, whereas for the third space it is 2.For approximation we also consider algorithms that use general evaluations given by arbitrary continuous linear functionals as the information about the function. Our main result is that the ‐exponents are the same for general and function evaluations. This holds under the assumption that the orthonormal eigenfunctions of the covariance operator have uniformly bounded L∞ norms. This assumption holds for spaces with shift-invariant kernels. Examples of such spaces include weighted Korobov spaces. For a space with non‐shift‐invariant kernel, we construct the corresponding space with shift-invariant kernel and show that integration and approximation for the non-shift-invariant kernel are no harder than the corresponding problems with the shift-invariant kernel. If we apply this construction to a weighted Sobolev space, whose kernel is non-shift-invariant, then we obtain the corresponding Korobov space. This enables us to derive the results for weighted Sobolev spaces.

Convergence and Complexity of Newton Iteration for Operator Equations

Abstract : An optimal convergence condition for Newton iteration in a Banach space is established. There exist problems for which the iteration converges but the complexity is unbounded. It is shown which stronger condition must be imposed to also assure good complexity.

Good Lattice Rules in Weighted Korobov Spaces with General Weights

We study the problem of multivariate integration and the construction of good lattice rules in weighted Korobov spaces with general weights. These spaces are not necessarily tensor products of spaces of univariate functions. Sufficient conditions for tractability and strong tractability of multivariate integration in such weighted function spaces are found. These conditions are also necessary if the weights are such that the reproducing kernel of the weighted Korobov space is pointwise non-negative. The existence of a lattice rule which achieves the nearly optimal convergence order is proven. A component-by-component (CBC) algorithm that constructs good lattice rules is presented. The resulting lattice rules achieve tractability or strong tractability error bounds and achieve nearly optimal convergence order for suitably decaying weights. We also study special weights such as finite-order and order-dependent weights. For these special weights, the cost of the CBC algorithm is polynomial. Numerical computations show that the lattice rules constructed by the CBC algorithm give much smaller worst-case errors than the mean worst-case errors over all quasi-Monte Carlo rules or over all lattice rules, and generally smaller worst-case errors than the best Korobov lattice rules in dimensions up to hundreds. Numerical results are provided to illustrate the efficiency of CBC lattice rules and Korobov lattice rules (with suitably chosen weights), in particular for high-dimensional finance problems.

Weighted tensor product algorithms for linear multivariate problems

Abstract We study the e -approximation of linear multivariate problems defined over weighted tensor product Hilbert spaces of functions f of d variables. A class of weighted tensor product (WTP) algorithms is defined which depends on a number of parameters. Two classes of permissible information are studied. Λ all consists of all linear functionals while Λ std consists of evaluations of f or its derivatives. We show that these multivariate problems are sometimes tractable even with a worst-case assurance. We study problem tractability by investigating when a WTP algorithm is a polynomial-time algorithm, that is, when the minimal number of information evaluations is a polynomial in 1/ e and d . For Λ all we construct an optimal WTP algorithm and provide a necessary and sufficient condition for tractability in terms of the sequence of weights and the sequence of singular values for d =1. For Λ std we obtain a weaker result by constructing a WTP algorithm which is optimal only for some weight sequences.

Liberating the weights

A partial answer to why quasi-Monte Carlo (QMC) algorithms work well for multivariate integration was given in Sloan and Woźniakowski (J. Complexity 14 (1998) 1-33) by introducing weighted spaces. In these spaces the importance of successive coordinate directions is quantified by a sequence of weights. However, to be able to make use of weighted spaces for a particular application one has to make a choice of the weights. In this work, we take a more general view of the weights by allowing them to depend arbitrarily not only on the coordinates but also on the number of variables. Liberating the weights in this way allows us to give a recommendation for how to choose the weights in practice. This recommendation results from choosing the weights so as to minimize the error bound. We also consider how best to choose the underlying weighted Sobolev space within which to carry out the analysis. We revisit also lower bounds on the worst-case error, which change in many minor ways now, since the weights are allowed to depend on the number of variables, and we do not assume that the weights are uniformly bounded as has been assumed in previous papers. Necessary and sufficient conditions for QMC tractability and strong QMC tractability are obtained for the weighted Sobolev spaces with general weights. In the final section, we show that the analysis of variance decomposition of functions from one of the Sobolev spaces is equivalent to the decomposition of functions with respect to an orthogonal decomposition of this space.

Tractability and Strong Tractability of Linear Multivariate Problems

Linear multivariate problems are defined as the approximation of linear operators on functions of d variables. We study the complexity of computing an ?-approximation in different settings. We are particularly interested in large d and/or large ??1. Tractability means that the complexity is bounded by c(d) K(d, ?), where c(d) is the cost of one information operation and K(d, ?) is a polynomial in d and/or in ??1. Strong tractability means that K(d, ?) is a polynomial in ??1, independent of d. We provide necessary and sufficient conditions for linear multivariate problems to be tractable or strongly tractable in the worst case, average case, randomized, and probabilistic settings. This is done for the class ?all where an information operation is defined as the computation of any continuous linear functional. We also consider the class ?std where an information operation is defined as the computation of a function value. We show under mild assumptions that tractability in the class ?all is equivalent to tractability in the class ?std. The proof is, however, not constructive. Finally, we consider linear multivariate problems over reproducing kernel Hilbert spaces, showing that such problems are strongly tractable even in the worst case setting.