Alexander Tsybakov

Wavelets and Approximation

In this chapter we study the approximation properties of wavelet expansions on the Sobolev spaces. We specify how fast does the wavelet expansion converge to the true function f , if f belongs to some Sobolev space. This study is continued in Chapter 9 where we consider the approximation on the Besov spaces and show that it has an intrinsic relation to wavelet expansions. The presentation in this chapter and in Chapter 9 is more formal than in the previous ones. It is designed for the mathematically oriented reader who is interested in a deeper theoretical insight into the properties of wavelet bases.

Wavelet thresholding and adaptation

This chapter treats in more detail the adaptivity property of nonlinear (thresholded) wavelet estimates. We first introduce different modifications and generalizations of soft and hard thresholding. Then we develop the notion of adaptive estimators and present the results about adaptivity of wavelet thresholding for density estimation problems. Finally, we consider the data-driven methods of selecting the wavelet basis, the threshold value and the initial resolution level, based on Stein’s principle. We finish by a discussion of oracle inequalities and miscellaneous related topics.

Statistical estimation using wavelets

In Chapters 3, 5, 6 and 7 we discussed techniques to construct functions φ and ψ (father and mother wavelets), such that the wavelet expansion (3.5) holds for any function f in L2(IR). This expansion is a special kind of orthogonal series. It is “special”, since unlike the usual Fourier series, the approximation is both in frequency and space. In this chapter we consider the problem of nonparametric statistical estimation of a function f in L2(IR) by wavelet methods. We study the density estimation and nonparametric regression settings. We also present empirical results of wavelet smoothing.

Some facts from Fourier analysis

This small chapter is here to summarize the classical facts of Fourier analysis that will be used in the sequel. We omit the proofs (except for the Poisson summation formula). They can be found in standard textbooks on the subject, for instance in Katznelson (1976), Stein & Weiss (1971).

Wavelets and Besov Spaces

This chapter is devoted to approximation theorems in Besov spaces. The advantage of Besov spaces as compared to the Sobolev spaces is that they are much more general tool in describing the smoothness properties of functions. We show that Besov spaces admit a characterization in terms of wavelet coefficients, which is not the case for Sobolev spaces. Thus the Besov spaces are intrinsically connected to the analysis of curves via wavelet techniques. The results of Chapter 8 are substantially used throughout. General references about Besov spaces are Nikol‘skii (1975), Peetre (1975), Besov, Il‘in & Nikol‘skii (1978), Bergh & Lofstrom (1976), Triebel (1992), DeVore & Lorentz (1993).

Compactly supported wavelets

The original construction of compactly supported wavelets is due to Daubechies (1988). Here we sketch the main points of Daubechies’ theory. We are interested to find the exact form of functions m0(ξ), which are trigonometric polynomials, and produce father φ and mother ψ with compact supports such that, in addition, the moments of φ and ψ of order from 1 to n vanish. This property is necessary to guarantee good approximation properties of the corresponding wavelet expansions, see Chapter 8.

The idea of multiresolution analysis

The Haar system is not very convenient for approximation of smooth functions. In fact, any Haar approximation is a discontinuous function. One can show that even if the function f is very smooth, the Haar coefficients still decrease slowly. We therefore aim to construct wavelets that have better approximation properties.

The Haar basis wavelet system

The Haar basis is known since 1910. Here we consider the Haar basis on the real line IR and describe some of its properties which are useful for the construction of general wavelet systems. Let L2 (IR) be the space of all complex valued functions f on IR such that their L2-norm is finite:

Computational aspects and statistical software implementations

\left\| {f\left\| {2 = \left( {\int_{ - \infty }^\infty {\left| {f(x)} \right|^2 dx} } \right)} \right.} \right.^{\frac{1}{2}} \langle \infty .