Harro Walk

Stochastic approximation and optimization of random systems

I Foundations of stochastic approximation.- 1 Almost sure convergence of stochastic approximation procedures.- 2 Recursive methods for linear problems.- 3 Stochastic optimization under stochastic constraints.- 4 A learning model recursive density estimation.- 5 Invariance principles in stochastic approximation.- 6 On the theory of large deviations.- References for Part I.- II Applicational aspects of stochastic approximation.- 7 Markovian stochastic optimization and stochastic approximation procedures.- 8 Asymptotic distributions.- 9 Stopping times.- 10 Applications of stochastic approximation methods.- References for Part II.- III Applications to adaptation algorithms.- 11 Adaptation and tracking.- 12 Algorithm development.- 13 Asymptotic Properties in the decreasing gain case.- 14 Estimation of the tracking ability of the algorithms.- References for Part III.

Strong universal pointwise consistency of recursive regression estimates

For semi-recursive and recursive kernel estimates of a regression of Y on X (d-dimensional random vector X, integrable real random variable Y), introduced by Devroye and Wagner and by Révész, respectively, strong universal pointwise consistency is shown, i.e. strong consistency PX-almost everywhere for general distribution of (X, Y). Similar results are shown for the corresponding partitioning estimates.

Convergence of the Robbins-Monro method for linear problems in a Banach space

On etudie la convergence de la methode de Robbins-Monro pour les problemes lineaires dans un espace de Banach

On the strong universal consistency of a recursive regression estimate by Pál Révész

A result is presented concerning the strong universal consistency of a recursive kernel-type regression function estimate introduced by P. Revesz.

Almost Sure Convergence Properties of Nadaraya-Watson Regression Estimates

For Nadaraya-Watson regression estimates with window kernel self-contained proofs of strong universal consistency for special bandwidths and of the corresponding Cesaro summability for general bandwidths are given.

On the Averaged Stochastic Approximation for Linear Regression

For a linear regression function the average of stochastic approximation with constant gain is considered. In case of ergodic observations almost sure convergence is proved, where the limit is biased with small bias for small gain. For independent and identically distributed observations and also under martingale and mixing assumptions, asymptotic normality with

Stochastic iteration for a constrained optimization problem

(n^{-1/2})

Foundations of stochastic approximation

-convergence order is obtained. In the martingale case the asymptotic covariance matrix is close to the optimum one if the gain is small.

Empirical Portfolio Selection Strategies With Proportional Transaction Costs

A primal dual method of Kushner and Sanvicente for a constrained optimization problem with convex regression functions is investigated without a priori bounds. For the stochastic approximation sequence almost sure convergence to a random optimal solution and a random Kuhn-Tucker vector is shown, and for the uniqueness case, a functional central limit theorem is given.

Strong universal consistency of smooth kernel regression estimates

Stochastic approximation or stochastic iteration concerns recursive estimation of quantities in connection with noise contaminated observations. Historical starting points are the papers of Robbins and Monro (1951) and of Kiefer and Wolfowitz (1952) on recursive estimation of zero and extremal points, resp., of regression functions, i.e. of functions whose values can be observed with zero expectation errors.