Pierre Tarrès

Edge-reinforced random walk, vertex-reinforced jump process and the supersymmetric hyperbolic sigma model

Edge-reinforced random walk (ERRW), introduced by Coppersmith and Diaconis in 1986, is a random process that takes values in the vertex set of a graph G, which is more likely to cross edges it has visited before. We show that it can be interpreted as an annealed version of the Vertex-reinforced jump process (VRJP), conceived by Werner and first studied by Davis and Volkov (2002,2004), a continuous-time process favouring sites with more local time. We calculate, for any finite graph G, the limiting measure of the centred occupation time measure of VRJP, and interpret it as a supersymmetric hyperbolic sigma model in quantum field theory. This enables us to deduce that VRJP is recurrent in any dimension for large reinforcement, using a localisation result of Disertori and Spencer (2010).

When can the two-armed bandit algorithm be trusted?

We investigate the asymptotic behaviour of one version of the so-called two-armed bandit algorithm. It is an example of stochastic approximation procedure whose associated ODE has both a repulsive and an attractive equilibrium, at which the procedure is noiseless. We show that if the gain parameter is constant or goes to 0 not too fast, the algorithm does fall in the noiseless repulsive equilibrium with positive probability whereas it always converges to its natural attractive target when the gain parameter goes to zero at some appropriate rates depending on the parameters of the model. We also elucidate the behaviour of the constant step algorithm when the step goes to 0. Finally, we highlight the connection between the algorithm and the Polya urn. An application to asset allocation is briefly described.

Vertex-reinforced random walk on ℤ eventually gets stuck on five points

Vertex-reinforced random walk (VRRW), defined by Pemantle in 1988, is a random process that takes values in the vertex set of a graph G, which is more likely to visit vertices it has visited before. Pemantle and Volkov considered the case when the underlying graph is the one-dimensional integer lattice ℤ. They proved that the range is almost surely finite and that with positive probability the range contains exactly five points. They conjectured that this second event holds with probability 1. The proof of this conjecture is the main purpose of this paper.

Online learning as stochastic approximation of regularization paths: Optimality and almost-sure convergence

In this paper, an online learning algorithm is proposed as sequential stochastic approximation of a regularization path converging to the regression function in reproducing kernel Hilbert spaces (RKHSs). We show that it is possible to produce the best known strong (RKHS norm) convergence rate of batch learning, through a careful choice of the gain or step size sequences, depending on regularity assumptions on the regression function. The corresponding weak (mean square distance) convergence rate is optimal in the sense that it reaches the minimax and individual lower rates in this paper. In both cases, we deduce almost sure convergence, using Bernstein-type inequalities for martingales in Hilbert spaces. To achieve this, we develop a bias-variance decomposition similar to the batch learning setting; the bias consists in the approximation and drift errors along the regularization path, which display the same rates of convergence, and the variance arises from the sample error analyzed as a (reverse) martingale difference sequence. The rates above are obtained by an optimal tradeoff between the bias and the variance.

An asymptotic result for Brownian polymers

We consider a model of the shape of a growing polymer introduced by Durrett and Rogers (Probab. Theory Related Fields 92 (1992) 337-349). We prove their conjecture about the asymptotic behavior of the underlying continuous process X-t (corresponding to the location of the end of the polymer at time t) for a particular type of repelling interaction function without compact support.

Some dynamics of signaling games

Information transfer is a basic feature of life that includes signaling within and between organisms. Owing to its interactive nature, signaling can be investigated by using game theory. Game theoretic models of signaling have a long tradition in biology, economics, and philosophy. For a long time the analyses of these games has mostly relied on using static equilibrium concepts such as Pareto optimal Nash equilibria or evolutionarily stable strategies. More recently signaling games of various types have been investigated with the help of game dynamics, which includes dynamical models of evolution and individual learning. A dynamical analysis leads to more nuanced conclusions as to the outcomes of signaling interactions. Here we explore different kinds of signaling games that range from interactions without conflicts of interest between the players to interactions where their interests are seriously misaligned. We consider these games within the context of evolutionary dynamics (both infinite and finite population models) and learning dynamics (reinforcement learning). Some results are specific features of a particular dynamical model, whereas others turn out to be quite robust across different models. This suggests that there are certain qualitative aspects that are common to many real-world signaling interactions.

Diffusivity bounds for 1D Brownian polymers

We study the asymptotic behavior of a self-interacting one-dimensional Brownian polymer first introduced by Durrett and Rogers [Probab. Theory Related Fields 92 (1992) 337–349]. The polymer describes a stochastic process with a drift which is a certain average of its local time. We show that a smeared out version of the local time function as viewed from the actual position of the process is a Markov process in a suitably chosen function space, and that this process has a Gaussian stationary measure. As a first consequence, this enables us to partially prove a conjecture about the law of large numbers for the end-to-end displacement of the polymer formulated in Durrett and Rogers [Probab. Theory Related Fields 92 (1992) 337–349]. Next we give upper and lower bounds for the variance of the process under the stationary measure, in terms of the qualitative infrared behavior of the interaction function. In particular, we show that in the locally self-repelling case (when the process is essentially pushed by the negative gradient of its own local time) the process is super-diffusive.

On ergodic two-armed bandits

A device has two arms with unknown deterministic payo! s, and the aim is to asymptotically identify the best one without spending too much time on the other. The Narendra algorithm o! ers a stochastic procedure to this end. We show under weak ergodic assumptions on these deterministic payo! s that the procedure eventually chooses the best arm (i.e. with greatest Cesaro limit) with probability one, for appropriate step sequences of the algorithm. In the case of i.i.d. payo! s, this implies a “quenched” version of the “annealed” result

What is the Difference Between a Square and a Triangle

We offer a reader-friendly introduction to the attracting edge problem (also known as the “triangle conjecture”) and its most general current solution of Limic and Tarres (2007). Little original research is reported; rather this article “zooms in” to describe the essential characteristics of two different techniques/approaches verifying the almost sure existence of the attracting edge for the strongly edge reinforced random walk (SERRW) on a square. Both arguments extend straightforwardly to the SERRW on even cycles. Finally, we show that the case where the underlying graph is a triangle cannot be studied by a simple modification of either of the two techniques.