Peter Whittle

Risk-sensitive linear/quadratic/gaussian control

The conventional linear/quadratic/Gaussian assumptions are modified in that minimisation of the expectation of cost G defined by (2) is replaced by minimisation of the criterion function (5). The scalar – θ is a measure of risk-aversion. It is shown that modified versions of certainty equivalence and the separation theorem still hold, that optimal control is still linear Markov, and state estimate generated by a version of the Kalman filter. There are also various new features, remarked upon in Sections 5 and 7. The paper generalises earlier work of Jacobson.

A risk-sensitive maximum principle: the case of imperfect state observation

The risk-sensitive maximum principle for optimal stochastic control derived by the author in an earlier work (System Control Letters, vol.15, 1990) is restated. This is an immediate generalization of the classic Pontryagin principle, to which it reduces in the deterministic case, and is expressed immediately in terms of observables. It is derived on the assumption that the criterion function is the exponential of an additive cost function, and is exact under linear-quadratic Gaussian assumptions, but is otherwise valid as a large deviation approximation. The principle is extended to the case of imperfect state observation after preliminary establishment of a certainty-equivalence principle. The derivation yields as byproduct a large-deviation version of the updating equation for nonlinear filtering. The development is heuristic. It is believed that the mathematical arguments given are the essential ones, and provide a self-contained treatment at this level. >

A risk-sensitive maximum principle

Abstract A stochastic maximum principle is derived which is formally of general application (and so not restricted to the diffusion case). It moreover differs from the classic deterministic principle only by a modification of the Hamiltonian, and so requires no solution of stochastic differential equations or calculation of conditional expectations. The two essential points in its construction are that one considers the family of criterion functions exponential in an additive cost function (of which the conventional expected-cost case is a degenerate member) and that, in the non-LQG case, appeal to large-deviation theory must be valid.

Weak Coupling in Stochastic Systems

Weak coupling is defined as a coupling between components which affects only their ‘external statistics’, ‘internal statistics’ adapting to external statistics as they would if components were isolated. The energy and matter transfers that lead to the Gibbs distribution in statistical mechanics exemplify such a mechanism, as does the transfer of jobs in Jackson networks of processors. The class of components that show weak coupling under a certain class of communication rules is determined exactly, and generalizes the two examples mentioned.

Risk Sensitivity, A Strangely Pervasive Concept

The theme of this paper is that the simple concept of risk sensitivity raises ideas, shows associations, and forces clarification of issues that one would have thought quite unrelated.

Probability, statistics and optimisation : a tribute to Peter Whittle

Partial table of contents: PROBABILITY Probability Vicit Expectation (D. Stirzaker) The Random-Cluster Model (G. Grimmett) Fractal Dynamics of Eden Clusters (J. Hammersley) APPLIED PROBABILITY Threshold Phenomena in Epidemic Theory (A. Barbour) On the Interaction of Unreliable Routes (I. Mitrani & P. Wright) Large Deviation and Fluid Approximations in Control of Stochastic Systems (R. Weber) TIME SERIES ANALYSIS The Whittle Likelihood and Frequency Estimation (E. Hannan) Comments on Prediction by Nonlinear Least-Square Methods (H. Tong) NEURAL NETS AND COMPUTATIONAL STATISTICS Network Methods in Statistics (B. Ripley) Monte-Carlo Likelihood in Genetic Analysis (E. Thompson) STATISTICS How to Look at Objects in a 5-Dimensional Shape Space II: Looking at Diffusions (D. Kendall) Applications of Quadratic Programming in Statistics (B. Brown & C. Goodall) OPTIMISATION OF NETWORK FLOW Optimisation of Flows in Networks Over Time (E. Anderson & A. Philpott) Optimal Routeing in Loss Networks (F. Kelly) OPTIMISATION OVER TIME Indices on Thin Ice (J. Gittins) A New Formula for the Deviation Matrix (A. Hordijk & F. Spieksma) Index.

Entropy-minimizing and risk-sensitive control rules

Abstract Glover and Doyle [4] have shown that the entropy-minimising criterion is identical with the infinite-horizon form of the LEQG-optimality criterion (LEQG = linear/exponential-of-quadratic/Gaussian). Whittle and Kuhn [11] have given a Hamiltonian formulation which provides a natural treatment of LEQG-optimisation. This paper contains a direct proof that the control yielded by these Hamiltonian methods is entropy-minimising. The conclusion would follow from the two assertions above, but a simple adjoining of the proofs of these two assertions yields an argument so circuitous that a direct proof is of interest.

Risk-sensitivity, large deviations and stochastic control

Abstract An exponential function of cost is adopted as a risk-sensitive criterion, and reasons given that this choice should be a natural one. It is shown that the analysis leads to risk-sensitive versions of the certainty-equivalence principle (separation principle) and of the maximum principle, and that these have a validity even outside the usual linear/quadratic/Gaussian framework. The methods are applied to some simple examples of economic interest.

The antiphon. II : The exact evaluation of memory capacity

An improved analysis is given of the antiphon: the device proposed in Whittle (1989) to achieve reliable memory from unreliable components. It is shown that for large systems of N such components one can reliably store KN + o(N) nats of information, and the supremum value of K (the capacity, C) is determined. It is moreover shown that a positive capacity (which is evaluated) can be achieved with the number M of processing units of no more than order N.

Artificial memories: capacity, basis rate and inference

Abstract We study associative and storage memories for memory traces of size N and aim to establish that both the size of the system (as measured by, e.g., the number of nodes in a network) can be of order N and the number of traces consistent with reliable operation can be exponentially large in N, so that a positive capacity (in bits per node) can be achieved. It is well known that, if the traces are generated as M random vectors, then reliability imposes a linear bound on M, in that it implies an upper bound on the asymptotic (large N) value of α = M/N. For the noise-free Hopfield net this critical bound is about 0.138. We show that, if superposition of traces is allowed, so that the M given traces constitute the random basis of a linear code, then exponential memory size and a positive capacity can be achieved. However, there is still a critical upper bound on the basis rate α = M/N, implied now, not by the condition of reliability, but by the necessity that the recursion realising the calculation should be stable. For our model we determine this critical value exactly as α c = 3 − √8 ≐ 0.172. Our model is based upon inference concepts and differs in slight but important respects from the Hopfield model. We do not use replica methods, but appeal to a generalised version of the Wigner semi-circle theorem on the asymptotic distribution of eigenvalues.