G. Yin

Stochastic approximation algorithms and applications

Applications and issues application to learning, state dependent noise and queueing applications to signal processing and adaptive control mathematical background convergence with probability one - Martingale difference noise convergence with probability one - correlated noise weak convergence - introduction weak convergence methods for general algorithms applications - proofs of convergence rate of convergence averaging of the iterates distributed/decentralized and asynchronous algorithms.

Markowitz's Mean-Variance Portfolio Selection with Regime Switching: A Continuous-Time Model

A continuous-time version of the Markowitz mean-variance portfolio selection model is proposed and analyzed for a market consisting of one bank account and multiple stocks. The market parameters, including the bank interest rate and the appreciation and volatility rates of the stocks, depend on the market mode that switches among a finite number of states. The random regime switching is assumed to be independent of the underlying Brownian motion. This essentially renders the underlying market incomplete. A Markov chain modulated diffusion formulation is employed to model the problem. Using techniques of stochastic linear-quadratic control, mean-variance efficient portfolios and efficient frontiers are derived explicitly in closed forms, based on solutions of two systems of linear ordinary differential equations. Related issues such as a minimum-variance portfolio and a mutual fund theorem are also addressed. All the results are markedly different from those for the case when there is no regime switching. An interesting observation is, however, that if the interest rate is deterministic, then the results exhibit (rather unexpected) similarity to their no-regime-switching counterparts, even if the stock appreciation and volatility rates are Markov-modulated.

Continuous-time Markov chains and applications: a singular perturbation approach

Prologue and Preliminaries: Introduction and overview- Mathematical preliminaries. Markovian models.- Singularly perturbed Markov chains: Asymptotic expansion: Irreducible generators. Asymptotic normality and exponential bounds. Asymptotic expansion: Weak and strong interactions. Weak and strong interactions: Asymptotic properties and ramification.- Optimizations and numerical methods: Markov decision problems. Stochastic control of dynamical systems. Numerical methods for control and optimization.

Asymptotic Properties of Hybrid Diffusion Systems

In response to the increasing needs for control and optimization of hybrid systems, this work is concerned with such asymptotic properties as recurrence (also known as weak stochastic stability in the literature) and ergodicity of regime-switching diffusions. Using Liapunov functions, necessary and sufficient conditions for positive recurrence are developed. Then, ergodicity of positive recurrent regime-switching diffusions is obtained by constructing cycles using the associated discrete-time Markov chains.

System identification using binary sensors

System identification is investigated for plants that are equipped with only binary-valued sensors. Optimal identification errors, time complexity, optimal input design, and impact of disturbances and unmodeled dynamics on identification accuracy and complexity are examined in both stochastic and deterministic information frameworks. It is revealed that binary sensors impose fundamental limitations on identification accuracy and time complexity, and carry distinct features beyond identification with regular sensors. Comparisons between the stochastic and deterministic frameworks indicate a complementary nature in their utility in binary-sensor identification.

Stabilization and destabilization of hybrid systems of stochastic differential equations

This paper aims to determine whether or not a stochastic feedback control can stabilize or destabilize a given nonlinear hybrid system. New methods are developed and sufficient conditions on the stability and instability for hybrid stochastic differential equations are provided. These results are then used to examine stochastic stabilization and destabilization.

Markowitz's mean-variance portfolio selection with regime switching: from discrete-time models to their continuous-time limits

We study a discrete-time version of Markowitzs mean-variance portfolio selection problem where the market parameters depend on the market mode (regime) that jumps among a finite number of states. The random regime switching is delineated by a finite-state Markov chain, based on which a discrete-time Markov modulated portfolio selection model is presented. Such models either arise from multiperiod portfolio selections or result from numerical solution of continuous-time problems. The natural connections between discrete-time models and their continuous-time counterpart are revealed. Since the Markov chain frequently has a large state space, to reduce the complexity, an aggregated process with smaller state-space is introduced and the underlying portfolio selection is formulated as a two-time-scale problem. We prove that the process of interest yields a switching diffusion limit using weak convergence methods. Next, based on the optimal control of the limit process obtained from our recent work, we devise portfolio selection strategies for the original problem and demonstrate their asymptotic optimality.

System identification with quantized observations

Overview.- System Settings.- Stochastic Methods for Linear Systems.- Empirical-Measure-Based Identification: Binary-Valued Observations.- Estimation Error Bounds: Including Unmodeled Dynamics.- Rational Systems.- Quantized Identification and Asymptotic Efficiency.- Input Design for Identification in Connected Systems.- Identification of Sensor Thresholds and Noise Distribution Functions.- Deterministic Methods for Linear Systems.- Worst-Case Identification under Binary-Valued Observations.- Worst-Case Identification Using Quantized Observations.- Identification of Nonlinear and Switching Systems.- Identification of Wiener Systems with Binary-Valued Observations.- Identification of Hammerstein Systems with Quantized Observations.- Systems with Markovian Parameters.- Complexity Analysis.- Space and Time Complexities, Threshold Selection, Adaptation.- Impact of Communication Channels on System Identification.

Regime Switching Stochastic Approximation Algorithms with Application to Adaptive Discrete Stochastic Optimization

This work is devoted to a class of stochastic approximation problems with regime switching modulated by a discrete-time Markov chain. Our motivation stems from using stochastic recursive algorithms for tracking Markovian parameters such as those in spreading code optimization in CDMA (code division multiple access) wireless communication. The algorithm uses constant step size to update the increments of a sequence of occupation measures. It is proved that least squares estimates of the tracking errors can be developed. Assume that the adaptation rate is of the same order of magnitude as that of the time-varying parameter, which is more difficult to deal with than that of slower parameter variations. Due to the time-varying characteristics and Markovian jumps, the usual stochastic approximation (SA) techniques cannot be carried over in the analysis. By a combined use of the SA method and two-time-scale Markov chains, asymptotic properties of the algorithm are obtained, which are distinct from the usual SA ...

Asymptotically efficient parameter estimation using quantized output observations

This paper studies identification of systems in which only quantized output observations are available. An identification algorithm for system gains is introduced that employs empirical measures from multiple sensor thresholds and optimizes their convex combinations. Strong convergence of the algorithm is first derived. The algorithm is then extended to a scenario of system identification with communication constraints, in which the sensor output is transmitted through a noisy communication channel and observed after transmission. The main results of this paper demonstrate that these algorithms achieve the Cramer-Rao lower bounds asymptotically, and hence are asymptotically efficient algorithms. Furthermore, under some mild regularity conditions, these optimal algorithms achieve error bounds that approach optimal error bounds of linear sensors when the number of thresholds becomes large. These results are further extended to finite impulse response and rational transfer function models when the inputs are designed to be periodic and full rank.