Xi-Ren Cao

General approach to blind source separation

This paper identifies and studies two major issues in the blind source separation problem: separability and separation principles. We show that separability is an intrinsic property of the measured signals and can be described by the concept of m-row decomposability introduced in this paper; we also show that separation principles can be developed by using the structure characterization theory of random variables. In particular, we show that these principles can be derived concisely and intuitively by applying the Darmois-Skitovich theorem, which is well known in statistical inference theory and psychology. Some new insights are gained for designing blind source separation filters.

Perturbation realization, potentials, and sensitivity analysis of Markov processes

Two fundamental concepts and quantities, realization factors and performance potentials, are introduced for Markov processes. The relations among these two quantities and the group inverse of the infinitesimal generator are studied. It is shown that the sensitivity of the steady-state performance with respect to the change of the infinitesimal generator can be easily calculated by using either of these three quantities and that these quantities can be estimated by analyzing a single sample path of a Markov process. Based on these results, algorithms for estimating performance sensitivities on a single sample path of a Markov process can be proposed. The potentials in this paper are defined through realization factors and are shown to be the same as those defined by Poisson equations. The results provide a uniform framework of perturbation realization for infinitesimal perturbation analysis (IPA) and non-IPA approaches to the sensitivity analysis of steady-state performance; they also provide a theoretical background for the PA algorithms developed in recent years.

Stochastic Learning and Optimization

Performance optimization is vital in the design and operation of modern engineering systems, including communications, manufacturing, robotics, and logistics. Most engineering systems are too complicated to model, or the system parameters cannot be easily identified, so learning techniques have to be applied. This book provides a unified framework based on a sensitivity point of view. It also introduces new approaches and proposes new research topics within this sensitivity-based framework. This new perspective on a popular topic is presented by a well respected expert in the field.

Convergence of parameter sensitivity estimates in a stochastic experiment

To reduce the error in estimating the gradient (parameter sensitivity) of an unknown function is of great importance in stochastic optimization problems. Three kinds of parameter sensitivity estimates using the Monte Carlo method are discussed in this paper. The estimates depend on the number of replications, N, and the change in parameter, ¿v. The convergence properties as N¿¿ and ¿v¿0 for these estimates are obtained. The result explains many theoretical and practical issues in the study of discrete event dynamic systems, as well as continuous dynamic systems, by the Monte Carlo method. It is proved that an estimate based on averaging the gradients calculated along each sample path by a perturbation of the path is much better than the other estimates if the output functions are uniformly differentiable with probability one (w.p.1). It is also concluded that in computer simulations one should always choose the same seed for both v and v+¿v in estimating the parameter sensitivity. Combining the results in this paper with existing stochastic approximation algorithms may yield algorithms with faster convergence.

Brief paper: Infinitesimal and finite perturbation analysis for queueing networks

The sample-path pertrubation analysis technique introduced in refs. (1-4) is extended to included finite (and possibly large) pertrubations typically introduced by changes in queue sizes or other parameter. It is shown that there is a natural hierarchy of perturbation analysis which takes care of increasingly large perturbations. Experiments with zeroth (infinitesmal) and first order (finite) pertrubation analysis show that significant accuracy improvement can be obtained with small increase in computational effort.

Call admission control for voice/data integrated cellular networks: performance analysis and comparative study

In this paper, we propose a new call admission control scheme called dual threshold bandwidth reservation, or DTBR scheme. The main novelty is that it builds upon a complete sharing approach, in which the channels in each cell are shared among the different traffic types and multiple thresholds are used to meet the specific quality-of-service (QoS) requirements. We present a detailed comparative study based on mathematical and simulation models, and quantitatively demonstrate that the DTBR is capable of providing the QoS guarantee for each type of traffic, while at the same time leading to much better channel efficiency. We further show that the DTBR scheme with elastic data service can offer both service guarantee and service differentiation for voice and data services, and enhance the bandwidth utilization.

Models of discrete event dynamic systems

Many new techniques for modeling discrete event dynamic systems have been developed in recent years; among them are Markov processes and their imbedded Markov chains, Petri nets, queuing networks, automata and finite-state machines, finitely recursive processes, min-max algebra models, and discrete event simulation and generalized semi-Markov processes. The authors demonstrate the main features of these models by applying them to a simple example and briefly compare their features.<<ETX>>

Basic Ideas for Event-Based Optimization of Markov Systems

The goal of this paper is two-fold: First, we present a sensitivity point of view on the optimization of Markov systems. We show that Markov decision processes (MDPs) and the policy-gradient approach, or perturbation analysis (PA), can be derived easily from two fundamental sensitivity formulas, and such formulas can be flexibly constructed, by first principles, with performance potentials as building blocks. Second, with this sensitivity view we propose an event-based optimization approach, including the event-based sensitivity analysis and event-based policy iteration. This approach utilizes the special feature of a system characterized by events and illustrates how the potentials can be aggregated using the special feature and how the aggregated potential can be used in policy iteration. Compared with the traditional MDP approach, the event-based approach has its advantages: the number of aggregated potentials may scale to the system size despite that the number of states grows exponentially in the system size, this reduces the policy space and saves computation; the approach does not require actions at different states to be independent; and it utilizes the special feature of a system and does not need to know the exact transition probability matrix. The main ideas of the approach are illustrated by an admission control problem.

Algorithms for sensitivity analysis of Markov systems through potentials and perturbation realization

We provide algorithms to compute the performance derivatives of Markov chains with respect to changes in their transition matrices and of Markov processes with respect to changes in their infinitesimal generators. Our algorithms are readily applicable to the control and optimization of these Markov systems, since they are based on analyzing a single sample path and do not need explicit specification of transition matrices, nor infinitesimal generators. Compared to the infinitesimal perturbation analysis, the algorithms have a wider scope of application and require nearly the same computational effort. Numerical examples are provided to illustrate the applications of the algorithms. In particular, we apply one of our algorithms to a closed queueing network and the results are promising.

A time aggregation approach to Markov decision processes

We propose a time aggregation approach for the solution of infinite horizon average cost Markov decision processes via policy iteration. In this approach, policy update is only carried out when the process visits a subset of the state space. As in state aggregation, this approach leads to a reduced state space, which may lead to a substantial reduction in computational and storage requirements, especially for problems with certain structural properties. However, in contrast to state aggregation, which generally results in an approximate model due to the loss of Markov property, time aggregation suffers no loss of accuracy, because the Markov property is preserved. Single sample path-based estimation algorithms are developed that allow the time aggregation approach to be implemented on-line for practical systems. Some numerical and simulation examples are presented to illustrate the ideas and potential computational savings.