Boris M. Miller

The generalized solutions of nonlinear optimization problems with impulse control

The optimal control problem described by a nonlinear ordinary differential equation is considered with unbounded controls. The aim of this paper is to provide a representation of generalized (discontinuous) solutions in terms of differential equations with a measure. By using the method of discontinuous time change the problem with nonlinear dependence on the unbounded controls can be reduced to the simpler one with bounded controls. These results are applied to solving the problem of generalized (discontinuous) solution representation in the dynamic system with impulse-type controls.

Kalman filtering for linear systems with coefficients driven by a hidden Markov jump process

Abstract We present an explicit algorithm for the solution of the nonlinear filtering problem concerning a linear, partially observed diffusion-type model, where the coefficients are driven by a hidden Markov jump process, of which one can observe the occurrence of a jump. The (recursive) filter is of a “branching” type with its (finite) dimension growing for each jump of the hidden Markov process.

Optimization of Observations: a Stochastic Control Approach

We study a stochastic control problem for the optimization of observations in a partially observable stochastic system. Using a method of discontinuous time transformation, we associate with the original problem with unbounded controls a problem that has bounded controls. This latter problem allows us to construct nearly optimal nonanticipative Lipschitz Markov controls with finite observation power for the original problem. Since the controlled observation equation may degenerate, we also derive a corresponding filtering result and show a separation property of the optimal controls.

Flow control as stochastic optimal control problem with incomplete information

The nonlinear stochastic control problem related with flow control is considered. The state of the link is described by controlled hidden Markov process while the loss flow is described by the counting process with the intensity depending on the current transmission rate and unobserved link state. The control is the transmission rate and have to be chosen as non-anticipating process depending on the observation of the loss process. The aim of the control is to achieve the maximum of some utility function taking into account the losses of transmitted information. Originally the problem belongs to a class of stochastic control with incomplete information, however, the optimal filtering equations giving the estimation of the current link state based on the observation of the loss process give the opportunity to reduce the problem to the standard stochastic control problem with full observations. Then, a necessary optimality condition is derived in the form of stochastic maximum principle which allows us to obtain explicit analytic expressions for the optimal control in some particular cases. The optimal and suboptimal controls are investigated and compared with the flow control schemes which is used in TCP/IP networks. In particular, the optimal control demonstrates a much smoother behavior than the currently used TCP/IP congestion control.

Dynamical Systems With Active Singularities of Elastic Type: A Modeling and Controller Synthesis Framework

A new class of systems characterized by admitting impulsive control action within their singular motion phases, such as dimension changes, state discontinuities, and other irregularities, is introduced. This class, termed dynamical systems with active, or controlled, singularities, encompasses various applications with sensing and/or actuation ultra-fast in comparison with the natural system time scale, including mechanisms with impact-induced motion, power and sensor networks under faults, fast positioning devices, smart skins, and switching electronic circuits. The present work focuses on systems with impact-type singularities induced through system interaction with controlled, or active, state constraints. The latter are assumed to be characterized by parameter-dependent elastic-type constraint violation that vanishes in the limit as the parameter value tends to infinity. A physically well justified representation of this class of systems is proposed, but found to be not well suited for controller synthesis in the singular phase. To address this problem, two equations are derived - one describing controlled rigid impacts in auxiliary stretched time and compatible with the regular control design techniques, termed controlled infinitesimal dynamics equation, and another incorporating controlled impact dynamics in the infinitesimal form in terms of the shift operator along the trajectories of the first equation, termed the limit model. The latter is demonstrated to generate a unique isolated discontinuous system motion, i.e., to provide a tight and well-behaved description of the collision with rigid constraint. It is then shown that the corresponding paths generated by the original physically-based and the limit representations can be made arbitrarily close to each other uniformly except, possibly, in the vicinities of the jump points, by the appropriate choice of the value of the constraint violation parameter. This is shown to permit enforcing, for sufficiently large values of the parameter, the desired limit system behavior onto the original system by simply taking the control signals found through the limit representation, time-rescaling them, and inserting the resulting signals directly into the original system. These features show that the procedure developed, in fact, provides a well-posed controller synthesis framework for the class of systems considered. Using this framework, a ball/racket rotationally controlled impact representation is developed, and on its basis, a singular phase control signal in the form of racket rotation velocity is designed, demonstrating that the soft racket provides the specified bounce-off angle increment under a noticeably lower racket rotation velocity and a longer rotation phase than the hard racket

Brief paper: Optimization of queuing system via stochastic control

The problem of access and service rate control in queuing systems as a general optimization problem for controlled Markov process with finite state space is considered. By using the dynamic programming approach we obtain the explicit form of the optimal control in the case of minimizing cost given as a mixture of an average queue length, number of lost jobs, and service resources. The problem is considered on a finite time interval in the case of nonstationary input flow. In this case we suggest the general procedure of the numerical solution which can be applied to a problems with constraints.

Maximum Principle for Singular Stochastic Control Problems

In this paper, an optimal singular stochastic control problem is considered. For this model, a general stochastic maximum principle is obtained by using a time transformation. This is the first version of the stochastic maximum principle that covers nonlinear cases.

Singular Stochastic Control Problems

In this paper, we study an optimal singular stochastic control problem. By using a time transformation, this problem is shown to be equivalent to an auxiliary control problem defined as a combination of an optimal stopping problem and a classical control problem. For this auxiliary control problem, the controller must choose a stopping time (optimal stopping), and the new control variables belong to a compact set. This equivalence is obtained by showing that the (discontinuous) state process governed by a singular control is given by a time transformation of an auxiliary state process governed by a classical bounded control. It is proved that the value functions for these two problems are equal. For a general form of the cost, the existence of an optimal singular control is established under certain technical hypotheses. Moreover, the problem of approximating singular optimal control by absolutely continuous controls is discussed in the same class of admissible controls.

Generalized Solutions in Nonlinear Stochastic Control Problems

An optimal stochastic control problem is considered for systems with unbounded controls satisfying an integral constraint. It is shown that there exists an optimal control within the class of generalized controls leading to impulse actions. Applying an approach of time transformation, developed recently for deterministic systems, the original control problem is shown to be equivalent to an optimal stopping problem. Moreover, the description of generalized solutions is given in terms of stochastic differential equations governed by a measure.

UAV Control on the Basis of 3D Landmark Bearing-Only Observations

The article presents an approach to the control of a UAV on the basis of 3D landmark observations. The novelty of the work is the usage of the 3D RANSAC algorithm developed on the basis of the landmarks’ position prediction with the aid of a modified Kalman-type filter. Modification of the filter based on the pseudo-measurements approach permits obtaining unbiased UAV position estimation with quadratic error characteristics. Modeling of UAV flight on the basis of the suggested algorithm shows good performance, even under significant external perturbations.