Jacek B. Krawczyk

A modified genetic algorithm for optimal control problems

Abstract This paper studies the application of a genetic algorithm to discrete-time optimal control problems. Numerical results obtained here are compared with ones yielded by GAMS, a system for construction and solution of large and complex mathematical programming models. While GAMS appears to work well only for linear quadratic optimal control problems or problems with short horizon, the genetic algorithm applies to more general problems equally well.

Relaxation algorithms to find Nash equilibria with economic applications

Recent theoretical studies have shown that a relaxation algorithm can be used to find noncooperative equilibria of synchronous infinite games with nonlinear payoff functions and coupled constraints. In this study, we introduce an improvement to the algorithm, such as the steepest-descent step-size control, for which the convergence of the algorithm is proved. The algorithm is then tested on several economic applications. In particular, a River Basin Pollution problem is considered where coupled environmental constraints are crucial for the relevant model definition. Numerical runs demonstrate fast convergence of the algorithm for a wide range of parameters.

Genetic algorithms and optimal control problems

The application of the genetic algorithm to discrete-time optimal control problems is studied. The numerical results obtained are compared with a system for construction and solution of large and complex mathematical programming models, GAMS. It is shown that while GAMS appears to work well only for linear-quadratic optimal control problems or problems with a short horizon, the genetic algorithm applies to more general problems and appears to be competitive with search-based methods.<<ETX>>

Games and Dynamic Games

Dynamic games arise between players (individuals, firms, countries, animals, etc.) when the strategic interactions among them recur over time and decisions made during one period affect both current and future payoffs. Dynamic games provide conceptually rich paradigms and tools to deal with these situations. This volume provides a uniform approach to game theory and illustrates it with present-day applications to economics and management, including environmental, with the emphasis on dynamic games. At the end of each chapter a case study called game engineering (GE) is provided, to help readers understand how problems of high social priority, such as environmental negotiations, exploitation of common resources, can be modeled as games and how solutions can be engineered.

Numerical solutions to coupled-constraint (or generalised Nash) equilibrium problems

This paper is about games where the agents face constraints in the combined strategy space (unlike in standard games where the action sets are defined separately for each player) and about computational methods for solutions to such games. The motivation examples for such games include electricity generation problems with transmission capacity constraints, environmental management to control pollution and internet switching to comply to buffers of bounded capacity. In each such problem a regulator may aim at compliance to standards or quotas through taxes or charges. The relevant solution concept for these games has been known under several names like generalised Nash equilibrium, coupled constraint equilibrium and more. Existing numerical methods converging to such an equilibrium will be explained. Application examples of use of NIRA, which is a suite of Matlab routines that implement one of the methods, will be provided.

Relaxation Algorithms in Finding Nash Equilibria

Relaxation algorithms provide a powerful method of finding noncooperative equilibria in general synchronous games. Through use of the Nikaido-Isoda function, the Nash solution to a broad category of constrained, multiplayer, non-zerosum games can easily be found. We provide solutions to some simple games using this procedure and extend ourselves to more difficult games involving coupled constraints and multiple discrete time periods using a program developed in Matlab.

SATISFICING SOLUTIONS TO A MONETARY POLICY PROBLEM

Herbert A. Simon, 1978 Economics Nobel Prize laureate, talked about satisficing (his neologism) rather than optimizing as being what economists really need. Indeed, optimization might be an unsuitable solution procedure (in that it suggests a unique “optimal†solution) for problems where many solutions could be satisfactory. We think that looking for an applicable monetary policy is a problem of this kind because there is no unique way in which a central bank can achieve a desired inflation (unemployment, etc.) path. We think that it is viability theory, which is a relatively young area of mathematics, that rigorously captures the essence of satisficing. We aim to use viability analysis to analyze a simple macro policy model and show how some robust adjustment rules can be endogenously obtained.

Computation of viability kernels: a case study of by-catch fisheries

Traditional means of studying environmental economics and management problems consist of optimal control and dynamic game models that are solved for optimal or equilibrium strategies. Notwithstanding the possibility of multiple equilibria, the models’ users—managers or planners—will usually be provided with a single optimal or equilibrium strategy no matter how reliable, or unreliable, the underlying models and their parameters are. In this paper we follow an alternative approach to policy making that is based on viability theory. It establishes “satisficing” (in the sense of Simon), or viable, policies that keep the dynamic system in a constraint set and are, generically, multiple and amenable to each manager’s own prioritisation. Moreover, they can depend on fewer parameters than the optimal or equilibrium strategies and hence be more robust. For the determination of these (viable) policies, computation of “viability kernels” is crucial. We introduce a MATLAB application, under the name of VIKAASA, which allows us to compute approximations to viability kernels. We discuss two algorithms implemented in VIKAASA. One approximates the viability kernel by the locus of state space positions for which solutions to an auxiliary cost-minimising optimal control problem can be found. The lack of any solution implies the infinite value function and indicates an evolution which leaves the constraint set in finite time, therefore defining the point from which the evolution originates as belonging to the kernel’s complement. The other algorithm accepts a point as viable if the system’s dynamics can be stabilised from this point. We comment on the pros and cons of each algorithm. We apply viability theory and the VIKAASA software to a problem of by-catch fisheries exploited by one or two fleets and provide rules concerning the proportion of fish biomass and the fishing effort that a sustainable fishery’s exploitation should follow.

Numerical Solutions to Lump-Sum Pension Fund Problems that Can Yield Left-Skewed Fund Return Distributions

The paper is about pension fund problems where an agent pays an amount x0 to the fund manager and is repaid, after time T, a lump sum x(T). Such problems admit an analytical solution for specific, rather unrealistic formulations. Several practical pension fund problems are converted in the paper into Markov decision chains solvable through approximations. In particular, a couple of problems with a non-differentiable asymmetric (with respect to risk) utility function are solved, for which left-skewed fund-return distributions are reported. Such distributions ascribe more probability to higher payoffs than the right-skewed ones that are common among analytical solutions.

A parallel Matlab package for approximating the solution to a continuous-time stochastic optimal control problem

Computing the solution to a stochastic optimal control problem is difficult. A method of approximating a solution to a given stochatic optimal problem was developed in [1]. This paper describes a suite of Matlab functions implementing this method of approximating a solution to a given continuous stochastic optimal control problem.