Kenneth Holmström

Global Optimization of Costly Nonconvex Functions Using Radial Basis Functions

The paper considers global optimization of costly objective functions, i.e. the problem of finding the global minimum when there are several local minima and each function value takes considerable CPU time to compute. Such problems often arise in industrial and financial applications, where a function value could be a result of a time-consuming computer simulation or optimization. Derivatives are most often hard to obtain, and the algorithms presented make no use of such information. Several algorithms to handle the global optimization problem are described, but the emphasis is on a new method by Gutmann and Powell, A radial basis function method for global optimization. This method is a response surface method, similar to the Efficient Global Optimization (EGO) method of Jones. Our Matlab implementation of the Radial Basis Function (RBF) method is described in detail and we analyze its efficiency on the standard test problem set of Dixon-Szego, as well as its applicability on a real life industrial problem from train design optimization. The results show that our implementation of the RBF algorithm is very efficient on the standard test problems compared to other known solvers, but even more interesting, it performs extremely well on the train design optimization problem.The paper considers global optimization of costly objective functions, i.e. the problem of finding the global minimum when there are several local minima and each function value takes considerable CPU time to compute. Such problems often arise in industrial and financial applications, where a function value could be a result of a time-consuming computer simulation or optimization. Derivatives are most often hard to obtain, and the algorithms presented make no use of such information.Several algorithms to handle the global optimization problem are described, but the emphasis is on a new method by Gutmann and Powell, A radial basis function method for global optimization. This method is a response surface method, similar to the Efficient Global Optimization (EGO) method of Jones. Our Matlab implementation of the Radial Basis Function (RBF) method is described in detail and we analyze its efficiency on the standard test problem set of Dixon-Szegö, as well as its applicability on a real life industrial problem from train design optimization. The results show that our implementation of the RBF algorithm is very efficient on the standard test problems compared to other known solvers, but even more interesting, it performs extremely well on the train design optimization problem.

A review of the parameter estimation problem of fitting positive exponential sums to empirical data

Exponential sum models are used frequently: in heat diffusion, diffusion of chemical compounds, time series in medicine, economics, physical sciences and technology. Thus it is important to find good methods for the estimation of parameters in exponential sums. In this paper we review and discuss results from the last forty years of research. There are many different ways of estimating parameters in exponential sums and model a fit criterion, which gives a valid result from the fit. We find that a good choice is a weighted two-norm objective function, with weights based on the maximum likelihood (ML) criterion. If the number of exponential terms is unknown, statistical methods based on an information criterion or cross-validation can be used to determine the optimal number. It is suitable to use hybrid Gauss-Newton (GN) and quasi-Newton algorithms to find the unknown parameters in the constrained weighted nonlinear least-squares (NLLS) problem formulated using an maximal likelihood (ML) objective function. The problem is highly ill conditioned and it is crucial to find good starting values for the parameters. To find the initial parameter values, a modified Prony method or a method based upon rewriting partial sums as geometrical sums is proposed.

The TOMLAB Optimization Environment

The TOMLAB Optimization Environment is a powerful optimization tool in MATLAB, which incooperates many results from the last 40 years of research in the field. More than 70 different algorithms for linear, discrete, global and nonlinear optimization are implemented in TOMLAB, and a large number of C and Fortran solvers are also fully integrated. The environment is call-compatible with Math-Works’ Optimization Toolbox, and supports problems formulated in AMPL. This chapter discusses the design and contents of TOMLAB, and examplifies its usage on a practical optimization problem. The objective is to present the overall design and describe how to efficiently model a problem in TOMLAB using the standard structures and assign statements. More information about TOMLAB is available at URL: http: //tomlab. BIZ.

Computing stable loads for pallets

This paper describes an Integer Programming model for generating stable loading patterns for the Pallet Loading Problem under several stability criteria. The results obtained during evaluation show great improvement in the number of stable patterns in comparison with results reported earlier. Moreover, most of the solved cases also ensure optimality in terms of utilization of a pallet.

Global Controller Optimization Using Horowitz Bounds

Abstract A procedure for global optimization of PID type controller parameters for SISO plants with model uncertainty is presented. Robustness to the uncertainties is guaranteed by the use of Horowitz bounds, which are used as constraints when low frequency performance is optimized. The basic idea of both the optimization and the parameter tuning is to formulate separate criteria for low, mid and high frequency closed loop properties. The trade-off between stability margins, high frequency robustness and low frequency performance is then elucidated and, hence, the final choice of parameters is facilitated. The optimization problems are non-convex and ill-conditioned and we use a combination of new global and standard local optimization algorithms available in the TOMLAB optimization environment to solve the problem. The method does not rely on a good initial guess and converges fast and robustly. It is applied to a controller structure comparison for a plant with an uncertain mechanical resonance. For a given control activity and stability margin as well as identical tuning parameters it is shown that a PID controller achieves slightly improved low frequency performance compared to an ℋ ∞ controller based on loop-shaping. The reason for this somewhat surprising result is the roll-off in the ℋ ∞ controller, which adds additional high frequency robustness compared to the PID controller. Computationally, a factor of 10–20 has been gained compared to an earlier, less general, version of the procedure.

Multi criteria controller design for uncertain MIMO systems using global non-convex optimization

A controller design procedure for MIMO systems with explicit plant uncertainties is suggested. In the proposed method, the closed loop analysis is integrated in the controller synthesis by an outer optimization loop that optimizes the low frequency performance of the closed loop system subject to additional performance and robustness criteria constrained by specifications. Modifications of the specifications then clearly shows the trade off between performance and robustness in different frequency regions. The controller synthesis is based on PID weighted H/sub /spl infin// loop shaping where the structured singular value (/spl mu/) is used in the optimization to guarantee that the performance specifications are met in spite of the uncertainties. The optimization problems are costly, non-convex and ill-conditioned and we use a combination of new global and standard local optimization algorithms available in the TOMLAB optimization environment to solve the problem. The method, which does not rely on a good initial guess and converges robustly in finite time, is applied to a 2/spl times/2 model of a distillation column and the example shows how a slight modification of specifications on the sensitivity function and control activity may give large improvements in the load disturbance rejection.

The relevance of trends for predictions of stock returns

Whfkqlfdo suhglfwlrq ri vwrfn uhwxuqv lv nqrzq wr eh dq h{wuhpho| gl!fxow sureohp1 Lq wklv uhsruw zh xvh wkh frqfhsw ri wuhqgv dv suhglfwru yduldeohv1 D vwdwlvwlfdo lqyhvwljdwlrq ri wkh uhohydqfh ri d wuhqg frqfhsw iru suhglfwlrqv ri ixwxuh uhwxuqv lv suhvhqwhg iru lqglylgxdo vwrfnv rq wkh Vzhglvk vwrfn pdunhw1 Qhjdwlyh fruuhodwlrq iru vwurqj qhjdwlyh wuhqgv lv vkrzq1 Wkh ryhudoo fruuhodwlrq ehwzhhq wuhqgv dqg ixwxuh uhwxuqv lv krzhyhu irxqg wr eh yhu| zhdn1 Wuhqg yduldeohv duh ixuwkhu xvhg dv lqsxw yduldeohv lq d n0qhduhvw qhljkeru dqdo|vlv wr qg sdwwhuqv ri wuhqg ydoxhv wkdw uhvxow lq qrq0udqgrp ixwxuh uhwxuqv1 Wkh n0qhduhvw qhljkeru dojrulwkp lv h{whqghg zlwk d vhohfwlrq surfhgxuh wr qg uhjlrqv lq wkh lqsxw vsdfh zkhuh wkh ixwxuh uhwxuqv duh qrq0v|pphwulfdoo| glvwulexwhg1 Wkh qhz dojrulwkp lv vxffhvvixoo| whvwhg rq duwl fldo vwrfn gdwd zlwk wuhqglqj sdwwhuqv lqwurgxfhg1 Wkh dojrulwkp lv dovr dssolhg wr d qxpehu ri qdwlrqdo vwrfn lqgh{hv= wkh Dphulfdq Grz Mrqhv/ wkh Jhupdq GD[/ wkh Eulwlvk IWVH/ dqg wkh Vzhglvk Jhqhudolqgh{1 Wkh uhvxowv duh srvlwyh iru vrph ri wkh whvwhg lqgh{hv dqg qhjdwlyh iru rwkhuv1 Ixuwkhu whvwv pxvw eh frqgxfwhg lq rughu wr jlyh vwdwlvwlfdoo| vljql fdqw uhvxowv1 Wkh vxjjhvwhg dojrulwkp dovr kdv qdwxudo dssolfdwlrq lq duhdv rwkhu wkdq vwrfn suhglfwlrq1 Lw dgguhvvhv vlwxdwlrqv lq zklfk wkh ryhudoo suhglfwdelolw| lv orz dqg fdq rqo| eh h{shfwhg wr dsso| lq lqghwhuplqdwh uhjlrqv ri wkh lqsxw vsdfh1