Remigijus Paulavičius

Globally-biased Disimpl algorithm for expensive global optimization

Direct-type global optimization algorithms often spend an excessive number of function evaluations on problems with many local optima exploring suboptimal local minima, thereby delaying discovery of the global minimum. In this paper, a globally-biased simplicial partition Disimpl algorithm for global optimization of expensive Lipschitz continuous functions with an unknown Lipschitz constant is proposed. A scheme for an adaptive balancing of local and global information during the search is introduced, implemented, experimentally investigated, and compared with the well-known Direct and Directl methods. Extensive numerical experiments executed on 800 multidimensional multiextremal test functions show a promising performance of the new acceleration technique with respect to competitors.

Investigation of selection strategies in branch and bound algorithm with simplicial partitions and combination of Lipschitz bounds

Speed and memory requirements of branch and bound algorithms depend on the selection strategy of which candidate node to process next. The goal of this paper is to experimentally investigate this influence to the performance of sequential and parallel branch and bound algorithms. The experiments have been performed solving a number of multidimensional test problems for global optimization. Branch and bound algorithm using simplicial partitions and combination of Lipschitz bounds has been investigated. Similar results may be expected for other branch and bound algorithms.

Simplicial Lipschitz optimization without the Lipschitz constant

In this paper we propose a new simplicial partition-based deterministic algorithm for global optimization of Lipschitz-continuous functions without requiring any knowledge of the Lipschitz constant. Our algorithm is motivated by the well-known Direct algorithm which evaluates the objective function on a set of points that tries to cover the most promising subregions of the feasible region. Almost all previous modifications of Direct algorithm use hyper-rectangular partitions. However, other types of partitions may be more suitable for some optimization problems. Simplicial partitions may be preferable when the initial feasible region is either already a simplex or may be covered by one or a manageable number of simplices. Therefore in this paper we propose and investigate simplicial versions of the partition-based algorithm. In the case of simplicial partitions, definition of potentially optimal subregion cannot be the same as in the rectangular version. In this paper we propose and investigate two definitions of potentially optimal simplices: one involves function values at the vertices of the simplex and another uses function value at the centroid of the simplex. We use experimental investigation to compare performance of the algorithms with different definitions of potentially optimal partitions. The experimental investigation shows, that proposed simplicial algorithm gives very competitive results to Direct algorithm using standard test problems and performs particularly well when the search space and the numbers of local and global optimizers may be reduced by taking into account symmetries of the objective function.

Parallel branch and bound for global optimization with combination of Lipschitz bounds

The solution of multidimensional Lipschitz optimization problem requires a lot of computing time and memory resources. Parallel OpenMP and MPI versions of branch and bound algorithm with simplicial partitions and Lipschitz bounds were created, investigated and compared in this paper. The efficiency of the developed parallel algorithms is investigated by solving multidimensional test problems for global optimization.

Advantages of simplicial partitioning for Lipschitz optimization problems with linear constraints

The well known DIRECT (DIviding RECTangles) algorithm for global optimization requires bound constraints on variables and does not naturally address additional linear or nonlinear constraints. A feasible region defined by linear constraints may be covered by simplices, therefore simplicial partitioning may tackle linear constraints in a very subtle way. In this paper we demonstrate this advantage of simplicial partitioning by applying a recently proposed deterministic simplicial partitions based DISIMPL algorithm for optimization problems defined by general linear constraints (Lc-DISIMPL). An extensive experimental investigation reveals advantages of this approach to such problems comparing with different constraint-handling methods, proposed for use with DIRECT. Furthermore the Lc-DISIMPL algorithm gives very competitive results compared to a derivative-free particle swarm algorithm (PSwarm) which was previously shown to give very promising results. Moreover, DISIMPL guarantees the convergence to the global solution, whereas the PSwarm algorithm sometimes fails to converge to the global minimum.

Global optimization using the branch‐and‐bound algorithm with a combination of Lipschitz bounds over simplices

Abstract Many problems in economy may be formulated as global optimization problems. Most numerically promising methods for solution of multivariate unconstrained Lipschitz optimization problems of dimension greater than 2 use rectangular or simplicial branch‐and‐bound techniques with computationally cheap, but rather crude lower bounds. The proposed branch‐and‐bound algorithm with simplicial partitions for global optimization uses a combination of 2 types of Lipschitz bounds. One is an improved Lipschitz bound with the first norm. The other is a combination of simple bounds with different norms. The efficiency of the proposed global optimization algorithm is evaluated experimentally and compared with the results of other well‐known algorithms. The proposed algorithm often outperforms the comparable branch‐and‐bound algorithms.

Analysis of different norms and corresponding Lipschitz constants for global optimization

Abstract The paper discusses how the used norm and corresponding Lipschitz constant influence the speed of algorithms for global optimization. For this reason Lipschitz constants corresponding to different norms were estimated. Different test functions for global optimization were solved using branch‐and‐bound algorithm for Lipschitz optimization with different norms. Experiments have shown that the best results are achieved when combination of extreme (infinite and first) and sometimes Euclidean norms is used.

On a Hybrid MPI-Pthread Approach for Simplicial Branch-and-Bound

We investigate models that efficiently map branch-and-bound algorithms on a distributed computer architecture using a case of multidimensional Lipschitz Global Optimization. A combination of MPI and Pthreads is studied: MPI for distributed computation (inter-node) and Pthreads for multicore computation (intra-node). That model adapts the algorithm to the characteristics of the architecture at hand with an increasing number of nodes. Dynamic load balancing is performed in intra-node space through dynamic generation of threads. Results show performance improvements compared to OpenMP and MPI versions used in previous work.

Influence of Lipschitz bounds on the speed of global optimization

Abstract Global optimization methods based on Lipschitz bounds have been analyzed and applied widely to solve various optimization problems. In this paper a bound for Lipschitz function is proposed, which is computed using function values at the vertices of a simplex and the radius of the circumscribed sphere. The efficiency of a branch and bound algorithm with proposed bound and combinations of bounds is evaluated experimentally while solving a number of multidimensional test problems for global optimization. The influence of different bounds on the performance of a branch and bound algorithm has been investigated.

Dynamic and hierarchical load-balancing techniques applied to parallel branch-and-bound methods

Most Lipschitzian Global Optimization algorithms perform an exhaustive search using a branch-and-bound (B&B) scheme. The question is how to run multi-dimensional Lipschitz Global Optimization in parallel, such that the implementation depending on the used platform is efficient. Previous work shows a parallel version developed for multicore nodes with two levels of parallelism: intra-node and inter-node. On intra-node level, one can perform dynamic load balancing by generating threads dynamically. Threads end when they complete their assigned work. The inter-node level carries out a static load balancing using MPI. In general, algorithm design depends on the characteristics of problems to be solved. There are several ways to improve performance of general parallel B&B algorithms. Specifically, we are interested in how to apply them to parallel Lipschitz Global Optimization algorithms. Operations like selecting the next subproblem to be evaluated become critical in parallel B&B schemes. We study Depth and Hybrid (Best-Depth) options as selection criterion. Previous work, using only MPI or OpenMP, discard not only the broadcasting of the best found upper bound of the solution due to its high average cost/performance but also the use of dynamic load balancing. Here we check how broadcasting the upper bound affects the developed MPI-Pthreads algorithm. Additionally, we study how to perform dynamic load balancing at inter-node level. Experimental results show which designs perform better on which type of instances for the used computational architecture.