Abdullah Al-Dujaili

Dividing rectangles attack multi-objective optimization

Decomposition-based evolutionary algorithms have been applied with success to multi-objective optimization problems where they are broken into several subproblems, and solutions for the original problem are recognized in a coordinated manner. Motivated by the working principle of decomposition-based methods, viz. the “divide-and-conquer” paradigm; this paper is concerned with solving black-box multi-objective problems given a finite number of function evaluations by taking inspiration from the single-objective deterministic sampling method, DIRECT. In particular, we provide a multi-objective algorithmic instance of DIRECT, which we refer to as MO-DIRECT and investigate its performance with respect to established decomposition-based multi-objective techniques. Besides its asymptotic convergence to the Pareto front and its inherent balance between exploration and exploitation of the decision space, the proposed framework is flexible enough to incorporate indicator-based techniques, albeit at the cost of a greater space complexity when compared to the evolutionary counterpart.

Guppy: A GPU-like soft-core processor

The popularity of GPU programming languages that explicitly express thread-level parallelism leads to the question of whether they can also be used for programming reconfigurable accelerators. This paper describes Guppy, a GPU-like softcore processor based on the in-order LEON3 core. Our long-term vision is to have a unified programming paradigm for accelerators - regardless of whether they are FPGA or GPU based. While others have explored this from a high level hardware synthesis perspective, we chose to adopt the approach of a parametrically reconfigurable softcore. We will discuss the main architecture features of Guppy, compare its performance to the original core. Our design has been synthesized on a Xilinx Virtex 5 FPGA.

Multi-Objective Simultaneous Optimistic Optimization

Optimistic methods have been applied with success to single-objective optimization. Here, we attempt to bridge the gap between optimistic methods and multi-objective optimization. In particular, this paper is concerned with solving black-box multi-objective problems given a finite number of function evaluations and proposes an optimistic approach, which we refer to as the Multi-Objective Simultaneous Optimistic Optimization (MO-SOO). Popularized by multi-armed bandits, MO-SOO follows the optimism in the face of uncertainty principle to recognize Pareto optimal solutions, by combining several multi-armed bandits in a hierarchical structure over the feasible decision space of a multi-objective problem. Based on three assumptions about the objective functions smoothness and hierarchical partitioning, the algorithm finite-time and asymptotic convergence behaviors are analyzed. The finite-time analysis establishes an upper bound on the Pareto-compliant unary additive epsilon indicator characterized by the objectives smoothness as well as the structure of the Pareto front with respect to its extrema. On the other hand, the asymptotic analysis indicates the consistency property of MO-SOO. Moreover, we validate the theoretical provable performance of the algorithm on a set of synthetic problems. Finally, three-hundred bi-objective benchmark problems from the literature are used to substantiate the performance of the optimistic approach and compare it with three state-of-the-art stochastic algorithms, namely MOEA/D, MO-CMA-ES, and SMS-EMOA in terms of two Pareto-compliant quality indicators. Besides sound theoretical properties, MO-SOO shows a performance on a par with the top performing stochastic algorithm, viz. SMS-EMOA.

MSO: a framework for bound-constrained black-box global optimization algorithms

This paper addresses a class of algorithms for solving bound-constrained black-box global optimization problems. These algorithms partition the objective function domain over multiple scales in search for the global optimum. For such algorithms, we provide a generic procedure and refer to as multi-scale optimization (MSO). Furthermore, we propose a theoretical methodology to study the convergence of MSO algorithms based on three basic assumptions: (a) local Hölder continuity of the objective function f, (b) partitions boundedness, and (c) partitions sphericity. Moreover, the worst-case finite-time performance and convergence rate of several leading MSO algorithms, namely, Lipschitzian optimization methods, multi-level coordinate search, dividing rectangles, and optimistic optimization methods have been presented.

HumanCog: A cognitive architecture for solving optimization problems

Humans seek to select the best decision for a given problem in a process that is highly efficient and often ends with success. This is due to a high-order thinking skill: metacognition, which enables humans to be successful decision makers by constantly monitoring their cognitive activities based on earlier experience. Besides this, the social aspect of metacognition helps humans in monitoring their cognitive activities based on their peers experience and knowledge. Inspired by this, we propose HumanCog: a generic 3-layer architecture for solving optimization problems. HumanCog functions in a way that mimics human cognitive and metacognitive (self as well as social) behavior. The three layers in the network are cognitive layer, metacognitive layer and social cognitive layer. These three layers interact with each other such that accurate decision is made. As an initial work, we provide a simple realization of the HumanCog referred to as HumanCog-ver1, which self-regulates decision based on best experience. The performance evaluation on CEC 2015 and 2013 benchmark problems indicates promising results.

Multi-Objective Self Regulating Particle Swarm Optimization algorithm for BMOBench platform

These days, most of the real-world problems have become multi-criteria in nature and the demand for an effective multi-objective optimization algorithm has been significantly increased. This paper presents a new Multi-Objective Self-Regulating Particle Swarm Optimization (MOSRPSO) algorithm whereby the SRPSO algorithm originally developed for single objective problems has been modified to tackle with Multi-objective Optimization Problems (MOPs). The classical approach of Pareto dominance has been applied in the SRPSO framework together with the roulette wheel selection scheme for leader identification. The proposed MOSRPSO algorithm has been evaluated on all the hundred problems from Black-Box Multi-Objective Optimization Benchmarking (BMOBench) platform and the results are presented. The performance clearly indicate that MOSRPSO is a potential candidate for solving MOPs.

A MATLAB Toolbox for Surrogate-Assisted Multi-Objective Optimization: A Preliminary Study

Surrogate modeling has been a powerful ingredient for several algorithms tailored towards computionally-expensive optimization problems. Concerned with solving black-box multi-objective problems given a finite number of function evaluations and inspired by the recent advances in multi-objective algorithms, this paper presents-based on the MATSuMoTo library for single-objective optimization-a surrogate-based optimization toolbox for multi-objective problems. Moreover, in attempt to highlight the strengths and weaknesses of the employed methods, we benchmark the presented toolbox within the Black-box Optimization Benchmarking framework (BBOB 2016).

Empirical Assessment of Human Learning Principles Inspired PSO Algorithms on Continuous Black-Box Optimization Testbed

This paper benchmarks the performance of one of the recent research directions in the performance improvement of particle swarm optimization algorithm; human learning principles inspired PSO variants. This article discusses and provides performance comparison of nine different PSO variants. The Comparing Continuous Optimizers (COCO) methodology has been adopted in comparing these variants on the noiseless BBOB testbed, providing useful insight regarding their relative efficiency and effectiveness. This study provides the research community a comprehensive account of suitability of a PSO variant in solving selective class of problems under different budget settings. Further, certain rectifications/extensions have also been suggested for the selected PSO variants for possible performance enhancement. Overall, it has been observed that SL-PSO and MePSO are most suited for expensive and moderate budget settings respectively. Further, iSRPSO and TPLPSO have provided better solutions under cheap budget settings where iSRPSO has shown robust behaviour (better solutions over dimensions). We hope this paper would mark a milestone in assessing the human learning principles inspired PSO algorithms and used as a baseline for performance comparison.

GraphBPT: An Efficient Hierarchical Data Structure for Image Representation and Probabilistic Inference

This paper presents GraphBPT, a tool for hierarchical representation of images based on binary partition trees. It relies on a new BPT construction algorithm that have interesting tuning properties. Besides, access to image pixels from the tree is achieved efficiently with data compression techniques, and a textual representation of BPT is also provided for interoperability. Finally, we illustrate how the proposed tool takes benefit from probabilistic inference techniques by empowering the BPT with its equivalent factor graph. The relevance of GraphBPT is illustrated in the context of image segmentation.

Revisiting norm optimization for multi-objective black-box problems: a finite-time analysis

The complexity of Pareto fronts imposes a great challenge on the convergence analysis of multi-objective optimization methods. While most theoretical convergence studies have addressed finite-set and/or discrete problems, others have provided probabilistic guarantees, assumed a total order on the solutions, or studied their asymptotic behaviour. In this paper, we revisit the Tchebycheff weighted method in a hierarchical bandits setting and provide a finite-time bound on the Pareto-compliant additive