Cristóbal Barba-González

jMetalSP: A framework for dynamic multi-objective big data optimization

Abstract Multi-objective metaheuristics have become popular techniques for dealing with complex optimization problems composed of a number of conflicting functions. Nowadays, we are in the Big Data era, so metaheuristics must be able to solve dynamic problems that may vary over time due to the processing and analysis of several streaming data sources. As this is a new field, there is a need for software platforms to solve dynamic multi-objective Big Data optimization problems. In this paper, we present jMetalSP, which combines the multi-objective optimization features of the jMetal framework with the streaming facilities of the Apache Spark cluster computing system. Thus, existing state-of-the-art multi-objective metaheuristics can be easily adapted to deal with dynamic optimization problems that are fed by multiple streaming data sources. Moreover, these algorithms can take advantage of the parallel computing features of Spark. We describe the architecture of jMetalSP and show how it can be used to solve a dynamic bi-objective instance of the Traveling Salesman Problem (TSP) based on New York Citys real-time traffic data. We have also carried out an experimental study to assess the performance of the resultant jMetalSP application in a Hadoop cluster composed of 100 nodes.

Dynamic Multi-Objective Optimization with jMetal and Spark: A Case Study

Technologies for Big Data and Data Science are receiving increasing research interest nowadays. This paper introduces the prototyping architecture of a tool aimed to solve Big Data Optimization problems. Our tool combines the jMetal framework for multi-objective optimization with Apache Spark, a technology that is gaining momentum. In particular, we make use of the streaming facilities of Spark to feed an optimization problem with data from different sources. We demonstrate the use of our tool by solving a dynamic bi-objective instance of the Traveling Salesman Problem (TSP) based on near real-time traffic data from New York City, which is updated several times per minute. Our experiment shows that both jMetal and Spark can be integrated providing a software platform to deal with dynamic multi-optimization problems.

Design and architecture of the jMetaISP framework

jMetaISP is a framework for dynamic multi-objective Big Data optimization. It combines the jMetal multi-objective framework with the Apache Spark cluster computing system to allow the solving of dynamic optimization problems from a number of external streaming data sources in Big Data contexts. In this paper, we describe the current status of the jMetaISP project, focusing mainly in its design and internal architecture, with the aim of offering a comprehensive view of its main features to interested researchers. Among the covered features, we describe the main components of a jMetalSP application, including dynamic problems, dynamic algorithms, streaming data sources, and data consumers. For practical purposes, we describe two test cases to illustrate how to address dynamic combinatorial and dynamic continuous optimization problems by using the proposed framework.

BIGOWL: Knowledge centered Big Data analytics

Abstract Knowledge extraction and incorporation is currently considered to be beneficial for efficient Big Data analytics. Knowledge can take part in workflow design, constraint definition, parameter selection and configuration, human interactive and decision-making strategies. This paper proposes BIGOWL, an ontology to support knowledge management in Big Data analytics. BIGOWL is designed to cover a wide vocabulary of terms concerning Big Data analytics workflows, including their components and how they are connected, from data sources to the analytics visualization. It also takes into consideration aspects such as parameters, restrictions and formats. This ontology defines not only the taxonomic relationships between the different concepts, but also instances representing specific individuals to guide the users in the design of Big Data analytics workflows. For testing purposes, two case studies are developed, which consists in: first, real-world streaming processing with Spark of traffic Open Data, for route optimization in urban environment of New York city; and second, data mining classification of an academic dataset on local/cloud platforms. The analytics workflows resulting from the BIGOWL semantic model are validated and successfully evaluated.

Extending the Speed-Constrained Multi-objective PSO (SMPSO) with Reference Point Based Preference Articulation

The Speed-constrained Multi-objective PSO (SMPSO) is an approach featuring an external bounded archive to store non-dominated solutions found during the search and out of which leaders that guide the particles are chosen. Here, we introduce SMPSO/RP, an extension of SMPSO based on the idea of reference point archives. These are external archives with an associated reference point so that only solutions that are dominated by the reference point or that dominate it are considered for their possible addition. SMPSO/RP can manage several reference point archives, so it can effectively be used to focus the search on one or more regions of interest. Furthermore, the algorithm allows interactively changing the reference points during its execution. Additionally, the particles of the swarm can be evaluated in parallel. We compare SMPSO/RP with respect to three other reference point based algorithms. Our results indicate that our proposed approach outperforms the other techniques with respect to which it was compared when solving a variety of problems by selecting both achievable and unachievable reference points. A real-world application related to civil engineering is also included to show up the real applicability of SMPSO/RP.

Artificial Decision Maker Driven by PSO: An Approach for Testing Reference Point Based Interactive Methods

Over the years, many interactive multiobjective optimization methods based on a reference point have been proposed. With a reference point, the decision maker indicates desirable objective function values to iteratively direct the solution process. However, when analyzing the performance of these methods, a critical issue is how to systematically involve decision makers. A recent approach to this problem is to replace a decision maker with an artificial one to be able to systematically evaluate and compare reference point based interactive methods in controlled experiments. In this study, a new artificial decision maker is proposed, which reuses the dynamics of particle swarm optimization for guiding the generation of consecutive reference points, hence, replacing the decision maker in preference articulation. We use the artificial decision maker to compare interactive methods. We demonstrate the artificial decision maker using the DTLZ benchmark problems with 3, 5 and 7 objectives to compare R-NSGA-II and WASF-GA as interactive methods. The experimental results show that the proposed artificial decision maker is useful and efficient. It offers an intuitive and flexible mechanism to capture the current context when testing interactive methods for decision making.

InDM2: Interactive Dynamic Multi-Objective Decision Making Using Evolutionary Algorithms

Abstract Dynamic optimization problems involving two or more conflicting objectives appear in many real-world scenarios, and more cases are expected to appear in the near future with the increasing interest in the analysis of streaming data sources in the context of Big Data applications. However, approaches combining dynamic multi-objective optimization with preference articulation are still scarce. In this paper, we propose a new dynamic multi-objective optimization algorithm called InDM2 that allows the preferences of the decision maker (DM) to be incorporated into the search process. When solving a dynamic multi-objective optimization problem with InDM2, the DM can not only express her/his preferences by means of one or more reference points (which define the desired region of interest), but these points can be also modified interactively. InDM2 is enhanced with methods to graphically display the different approximations of the region of interest obtained during the optimization process. In this way, the DM is able to inspect and change, in optimization time, the desired region of interest according to the information displayed. We describe the main features of InDM2 and detail how it is implemented. Its performance is illustrated using both synthetic and real-world dynamic multi-objective optimization problems.