Waldemar Korłub

Big Data Paradigm Developed in Volunteer Grid System with Genetic Programming Scheduler

Artificial intelligence techniques are capable to handle a large amount of information collected over the web. In this paper, big data paradigm has been studied in volunteer and grid system called Comcute that is optimized by a genetic programming scheduler. This scheduler can optimize load balancing and resource cost. Genetic programming optimizer has been applied for finding the Pareto solu-tions. Finally, some results from numerical experiments have been shown.

Genetic programming with negative selection for volunteer computing system optimization

Volunteer computing systems like BOINC or Comcute are strongly supported by a great number of volunteers who contribute resources of their computers via the Web. So, the high efficiency of such grid system is required, and that is why we have formulated a multi-criterion optimization problem for a volunteer grid system design. In that dilemma, both the cost of the host system and workload of a bottleneck host are minimized. On the other hand, a reliability of this grid structure is maximized. Moreover, genetic programming has been applied to determine the Pareto solutions. Finally, a negative selection procedure to handle constraints has been discussed.

Big Data Processing by Volunteer Computing Supported by Intelligent Agents

In this paper, volunteer computing systems have been proposed for big data processing. Moreover, intelligent agents have been developed to efficiency improvement of a grid middleware layer. In consequence, an intelligent volunteer grid has been equipped with agents that belong to five sets. The first one consists of some user tasks. Furthermore, two kinds of semi-intelligent tasks have been introduced to implement a middleware layer. Finally, two agents based on genetic programming as well as harmony search have been applied to optimize big data processing.

Depth images filtering in distributed streaming

Abstract In this paper, we propose a distributed system for point cloud processing and transferring them via computer network regarding to effectiveness-related requirements. We discuss the comparison of point cloud filters focusing on their usage for streaming optimization. For the filtering step of the stream pipeline processing we evaluate four filters: Voxel Grid, Radial Outliner Remover, Statistical Outlier Removal and Pass Through. For each of the filters we perform a series of tests for evaluating the impact on the point cloud size and transmitting frequency (analysed for various fps ratio). We present results of the optimization process used for point cloud consolidation in a distributed environment. We describe the processing of the point clouds before and after the transmission. Pre- and post-processing allow the user to send the cloud via network without any delays. The proposed pre-processing compression of the cloud and the post-processing reconstruction of it are focused on assuring that the end-user application obtains the cloud with a given precision.

Improving Effectiveness of SVM Classifier for Large Scale Data

The paper presents our approach to SVM implementation in parallel environment. We describe how classification learning and prediction phases were pararellised. We also propose a method for limiting the number of necessary computations during classifier construction. Our method, named one-vs-near, is an extension of typical one-vs-all approach that is used for binary classifiers to work with multiclass problems. We perform experiments of scalability and quality of the implementation. The results show that the proposed solution allows to scale up SVM that gives reasonable quality results. The proposed one-vs-near method significantly improves effectiveness of the classifier construction.

Harmony Search to Self-Configuration of Fault-Tolerant Grids for Big Data

In this paper, harmony search algorithms have been proposed to self-configuration of fault-tolerant grids for big data processing. Some tasks related to big data processing have been considered. Moreover, two criteria have been applied to evaluate quality of grids. The first criterion is a probability that all tasks meet their deadlines and the second one is grid reliability. Furthermore, some intelligent agents based on harmony search have been developed to support a middleware layer of grids.

Harmony Search for Data Mining with Big Data

In this paper, some harmony search algorithms have been proposed for data mining with big data. Three areas of big data processing have been studied to apply new metaheuristics. The first problem is related to MapReduce architecture that can be supported by a team of harmony search agents in grid infrastructure. The second dilemma involves development of harmony search in preprocessing of data series before data mining. Moreover, harmony search as a classification algorithm is studied as the third application. Finally, some outcomes for numerical experiments are submitted.

Harmony Search for Self-configuration of Fault–Tolerant and Intelligent Grids

In this paper, harmony search algorithms have been proposed to self-configuration of intelligent grids for big data processing. Self-configuration of computer grids lies in the fact that new computer nodes are automatically configured by software agents and then integrated into the grid. A base node works due to several configuration parameters that define some aspects of data communications and energy power consumption. We propose some optimization agents that are based on harmony search to find a suboptimal configuration of fault–tolerant grids processing big data. Criteria such as probability that all tasks meet their deadlines and also a reliability of grid are considered. Finally, some experimental results have been considered.

Collective citizens' behavior modelling with support of the Internet of Things and Big Data

In this paper, collective human behaviors are modelled by a development of Big Data mining related to the Internet of Things. Some studies under MapReduce architectures have been carried out to improve an efficiency of Big Data mining. Intelligent agents in data mining have been analyzed for smart city systems, as well as data mining has been described by genetic programming. Furthermore, artificial neural networks have been discussed in data mining as well as an analysis of Tweeters blogs for citizens has been proposed. Finally, some numerical experiments with fire spread around Tricity, Poland have been submitted.

Genetic Programming for Interaction Efficient Supporting in Volunteer Computing Systems

Volunteer computing systems provide a middleware for interaction between project owners and great number volunteers. In this chapter, a genetic programming paradigm has been proposed to a multi-objective scheduler design for efficient using some resources of volunteer computers via the web. In a studied problem, genetic scheduler can optimize both a workload of a bottleneck computer and cost of system. Genetic programming has been applied for finding the Pareto solutions by applying an immunological procedure. Finally, some numerical experiment outcomes have been discussed.