Reyes Pavón

Automatic parameter tuning with a Bayesian case-based reasoning system. A case of study

The parameter setting of an algorithm that will result in optimal performance differs across problem instance domains. Users spend a lot of time tuning algorithms for their specific problem domain, and this time could be saved by an automatic approach for parameter tuning. In this paper, we present a system that recommends the parameter configuration of an algorithm that solves a problem, conditioned by the particular features of the current problem instance to be solved. The proposed system is based on a basic adjustment model designed by authors (Pavon, R., Diaz, F., & Luzon, V. (2008). A model for parameter setting based on Bayesian networks. Engineering Applications of Artificial Intelligence, 21(1), 14-25) in which starting from experimental results concerning the search for solutions to several instances of the problem, a Bayesian network (BN) is induced and tries to infer the best configuration for the algorithm used. However, taking into account that the optimal parameter configuration may differ considerably across problem instances of a specific domain, the present work extends the former incorporating additional information about problem instances and using the case-based reasoning (CBR) methodology as the framework integrator for the different instances from the same problem, where each problem instance deals with a specific BN. In this way, the system will automatically recommend a parameter configuration for a given algorithm according to the characteristics of the problem instance at hand and past experience of similar instances. As an example, we empirically evaluate our Bayesian CBR system to tune a genetic algorithm for solving the root identification problem. The experimental results demonstrate the validity of the model proposed.

A Reasoning Model for CBR_BDI Agents Using an Adaptable Fuzzy Inference System

This paper proposes to automate the generation of shellfish exploitation plans, which are elaborated by Galician extracting entities. For achieving this objective a CBR-BDI agent will be used. This agent will adapt the exploitation plans to the environmental characteristics of each school of shellfish. This kind of agents develops its activity into changing and dynamic environments, so the reasoning model that they include must be emphasised. The agent reasoning model is guided by the phases of the CBR life cycle, using different technologies for each phase. The use of an adaptative neuro-fuzzy inference system in the reuse phase must be highlighted.

Evaluating the effect of unbalanced data in biomedical document classification.

Nowadays, document classification has become an interesting research field. Partly, this is due to the increasing availability of biomedical information in digital form which is necessary to catalogue and organize. In this context, machine learning techniques are usually applied to text classification by using a general inductive process that automatically builds a text classifier from a set of pre-classified documents. Related with this domain, imbalanced data is a well-known problem in many practical applications of knowledge discovery and its effects on the performance of standard classifiers are remarkable. In this paper, we investigate the application of a Bayesian Network (BN) model for the triage of documents, which are represented by the association of different MeSH terms. Our results show that BNs are adequate for describing conditional independencies between MeSH terms and that MeSH ontology is a valuable resource for representing Medline documents at different abstraction levels. Moreover, we perform an extensive experimental evaluation to investigate if the classification of Medline documents using a BN classifier poses additional challenges when dealing with class-imbalanced prediction. The evaluation involves two methods, under-sampling and cost-sensitive learning. We conclude that BN classifier is sensitive to both balancing strategies and existing techniques can improve its overall performance.

A model for parameter setting based on Bayesian networks

One of the difficulties that the user faces when using a model to solve a problem is that, before running the model, a set of parameter values have to be specified. Deciding on an appropriate set of parameter values is not an easy task. Over the years, several standard optimization methods, as well as various alternative approaches according to the problem at hand, have been proposed for parameter setting. These techniques have their merits and demerits, but usually they have a fairly restricted application range, including a lack of generality or the need of user supervision. This paper proposes a meta-model that generates the recommendations about the best parameter values for the model of interest. Its main characteristic is that it is an automatic meta-model that can be applied to any model. For evaluation purposes and in order to be able to compare our results with results obtained by others, a real geometric problem was selected. The experiments show the validity of the proposed adjustment model.

Automatic parameter tuning for Evolutionary Algorithms using a Bayesian Case-Based Reasoning system

The widespread use and applicability of Evolutionary Algorithms is due in part to the ability to adapt them to a particular problem-solving context by tuning their parameters. This is one of the problems that a user faces when applying an Evolutionary Algorithm to solve a given problem. Before running the algorithm, the user typically has to specify values for a number of parameters, such as population size, selection rate, and probability operators. This paper empirically assesses the performance of an automatic parameter tuning system in order to avoid the problems of time requirements and the interaction of parameters. The system, based on Bayesian Networks and Case-Based Reasoning methodology, estimates the best parameter setting for maximizing the performance of Evolutionary Algorithms. The algorithms are applied to solve a basic problem in constraint-based, geometric parametric modeling, as an instance of general constraint-satisfaction problems. The experimental results demonstrate the validity of the proposed system and its potential effectiveness for configuring algorithms.

A dynamic model for integrating simple web spam classification techniques

Techniques and heuristics applied for spam dissemination.Rule-based classification model for web spam detection.Knowledge actualization based on incremental learning.Filter performance improvement by classifier fusion.WSF2 framework publicly available under LGPL license. Over the last years, Internet spam content has spread enormously inside web sites mainly due to the emergence of new web technologies oriented towards the online sharing of resources and information. In such a situation, both academia and industry have shown their concern to accurately detect and effectively control web spam, resulting in a good number of anti-spam techniques currently available. However, the successful integration of different algorithms for web spam classification is still a challenge. In this context, the present study introduces WSF2, a novel web spam filtering framework specifically designed to take advantage of multiple classification schemes and algorithms. In detail, our approach encodes the life cycle of a case-based reasoning system, being able to use appropriate knowledge and dynamically adjust different parameters to ensure continuous improvement in filtering precision with the passage of time. In order to correctly evaluate the effectiveness of the dynamic model, we designed a set of experiments involving a publicly available corpus, as well as different simple well-known classifiers and ensemble approaches. The results revealed that WSF2 performed well, being able to take advantage of each classifier and to achieve a better performance when compared to other alternatives. WSF2 is an open-source project licensed under the terms of the LGPL publicly available at https://sourceforge.net/projects/wsf2c/.

Assessing the Impact of Class-Imbalanced Data for Classifying Relevant/Irrelevant Medline Documents

Imbalanced data is a well-known common problem in many practical applications of machine learning and its effects on the performance of standard classifiers are remarkable. In this paper we investigate if the classification of Medline documents using MeSH controlled vocabulary poses additional challenges when dealing with class-imbalanced prediction. For this task, we evaluate the performance of Bayesian networks by using some available strategies to overcome the effect of class imbalance. Our results show both that Bayesian network classifiers are sensitive to class imbalance and existing techniques can improve their overall performance.

WSF2: a novel framework for filtering web spam

Over the last years, research on web spam filtering has gained interest from both academia and industry. In this context, although there are a good number of successful antispam techniques available (i.e., content-based, link-based, and hiding), an adequate combination of different algorithms supported by an advanced web spam filtering platform would offer more promising results. To this end, we propose the WSF2 framework, a new platform particularly suitable for filtering spam content on web pages. Currently, our framework allows the easy combination of different filtering techniques including, but not limited to, regular expressions and well-known classifiers (i.e., Naive Bayes, Support Vector Machines, and C5.0). Applying our WSF2 framework over the publicly available WEBSPAM-UK2007 corpus, we have been able to demonstrate that a simple combination of different techniques is able to improve the accuracy of single classifiers on web spam detection. As a result, we conclude that the proposed filtering platform is a powerful tool for boosting applied research in this area.

Assessing the Suitability of MeSH Ontology for Classifying Medline Documents

Automated document classification has become an interesting research field due to the increasing availability of biomedical information in digital form which is necessary to catalogue and organize. In this context, the machine learning paradigm is usually applied to text classification, according to which a general inductive process automatically builds a text classifier from a set of pre-classified documents. In this work we investigate the application of a Bayesian network model for the triage of documents represented by the association of different MeSH terms. Our results show both that Bayesian networks are adequate for describing conditional independencies between MeSH terms and that MeSH ontology is a valuable resource for representing Medline documents at different abstraction levels.

Using inductive learning to assess compound feed production in cooperative poultry farms

Abstract Production scheduling is one of the most important functions in a production company. As a consequence, in recent decades various methods have been proposed for the modeling and solution of particular scheduling problems. In this context, a special case is that of centralized feed manufacturing plants supplying animal food in a cooperative poultry environment. In this paper, we present the SP4 system, an integrated software environment that combines a statistical method (used to calculate the previous consumption data, mortality indices and feed delivery types), a machine learning method (M5P and IBk models – used to calculate the total amount of feed consumed by type) and an ad hoc algorithm which makes flexible orders for compound feed production forecasting. The data used for this study was provided by a leading Spanish Company (Coren Cooperative) specialized in animal feed production and delivery. Raw data (from the years 2007 and 2008) was built from client orders, company production logs, information about the number of animals at different farms and truck trips to the clients. To ensure that the developed system is able to reproduce acceptable results for the unseeable future, we have evaluated various aggregate measures to forecast error (MSE, MAE, MAPE, ME) during the validation of the models. The results reveal that the proposed system performed well, being able to track the dynamic non-linear trend and seasonality, as well as the numerous interactions between correlated variables.