Luc Boullart

Genetic programming: principles and applications

Abstract Genetic algorithms (GA) has given rise to two new fields of research where (global) optimisation is of crucial importance: ‘ genetic based machine learning ’ (GBML) and ‘ genetic programming ’ (GP). An introduction by the authors to GA and GBML was given in two previous papers (Eng. Appl. Artif. Intell. 9(6) (1996) 681; Eng. Appl. Artif. Intell. 13(4) (2000) 381). In this paper, the last domain (GP) will be introduced, thereby making up a trilogy which gives a general overview of the whole field. In this third part, an overview will be given of the basic concepts of GP as defined by Koza. A first (educational) example of GP is given by solving a simple symbolic regression of a sinus function. Finally, a more complex application is presented in which GP is used to construct the mathematical equations for an industrial process. To this end, the case study ‘fibre-to-yarn production process’ is introduced. The goal of this study is the automatic development of mathematical equations for the prediction of spinnability and (possible) resulting yarn strength. It is shown that (relatively) simple equations can be obtained which describe accurately 90% of the fibre-to-yarn database.

DAS28 best reflects the physician's clinical judgment of response to infliximab therapy in rheumatoid arthritis patients: validation of the DAS28 score in patients under infliximab treatment

This study is based on an expanded access program in which 511 patients suffering from active refractory rheumatoid arthritis (RA) were treated with intravenous infusions of infliximab (3 mg/kg+methotrexate (MTX)) at weeks 0, 2, 6 and every 8 weeks thereafter. At week 22, 474 patients were still in follow-up, of whom 102 (21.5%), who were not optimally responding to treatment, received a dose increase from week 30 onward. We aimed to build a model to discriminate the decision to give a dose increase. This decision was based on the treating rheumatologists clinical judgment and therefore can be considered as a clinical measure of insufficient response. Different single and composite measures at weeks 0, 6, 14 and 22, and their differences over time were taken into account for the model building. Ranking of the continuous variables based on areas under the curve of receiver-operating characteristic (ROC) curve analysis, displayed the momentary DAS28 (Disease Activity Score including a 28-joint count) as the most important discriminating variable. Subsequently, we proved that the response scores and the changes over time were less important than the momentary evaluations to discriminate the physicians decision. The final model we thus obtained was a model with only slightly better discriminative characteristics than the DAS28. Finally, we fitted a discriminant function using the single variables of the DAS28. This displayed similar scores and coefficients as the DAS28. In conclusion, we evaluated different variables and models to discriminate the treating rheumatologists decision to increase the dose of infliximab (+MTX), which indicates an insufficient response to infliximab at 3 mg/kg in patients with RA. We proved that the momentary DAS28 score correlates best with this decision and demonstrated the robustness of the score and the coefficients of the DAS28 in a cohort of RA patients under infliximab therapy.

ROC analysis in ordinal regression learning

Nowadays the area under the receiver operating characteristics (ROC) curve, which corresponds to the Wilcoxon-Mann-Whitney test statistic, is increasingly used as a performance measure for binary classification systems. In this article we present a natural generalization of this concept for more than two ordered categories, a setting known as ordinal regression. Our extension of the Wilcoxon-Mann-Whitney statistic now corresponds to the volume under an r-dimensional surface (VUS) for r ordered categories and differs from extensions recently proposed for multi-class classification. VUS rather evaluates the ranking returned by an ordinal regression model instead of measuring the error rate, a way of thinking which has especially advantages with skew class or cost distributions. We give theoretical and experimental evidence of the advantages and different behavior of VUS compared to error rate, mean absolute error and other ranking-based performance measures for ordinal regression. The results demonstrate that the models produced by ordinal regression algorithms minimizing the error rate or a preference learning based loss, not necessarily impose a good ranking on the data.

OPTIMIZING THE FIBER-TO-YARN PRODUCTION PROCESS WITH A COMBINED NEURAL NETWORK/GENETIC ALGORITHM APPROACH

An important aspect of the fiber-to-yam production process is the quality of the resulting yarn. The yarn should have optimal product characteristics (and minimal faults). In theory, this objective can be realized using an optimization algorithm. The complexity of a fiber-to-yarn process is very high, however, and no mathematical function is known to exist that represents the whole process. This paper presents a method to simulate and optimize the fiber-to-yam production process using a neural network combined with a genetic algorithm. The neural network is used to model the process, with the machine settings and fiber quality parameters as input and the yarn tenacity and elongation as output. The genetic algorithm is used afterward to optimize the input parameters for obtaining the best yarns. Since this is a multi-objective optimization, the genetic algorithm is enforced with a sharing function and a Pareto optimization. The paper shows that simultaneous optimization of yarn qualities is easily achieved as a function of the necessary (optimal) input parameters, and that the results are considerably better than current manual machine intervention. The last part of the paper is dedicated to finding an optimal mixture of available fiber qualities based on the predictions of the genetic algorithm.

Four-year follow-up of infliximab therapy in rheumatoid arthritis patients with long-standing refractory disease: attrition and long-term evolution of disease activity

Although there is strong evidence supporting the short-term efficacy and safety of anti-tumour necrosis factor-α agents, few studies have examined the long-term effects. We evaluated 511 patients with long-standing refractory rheumatoid arthritis treated with intravenous infusions of infliximab 3 mg/kg at weeks 0, 2, 6, and 14 and every 8 weeks thereafter for 4 years. Among the initial 511 patients included in the study, 479 could be evaluated; of these, 295 (61.6%) were still receiving infliximab treatment at year 4 of follow-up. The most common reasons for treatment discontinuation were lack of efficacy (65 patients, 13.6%), safety (81 patients, 16.9%), and elective change (38 patients, 7.9%). Analysis of disease activity scores (DAS28 [disease activity score based on the 28-joint count]) over time showed that, after the initial rapid improvement during the first 6 to 22 weeks of therapy, a further decrease in disease activity of 0.2 units in the DAS28 score per year was observed. DAS28 scores, measured at week 14 or 22, were found to predict subsequent discontinuation due to lack of efficacy. In conclusion, long-term maintenance therapy with infliximab 3 mg/kg is effective in producing further reductions in disease activity. Disease activity measured by the DAS28 at week 14 or 22 of infliximab therapy was the best predictor of long-term attrition.

Optimising a production process by a neural network/genetic algorithm approach

Abstract An important aspect of production control is the quality of the resulting end product. The end product should have optimal product characteristics and minimal faults. In theory, both objectives can be realised using an optimisation algorithm. However, the complexity of a production process may be very high. In most cases no mathematical function can be found to represent the production process. In this paper a method is presented to simulate a complex production process using a neural network. The subsequent optimisation is done by means of a genetic algorithm. The method is applied to the case study of a spinning (fibre-yarn) production process. The neural network is used to model the process, with the machine settings and fibre quality parameters as input, and the yarn tenacity (yarn strength) and elongation as output. The genetic algorithm is then used to optimise the input parameters for obtaining the best yarns. Since it is a multiobjective optimisation, the genetic algorithm is enforced with a sharing function and a Pareto optimisation. The paper shows that simultaneous optimisation of yarn qualities is easily achieved as a function of the necessary (optimal) input parameters, and that the results are considerably better than current manual machine intervention. The paper concludes by indicating future research towards making an optimal mixture of available fibre qualities.

Using genetic algorithms to design a control strategy of an industrial process

Abstract In this paper a methodology is presented to design a control strategy to optimise a complex spinning (fibre-yarn) production process, using a neural network combined with genetic algorithms. The neural network is used to model the process, with the machine setpoints and raw fibre quality parameters as input, and with the yarn tenacity and elongation as output. Genetic algorithms are used in two ways: • to optimise the architecture and the underlying parameters of the neural network, in order to achieve the most effective model of the production process; • to obtain setpoint values and raw material characteristics for an optimal quality of the spinned yarns.

On the scalability of ordered multi-class ROC analysis

Receiver operating characteristics (ROC) analysis provides a way to select possibly optimal models for discriminating two kinds of objects without the need of specifying the cost or class distribution. It is nowadays established as a standard analysis tool in different domains, including medical decision making, pattern recognition and machine learning. Recently, an extension to the ordered multi-class case has been proposed, in which the concept of a ROC curve is generalized to an r-dimensional surface for r ordered categories, and the volume under this ROC surface (VUS) measures the overall power of a model to classify objects of the various categories. However, the computation of this criterion as well as the U-statistics estimators of its variance and covariance for two models is believed to be complex. New algorithms to compute VUS and its (co)variance estimator are presented. In particular, the volume under the ROC surface can be found very efficiently with a simple dynamic program dominated by a single sorting operation on the data set. For the variance and covariance, the respective estimators are reformulated as a series of recurrent functions over layered data graphs and subsequently these functions are rapidly evaluated with a dynamic program. Simulation experiments confirm that the presented algorithms scale well with respect to the size of the data set and the number of categories. For example, the volume under the ROC surface could be rapidly computed on very large data sets of more than 500 000 instances, while a naive implementation spent much more time on data sets of size less than 1000.

Classifying carpets based on laser scanner data

Nowadays the quality of carpets is in industry still determined through visual assessment by human experts, although this procedure suffers from a number of drawbacks. Existing computer models for automatic assessment of carpet wear are at this moment not capable of matching the human expertise. Therefore, we present a completely new approach to tackle this problem. A three-dimensional laser scanner is used to obtain a digital copy of the carpet. Due to the specific characteristics of the laser scanner data, new algorithms are developed to extract relevant information from the raw data. These features are used as input to a classifier system that defines a partial ranking over the objects. To this end, ordinal regression and multi-class classification models are applied. Experiments demonstrate that our approach gives rise to promising results with correlations up to 0.77 between extracted features and quality labels. The performance obtained with nested cross-validation, including a C-index of more than 0.95, an accuracy of 76% and only 3% serious errors of a full point, gives rise to a substantial improvement compared to other approaches mentioned in the literature.

Kernel-based learning methods for preference aggregation

The mathematical representation of human preferences has been a subject of study for researchers in different fields. In multi-criteria decision making (MCDM) and fuzzy modeling, preference models are typically constructed by interacting with the human decision maker (DM). However, it is known that a DM often has difficulties to specify precise values for certain parameters of the model. He/she instead feels more comfortable to give holistic judgements for some of the alternatives. Inference and elicitation procedures then assist the DM to find a satisfactory model and to assess unjudged alternatives. In a related but more statistical way, machine learning algorithms can also infer preference models with similar setups and purposes, but here less interaction with the DM is required/allowed. In this article we discuss the main differences between both types of inference and, in particular, we present a hybrid approach that combines the best of both worlds. This approach consists of a very general kernel-based framework for constructing and inferring preference models. Additive models, for which interpretability is preserved, and utility models can be considered as special cases. Besides generality, important benefits of this approach are its robustness to noise and good scalability. We show in detail how this framework can be utilized to aggregate single-criterion outranking relations, resulting in a flexible class of preference models for which domain knowledge can be specified by a DM.