Manuel Castejón Limas

Outlier Detection and Data Cleaning in Multivariate Non-Normal Samples: The PAELLA Algorithm

A new method of outlier detection and data cleaning for both normal and non-normal multivariate data sets is proposed. It is based on an iterated local fit without a priori metric assumptions. We propose a new approach supported by finite mixture clustering which provides good results with large data sets. A multi-step structure, consisting of three phases, is developed. The importance of outlier detection in industrial modeling for open-loop control prediction is also described. The described algorithm gives good results both in simulations runs with artificial data sets and with experimental data sets recorded in a rubber factory. Finally, some discussion about this methodology is exposed.

Hierarchical clustering of subpopulations with a dissimilarity based on the likelihood ratio statistic: application to clustering massive data sets

The problem of clustering subpopulations on the basis of samples is considered within a statistical framework: a distribution for the variables is assumed for each subpopulation and the dissimilarity between any two populations is defined as the likelihood ratio statistic which compares the hypothesis that the two subpopulations differ in the parameter of their distributions to the hypothesis that they do not. A general algorithm for the construction of a hierarchical classification is described which has the important property of not having inversions in the dendrogram. The essential elements of the algorithm are specified for the case of well-known distributions (normal, multinomial and Poisson) and an outline of the general parametric case is also discussed. Several applications are discussed, the main one being a novel approach to dealing with massive data in the context of a two-step approach. After clustering the data in a reasonable number of ‘bins’ by a fast algorithm such as k-Means, we apply a version of our algorithm to the resulting bins. Multivariate normality for the means calculated on each bin is assumed: this is justified by the central limit theorem and the assumption that each bin contains a large number of units, an assumption generally justified when dealing with truly massive data such as currently found in modern data analysis. However, no assumption is made about the data generating distribution.

A data-driven manufacturing support system for rubber extrusion lines

A better control of extrusion processes offers clear advantages in the manufacturing of rubber profiles for the automotive industry. This work reports our experience in developing a support system aimed to ease the work of the extruder machinist while improving the quality of the profiles obtained. In order to build the system, an approach based on facts was adopted, following ISO 9000 standard quality principles. The data warehouse service available provided a wealth of information on the conditions of the running processes. The collected data, after being analysed with the appropriate data-mining techniques, allowed us to gain a better understanding of the process and to identify the main causes of variance. In particular, principal components analysis, Sammon projection and several classification techniques were applied for exploratory purposes. Different behaviours could be described for the extrusion process, allowing for the definition of a control strategy and, eventually, the development of a manufacturing support system. The estimates displayed by the system greatly improve the responsiveness of the machinist when the process departs from expected behaviour. The results of using this system in a local factory proved highly satisfactory and encouraging.

A Multi-agent Data Mining System for Defect Forecasting in a Decentralized Manufacturing Environment

This paper reports an experience on setting a multi-agent system to control a complex production environment, a steelmaking manufacturing plant. The decentralized character of such a plant fits perfectly with the approach of a control system by means of a multi-agent configuration. The agents devoted to rendering the superficial and internal defects maps, to developing and maintaining the learning context, to evaluating the coils entering the pickling line and to forecasting the remaining defects on the coil are described. Data mining techniques are used by the agents to gain access to the actual status of the manufacturing process, thus helping in the decision-making processes. This proves to be a great aid in improving the quality of the products and reducing both costs and the environmental footprint of the manufacturing process. The results of using such a system reinforce our belief in the approach presented.

Insights into the Prevalence of Software Project Defects

This paper analyses the effect of the effort distribution along the software development lifecycle on the prevalence of software defects. This analysis is based on data that was collected by the International Software Benchmarking Standards Group (ISBSG) on the development of 4,106 software projects. Data mining techniques have been applied to gain a better understanding of the behaviour of the project activities and to identify a link between the effort distribution and the prevalence of software defects. This analysis has been complemented with the use of a hierarchical clustering algorithm with a dissimilarity based on the likelihood ratio statistic, for exploratory purposes. As a result, different behaviours have been identified for this collection of software development projects, allowing for the definition of risk control strategies to diminish the number and impact of the software defects. It is expected that the use of similar estimations might greatly improve the awareness of project managers on the risks at hand.