Davy Sannen

Assessment of the influence of adaptive components in trainable surface inspection systems

In this paper, we present a framework for the classification of images in surface inspection tasks and address several key aspects of the processing chain from the original image to the final classification result. A major contribution of this paper is a quantitative assessment of how incorporating adaptivity into the feature calculation, the feature pre-processing, and into the classifiers themselves, influences the final image classification performance. Hereby, results achieved on a range of artificial and real-world test data from applications in printing, die-casting, metal processing and food production are presented.

A multilevel information fusion approach for visual quality inspection

For visual quality inspection systems to be applicable in industrial settings, it is mandatory that they are highly flexible, robust and accurate. In order to improve these characteristics a multilevel information fusion approach is presented. A first fusion step at the feature-level enables the system to learn from an undefined number of potential defects which might be segmented from the images. This allows for the quality control operators to label the data at the image-level and the sub-image-level, and use this information during the learning process. Additionally, the operators are allowed to provide a confidence measure for their labelling. The additional information obtained from the increased flexibility of the operator inputs allows to build more accurate classifiers. A second fusion step at the decision-level combines the classifications of different classifiers, making the system more accurate and more robust with respect to the classification method chosen. The experimental results, using various artificial and real-world visual quality inspection data sets, show that each of these fusion approaches can significantly improve the classification accuracy. If both information fusion approaches are combined the accuracy increases even further, significantly outperforming each of the fusion approaches on their own.

Towards incremental classifier fusion

To process the large amounts of data industrial systems are producing nowadays, machine learning techniques have shown their usefulness in many applications. As the amounts of data being generated are getting huge, the need for machine learning methods which can deal with them in an appropriate way - i.e. methods which can be adapted incrementally - becomes very important. Ensembles of classifiers have been shown to be able to improve the predictive accuracy as well as the robustness of single classification methods. In this work novel incremental variants of several well-known classifier fusion methods (Fuzzy Integral, Decision Templates, Dempster-Shafer Combination and Discounted Dempster-Shafer Combination) are presented. Furthermore, a novel incremental classifier fusion method called Incremental Direct Cluster-based ensemble will be introduced, which exploits an evolving clustering approach. A flexible and interactive framework for on-line learning will be introduced, in which the ensemble methods are adapted incrementally in a sample-wise manner together with their base classifiers. The performance of this framework and the proposed incremental classifiers fusion methods therein are evaluated on five real-world visual quality inspection tasks, captured on-line from an industrial CD imprint production process, together with five data sets from the UCI repository.

Increasing On-line Classification Performance Using Incremental Classifier Fusion

To process the large amounts of data industrial systems are producing nowadays, machine learning techniques have shown their usefulness in many applications. As the amounts of data being generated are getting huge, the need for machine learning methods which can deal with them in an appropriate way -- i.e.\ methods which can be adapted incrementally -- becomes very important. Ensembles of classifiers have been shown to be able to improve the predictive accuracy as well as the robustness of single classification methods. In this work novel incremental variants of several well-known classifier fusion methods (Fuzzy Integral, Decision Templates, Dempster-Shafer Combination and Discounted Dempster-Shafer Combination) are presented. Furthermore, a novel incremental classifier fusion method called Incremental Direct Cluster-based fusion will be introduced, which exploits an evolving clustering approach. A flexible and interactive framework for on-line learning will be introduced, in which the ensemble (classifier fusion) methods are adapted incrementally in a sample-wise manner together with their base classifiers. The performance of this framework and the proposed incremental classifiers fusion methods therein are evaluated on five real-world visual quality inspection tasks, captured on-line from an industrial CD imprint production process.

Making early predictions of the accuracy of machine learning classifiers

The accuracy of machine learning systems is a widely studied research topic. Established techniques such as cross validation predict the accuracy on unseen data of the classifier produced by applying a given learning method to a given training data set. However, they do not predict whether incurring the cost of obtaining more data and undergoing further training will lead to higher accuracy. In this chapter, we investigate techniques for making such early predictions. We note that when a machine learning algorithm is presented with a training set the classifier produced, and hence its error, will depend on the characteristics of the algorithm, on training set’s size, and also on its specific composition. In particular we hypothesize that if a number of classifiers are produced, and their observed error is decomposed into bias and variance terms, then although these components may behave differently, their behavior may be predictable. Experimental results confirm this hypothesis, and show that our predictions are very highly correlated with the values observed after undertaking the extra training. This has particular relevance to learning in nonstationary environments, since we can use our characterization of bias and variance to detect whether perceived changes in the data stream arise from sampling variability or because the underlying data distributions have changed, which can be perceived as changes in bias.

Active Grading Ensembles for Learning Visual Quality Control from Multiple Humans

When applying Machine Learning technology to real-world applications, such as visual quality inspection, several practical issues need to be taken care of. One problem is posed by the reality that usually there are multiple human operators doing the inspection, who will inevitable contradict each other occasionally. In this paper a framework is proposed which is able to deal with this issue, based on trained ensembles of classifiers. Most ensemble techniques have however difficulties learning in these circumstances. Therefore several novel enhancements to the Grading ensemble technique are proposed within this framework --- called Active Grading . The Active Grading algorithm is evaluated on data obtained from a real-world industrial system for visual quality inspection of the printing of labels on CDs, which was labelled independently by four different human operators and their supervisor, and compared to the standard Grading algorithm and a range of other ensemble (classifier fusion) techniques.

Learning visual quality inspection from multiple humans using ensembles of classifiers

Visual quality inspection systems nowadays require the highest possible flexibility. Therefore, the reality that multiple human operators may be training the system has to be taken into account. This paper provides an analysis of this problem and presents a framework which is able to learn from multiple humans. This approach has important advantages over systems which are unable to do so, such as a consistent level of quality of the products, the ability to give operator-specific feedback, the ability to capture the knowledge of every operator separately and an easier training of the system. The level of contradiction between the decisions of the operators is assessed for data obtained from a real-world industrial system for visual quality inspection of the printing of labels on CDs, which was labelled separately by five different operators. The results of the experiments show that the system is able to resolve many of the contradictions which are present in the data. Furthermore, it is shown that in several cases the system even performs better than a classifier which is trained on the data provided by the supervisor itself.

The active grading ensemble framework for learning visual quality inspection from multiple humans

When applying machine learning technology to real-world applications, such as visual quality inspection, several practical issues need to be taken care of. One problem is posed by the reality that usually there are multiple human operators doing the inspection, who will inevitably contradict each other for some of the products to be inspected. In this paper an architecture for learning visual quality inspection is proposed which can be trained by multiple human operators, based on trained ensembles of classifiers. Most of the applicable ensemble techniques have however difficulties learning in these circumstances. In order to effectively train the system a novel ensemble framework is proposed as an enhancement of the grading ensemble technique—called active grading. The active grading algorithms are evaluated on data obtained from a real-world industrial system for visual quality inspection of the printing of labels on CDs, which was labelled independently by four different human operators and their supervisor, and compared to the standard grading algorithm and a range of other ensemble (classifier fusion) techniques.

Incremental Classifier Fusion and Its Applications in Industrial Monitoring and Diagnostics

Pattern recognition techniques have shown their usefulness for monitoring and diagnosing many industrial applications. The increasing production rates and the growing databases generated by these applications require learning techniques that can adapt their models incrementally, without revisiting previously used data. Ensembles of classifiers have been shown to improve the predictive accuracy as well as the robustness of classification systems. In this work, several well-known classifier fusion methods (Fuzzy Integral, Decision Templates, Dempster–Shafer Combination, and Discounted Dempster–Shafer Combination) are extended to allow incremental adaptation. Additionally, an incremental classifier fusion method using an evolving clustering approach is introduced—named Incremental Direct Cluster-based ensemble. A framework for strict incremental learning is proposed in which the ensemble and its member classifiers are adapted concurrently. The proposed incremental classifier fusion methods are evaluated within this framework for two industrial applications: online visual quality inspection of CD imprints and prediction of maintenance actions for copiers from a large historical database.