Clemens Otte

Safe and Interpretable Machine Learning: A Methodological Review

When learning models from data, the interpretability of the resulting model is often mandatory. For example, safety-related applications for automation and control require that the correctness of the model must be ensured not only for the available data but for all possible input combinations. Thus, understanding what the model has learned and in particular how it will extrapolate to unseen data is a crucial concern. The paper discusses suitable learning methods for classification and regression. For classification problems, we review an approach based on an ensemble of nonlinear low-dimensional submodels, where each submodel is simple enough to be completely verified by domain experts. For regression problems, we review related approaches that try to achieve interpretability by using low-dimensional submodels (for instance, MARS and tree-growing methods). We compare them with symbolic regression, which is a different approach based on genetic algorithms. Finally, a novel approach is proposed for combining a symbolic regression model, which is shown to be easily interpretable, with a Gaussian Process. The combined model has an improved accuracy and provides error bounds in the sense that the deviation from the verified symbolic model is always kept below a defined limit.

Improving the accuracy of network intrusion detectors by input-dependent stacking

Network intrusion detectors analyze network traffic for detecting attacks in computer networks. Achieving a high detection accuracy and in particular a low number of false alarms is crucial for their practical use. In this paper a new stacking approach is suggested for improving the detection accuracy of anomaly and misuse detectors in network intrusion detection systems. Each detector gets a stacked module as a corrective element that is learned on training data. The stacked module shall raise the detector score in case of a true attack and lower the score in case of a normal connection. This is achieved by combining the detector score with context information (statistical features) about the respective connection, making it possible for example to learn in which context a certain detector is reliable and where it is not. The approach is empirically evaluated using real HTTP and FTP network traffic. The results show that the detectors enhanced by stacking typically are significantly better than the original detectors.

Ensemble Learning for Multi-source Information Fusion

In this paper, a new ensemble learning method is proposed. The main objective of this approach is to jointly use knowledge-based and data-driven submodels in the modeling process. The integration of knowledge-based submodels is of particular interest, since they are able to provide information not contained in the data. On the other hand, data-driven models can complement the knowledge-based models with respect to input space coverage. For the task of appropriately integrating the different models, a method for partitioning the input space for the given models is introduced. The benefits of this approach are demonstrated for a real-world application.

Tackling Multiple-Instance Problems in Safety-Related Domains by Quasilinear SVM

In this paper we introduce a preprocessing method for safety-related applications. Since we concentrate on scenarios with highly unbalanced misclassification costs, we briefly discuss a variation of multiple-instance learning (MIL) and recall soft margin hyperplane classifiers; in particular the principle of a support vector machine (SVM). According to this classifier, we present a training set selection method for learning quasilinear SVMs which guarantee both high accuracy and model complexity to a lower degree. We conclude with annotating on a real-world application and potential extensions for future research in this domain.

An EM-Based Piecewise Linear Regression Algorithm

This contribution describes an EM-like piecewise linear regression algorithm that uses information about the target variable to determine a meaningful partitioning of the input space. The main goal of this approach is to incorporate information about the target variable in the prototype selection process of a piecewise regression approach. Furthermore, the proposed approach is designed to provide an interpretable solution by restricting the dimensionality of the local regression models. We will show that our approach achieves a similar predictive performance on benchmark problems compared to standard regression methods --- while the model complexity of our approach is reduced.

Interpretable semi-parametric regression models with defined error bounds

Unreliable extrapolation of data-driven models hinders their applicability not only in safety-related domains. The paper discusses how model interpretability and uncertainty estimates can address this problem. A new semi-parametric approach is proposed for providing an interpretable model with improved accuracy by combining a symbolic regression model with a residual Gaussian Process. While the learned symbolic model is highly interpretable the residual model usually is not. However, by limiting the output of the residual model to a defined range a worst-case guarantee can be given in the sense that the maximal deviation from the symbolic model is always below a defined limit. The limitation of the residual model can include the uncertainty estimate of the Gaussian Process, thus giving the residual model more impact in high-confidence regions. When ranking the accuracy and interpretability of several different approaches on the SARCOS data benchmark the proposed combination yields the best result.

Multi-Class Extension of Verifiable Ensemble Models for Safety-Related Applications

This contribution discusses two different strategies for extending a verifiable ensemble approach for binary classification tasks to also solve multi-class problems. The binary ensemble approach was developed with the objective of providing interpretable classification models for use in safety-related application domains. It is based on low-dimensional submodels. Each submodel uses only a low-dimensional subspace of the complete input space facilitating the visual interpretation and validation by domain experts. Thus, the correct inter- and extrapolation behavior can be guaranteed. The extension to multi-class problems is not straightforward because common multi-class extensions might induce inconsistent decisions. The proposed approaches avoid such inconsistencies by introducing a hierarchy of misclassification costs. We will show that by following such a hierarchy the extension of the binary ensemble becomes feasible and the desirable properties of the binary classification approach for safety-related problems can be maintained.

Maschinelles Lernen von validierbaren Klassifikatoren zur autonomen Steuerung sicherheitsrelevanter Systeme (Machine Learning of Verifiable Classifiers for Autonomous Control of Safety-related Systems).

Zusammenfassung Im Bereich sicherheitsrelevanter Anwendungen ist es besonders wichtig sicherzustellen, dass die gelernte Lösung im gesamten Eingaberaum korrekt ist. Es wird eine Methode zum Lernen von Klassifikationsmodellen vorgestellt, die auch bei sicherheitskritischen Steuerungsaufgaben einsetzbar ist. Hierfür wird ein Ensemble von niedrigdimensionalen Teilmodellen erzeugt, bei dem jedes Teilmodell von einem Experten interpretiert und validiert werden kann.

Verifiable Ensembles of Low-Dimensional Submodels for Multi-class Problems with Imbalanced Misclassification Costs

In this chapter, we discuss different strategies of extending an ensemble approach based on local binary classifiers to solve multi-class problems. The ensembles of binary classifiers were developed with the objective of providing interpretable submodels s for use in safety-related application domains. The ensembles assume highly imbalanced misclassification costs between the two classes. The extension to multi-class problems is not straightforward because common multi-class extensions might induce inconsistent decisions. We propose a solution of this problem that avoids such inconsistencies by introducing a hierarchy of misclassification costs. We show that by following such a hierarchy it becomes feasible to extend the binary ensemble, to maintain the desirable properties (that is, the good interpretability) of the binary ensemble, and to achieve a good predictive performance.