Jan Vanthienen

Bankruptcy prediction with least squares support vector machine classifiers

Classification algorithms like linear discriminant analysis and logistic regression are popular linear techniques for modelling and predicting corporate distress. These techniques aim at finding an optimal linear combination of explanatory input variables, such as, e.g., solvency and liquidity ratios, in order to analyse, model and predict corporate default risk. Recently, performant kernel based nonlinear classification techniques, like support vector machines, least squares support vector machines and kernel fisher discriminant analysis, have been developed. Basically, these methods map the inputs first in a nonlinear way to a high dimensional kernel-induced feature space, in which a linear classifier is constructed in the second step. Practical expressions are obtained in the so-called dual space by application of Mercers theorem. In this paper, we explain the relations between linear and nonlinear kernel based classification and illustrate their performance on predicting bankruptcy of mid-cap firms in Belgium and the Netherlands.

An empirical assessment of kernel type performance for least squares support vector machine classifiers

Recently, a modified version of support vector machines (SVMs), least-squares SVM (LS-SVM) classifiers, has been introduced, which is closely related to a form of ridge regression-type SVMs. In LS-SVMs, the classifier is obtained as the solution to a linear system instead of a quadratic programming problem. In this paper, UCI (University of California at Irvine) benchmark data sets are used to evaluate the performance of LS-SVM classifiers with linear, polynomial and radial basis function (RBF) kernels. The hyperparameters of the LS-SVM problem formulation are tuned using a 10-fold cross-validation procedure and a grid search mechanism. When comparing the performance of a nonlinear (RBF or polynomial) LS-SVM classifier with that of a linear LS-SVM, additional insight can be gained into the degree of nonlinearity of the classification problem at hand. Using a statistical motivation, it is concluded that RBF LS-SVM classifiers consistently yield among the best results for each data set.

A tool-supported approach to inter-tabular verification

Abstract The use of decision tables to verify knowledge based systems (KBS) has been advocated several times in the validation and verification (V&V) literature. However, one of the main drawbacks of these systems is that they fail to detect anomalies that occur over rule chains. In a decision table based context this means that anomalies that occur due to interactions between tables are neglected. These anomalies are called inter-tabular anomalies. In this paper we investigate an approach that deals with inter-tabular anomalies. One of the prerequisites for the approach was that it could be used by the knowledge engineer during the development of the KBS. This requires that the anomaly check can be performed on-line. As a result, the approach partly uses heuristics where exhaustive checks would be too inefficient. All detection facilities that will be described have been implemented in a table-based development tool called prologa . The use of this tool will be briefly illustrated. In addition, some experiences in verifying large knowledge bases are discussed.

An initial comparison of a fuzzy neural classifier and a decision tree based classifier

At the present time a large number of AI methods have been developed in the field of pattern classification. In this paper, we will compare the performance of a well-known algorithm in machine learning (C4.5) with a recently proposed algorithm in the fuzzy set community (NEFCLASS). We will compare the algorithms both on the accuracy attained and on the size of the induced rule base. Additionally, we will investigate how the selected algorithms perform after they have been pre-processed by discretization and feature selection.

Building Intelligent Credit Scoring Systems Using Decision Tables

Accuracy and comprehensibility are two important criteria when developing decision support systems for credit scoring. In this paper, we focus on the second criterion and propose the use of decision tables as an alternative knowledge visualisation formalism which lends itself very well to building intelligent and user-friendly credit scoring systems. Starting from a set of propositional if-then rules extracted by a neural network rule extraction algorithm, we construct decision tables and demonstrate their efficiency and user-friendliness for two real-life credit scoring cases.

A synthesis of fuzzy rule-based system verification

Abstract The verification of fuzzy rule bases for anomalies has received increasing attention these last few years. Many different approaches have been suggested and many are still under investigation. In this paper, we give a synthesis of methods proposed in literature that try to extend the verification of classical rule bases to the case of fuzzy knowledge modeling, without needing a set of representative input. Within this area of fuzzy validation and verification (V&V) we identify two dual lines of thought leading to what is identified as static and dynamic anomaly detection methods. Static anomaly detection essentially tries to use similarity, affinity or matching measures to identify anomalies within a fuzzy rule base. It is assumed that the detection methods can be the same as those used in a non-fuzzy environment, except that the former measures indicate the degree of matching of two fuzzy expressions. Dynamic anomaly detection starts from the basic idea that any anomaly within a knowledge representation formalism, i.e. fuzzy if–then rules, can be identified by performing a dynamic analysis of the knowledge system, even without providing special input to the system. By imposing a constraint on the results of inference for an anomaly not to occur, one creates definitions of the anomalies that can only be verified if the inference process, and thereby the fuzzy inference operator is involved in the analysis. The major outcome of the confrontation between both approaches is that their results, stated in terms of necessary and/or sufficient conditions for anomaly detection within a particular situation, are difficult to reconcile. The duality between approaches seems to have translated into a duality in results. This article addresses precisely this issue by presenting a theoretical framework which enables us to effectively evaluate the results of both static and dynamic verification theories.

Wrapped Feature Selection For BinaryClassification Bayesian Regularisation NeuralNetworks: A Database Marketing Application

In this paper, we try to validate existing theory on and develop additional insight into repeat purchasing behaviour in a direct-marketing setting by means of an illuminating case study. The case involves the detection and qualification of the most relevant RFM (Recency, Frequency and Monetary) features, using a wrapped feature selection method in a neural network context. Results indicate that elimination of redundant/irrelevant features by means of the discussed feature selection method, allows to significantly reduce model complexity without degrading generalisation ability. It is precisely this issue that will allow to infer some very interesting marketing conclusions concerning the relative importance of the RFM-predictor categories. The empirical findings highlight the importance of a combined use of all three RFM variables in predicting repeat purchase behaviour. However, the study also reveals the dominant role of the frequency variable. Results indicate that a model including only frequency variables still yields satisfactory classification accuracy compared to the optimally reduced model.