Karsten Luebke

klaR Analyzing German Business Cycles

Decision making often asks for classification. We will present a new R package klaR including functions to build, check, tune, visualize, and compare classification rules. The software is illustrated by means of a case study of prediction of the German economy’s business cycle phases.

Localized Linear Discriminant Analysis

Despite its age, the Linear Discriminant Analysis performs well even in situations where the underlying premises like normally distributed data with constant covariance matrices over all classes are not met. It is, however, a global technique that does not regard the nature of an individual observation to be classified. By weighting each training observation according to its distance to the observation of interest, a global classifier can be transformed into an observation specific approach. So far, this has been done for logistic discrimination. By using LDA instead, the computation of the local classifier is much simpler. Moreover, it is ready for applications in multi-class situations.

Rapid and brief communication: Improving feature extraction by replacing the Fisher criterion by an upper error bound

A lot of alternatives and constraints have been proposed in order to improve the Fisher criterion. But most of them are not linked to the error rate, the primary interest in many applications of classification. By introducing an upper bound for the error rate a criterion is developed which can improve the classification performance.

Generation of prediction optimal projection on latent factors by a stochastic search algorithm

Abstract The implementation of a new procedure for the determination of multiple multivariate linear models with latent factors is described. The aim of the new method is to find the prediction optimal projections on latent factors. In order to compare the new method with ‘classical’ methods, a factorial experimental design is used in which important characteristics of the model are varied.

Response Surface Methodology for Optimizing Hyper Parameters

The performance of an algorithm often largely depends on some hyper parameter which should be optimized before its usage. Since most conventional optimization methods suffer from some drawbacks, we developed an alternative way to find the best hyper parameter values. Contrary to the well known procedures, the new optimization algorithm is based on statistical methods since it uses a combination of Linear Mixed Effect Models and Response Surface Methodology techniques. In particular, the Method of Steepest Ascent which is well known for the case of an Ordinary Least Squares setting and a linear response surface has been generalized to be applicable for repeated measurements situations and for response surfaces of order o ?U 2.

Prediction Optimal Data Analysis by Means of Stochastic Search

In this work a new procedure for the determination of multiple multivariate linear models with latent factors is proposed with the aim of prediction optimal projections on latent factors. In order to compare the new method with ‘classical’ methods like Ordinary Least Squares, Principal Components Regression, Partial Least Squares and Reduced-Rank Regression, a factorial experimental design is used in which important characteristics of the model are varied.

Determination of hyper-parameters for kernel based classification and regression

The optimization of the hyper-parameters of a statistical procedure or machine learning task is a crucial step for obtaining a minimal error. Unfortunately, the optimization of hyper-parameters usually requires many runs of the procedure and hence is very costly. A more detailed knowledge of the dependency of the performance of a procedure on its hyper-parameters can help to speed up this process. In this paper, we investigate the case of kernel-based classifiers and regression estimators which belong to the class of convex risk minimization methods from machine learning. In an empirical investigation, the response surfaces of nonlinear support vector machines and kernel logistic regression are analyzed and the performance of several algorithms for determining hyper-parameters is investigated. The rest of the paper is organized as follows: Section 2 briefly outlines kernel based classification and regression methods. Section 3 gives details on several methods for optimizing the hyper-parameters of statistical procedures. Then, some numerical examples are presented in Section 4. Section 5 contains a discussion. Finally, all figures are given in the appendix.

A Note on the Dimension of the Projection Space in a Latent Factor Regression Model with Application to Business Cycle Classification

In this paper it is shown that the number of latent factors in a multiple multivariate regression model need not be larger than the number of the response variables in order to achieve an optimal prediction. The practical importance of this lemma is outlined and an application of such a projection on latent factors in a classification example is given.

Testing a Simulated Annealing Algorithm in a Classification Problem

In this work we develop a new classification algorithm based on simulated annealing. The new method is evaluated and tested in a variety of situations which are generated and simulated by a Design of Experiments. This way, it is possible to find data characteristics that influence the relative classification performance of different classification methods. It turns out that the new method improves the classification performance of the classical Linear Discriminant Analysis (LDA) significantly in some situations. Moreover, in a real life example the new algorithm appears to be better than LDA.

Linear dimension reduction in classification: adaptive procedure for optimum results

Linear dimension reduction plays an important role in classification problems. A variety of techniques have been developed for linear dimension reduction to be applied prior to classification. However, there is no single definitive method that works best under all circumstances. Rather a best method depends on various data characteristics. We develop a two-step adaptive procedure in which a best dimension reduction method is first selected based on the various data characteristics, which is then applied to the data at hand. It is shown using both simulated and real life data that such a procedure can significantly reduce the misclassification rate.