Anita Prinzie

Random Forests for multiclass classification: Random MultiNomial Logit

Several supervised learning algorithms are suited to classify instances into a multiclass value space. MultiNomial Logit (MNL) is recognized as a robust classifier and is commonly applied within the CRM (Customer Relationship Management) domain. Unfortunately, to date, it is unable to handle huge feature spaces typical of CRM applications. Hence, the analyst is forced to immerse himself into feature selection. Surprisingly, in sharp contrast with binary logit, current software packages lack any feature-selection algorithm for MultiNomial Logit. Conversely, Random Forests, another algorithm learning multiclass problems, is just like MNL robust but unlike MNL it easily handles high-dimensional feature spaces. This paper investigates the potential of applying the Random Forests principles to the MNL framework. We propose the Random MultiNomial Logit (RMNL), i.e. a random forest of MNLs, and compare its predictive performance to that of (a) MNL with expert feature selection, (b) Random Forests of classification trees. We illustrate the Random MultiNomial Logit on a cross-sell CRM problem within the home-appliances industry. The results indicate a substantial increase in model accuracy of the RMNL model to that of the MNL model with expert feature selection.

Incorporating sequential information into traditional classification models by using an element/position-sensitive SAM

The inability to capture sequential patterns is a typical drawback of predictive classification methods. This caveat might be overcome by modeling sequential independent variables by sequence-analysis methods. Combining classification methods with sequence-analysis methods enables classification models to incorporate non-time varying as well as sequential independent variables. In this paper, we precede a classification model by an element/position-sensitive Sequence-Alignment Method (SAM) followed by the asymmetric, disjoint Taylor-Butina clustering algorithm with the aim to distinguish clusters with respect to the sequential dimension. We illustrate this procedure on a customer-attrition model as a decision-support system for customer retention of an International Financial-Services Provider (IFSP). The binary customer-churn classification model following the new approach significantly outperforms an attrition model which incorporates the sequential information directly into the classification method.

Investigating purchasing-sequence patterns for financial services using Markov, MTD and MTDg models

In the past, several authors have found evidence for the existence of a priority pattern of acquisition for durable goods, as well as for financial services. Its usefulness lies in the fact that if the position of a particular customer in this acquisition sequence is known, one can predict what service will be acquired next by that customer. In this paper, we analyse purchase sequences of financial services to identify cross-selling opportunities as part of a CRM (customer relationship management). Hereby, special attention is paid to transitions, which might encourage bank- or insurance only customers to become financial services customers. We introduce the Mixture Transition Distribution model (MTD) as a parsimonious alternative to the Markov model for use in the analysis of marketing problems. An interesting extension on the MTD model is the MTDg model, which is able to represent situations where the relationship between each lag and the current state differs. We illustrate the MTD and MTDg model on acquisition sequences of customers of a major financial-services company and compare the fit of these models with that of the corresponding Markov model. Our results are in favor of the MTD and MTDg models. Therefore, the MTD as well as the MTDg transition matrices are investigated in order to reveal cross-sell opportunities. The results are of great value to the product managers as they clarify the customer flows among product groups. In some cases, the lag-specific transition matrices of the MTDg model are better for the guidance of cross-sell actions than the general transition matrix of the MTD model.

Constrained optimization of data-mining problems to improve model performance: A direct-marketing application

Although most data-mining (DM) models are complex and general in nature, the implementation of such models in specific environments is often subject to practical constraints (e.g. budget constraints) or thresholds (e.g. only mail to customers with an expected profit higher than the investment cost). Typically, the DM model is calibrated neglecting those constraints/thresholds. If the implementation constraints/thresholds are known in advance, this indirect approach delivers a sub-optimal model performance. Adopting a direct approach, i.e. estimating a DM model in knowledge of the constraints/thresholds, improves model performance as the model is optimized for the given implementation environment. We illustrate the relevance of this constrained optimization of DM models on a direct-marketing case, i.e. in the field of customer relationship management. We optimize an individual-level response model for specific mailing depths (i.e. the percentage of customers of the house list that actually receives a mail given the mailing budget constraint) and compare its predictive performance with that of a traditional response model, neglecting the mailing depth during estimation. The results are in favor of the constrained-optimization approach.

Looking for the value of mission statements: a meta‐analysis of 20 years of research

– After two decades of research, the effect of a mission statement on an organizations performance is still unclear. In order to address these shortcomings, a research project via the setting‐up of this paper seeks to identify all empirical studies addressing the mission statement‐financial performance relation, analyze how the mission statement‐financial performance relation is operationalized, and aggregate the findings of the identified studies by means of a meta‐analysis., – A systematic literature review procedure was developed to identify all relevant articles and meta‐analytic procedures were used to calculate the effect size of the selected studies., – The study results indicate a small positive relation between mission statements and measures of financial organizational performance. However, additional analyses indicated that interstudy differences in measures significantly influenced the estimates (population effect sizes of the created subsamples ranged from 0.0808 to 0.4100)., – These contradictive findings stress the importance and impact of operationalization decisions in mission statement‐performance research, and provide paths for future practice‐oriented research., – This study is the first to assess the performance impact of one of the most popular management instruments, namely mission statements, by means of meta‐analytical techniques and, to evaluate the moderation effect of operationalization decisions on the cited relationship. Furthermore, by aggregating research on the mission statement‐performance relationship, a knowledge base was devised which provides normative advice on the characteristics of a “good” mission statement.

Cash Demand Forecasting in ATMs by Clustering and Neural Networks

To improve ATMs’ cash demand forecasts, this paper advocates the prediction of cash demand for groups of ATMs with similar day-of-the week cash demand patterns. We first clustered ATM centers into ATM clusters having similar day-of-the week withdrawal patterns. To retrieve “day-of-the-week” withdrawal seasonality parameters (effect of a Monday, etc.) we built a time series model for each ATMs. For clustering, the succession of seven continuous daily withdrawal seasonality parameters of ATMs is discretized. Next, the similarity between the different ATMs’ discretized daily withdrawal seasonality sequence is measured by the Sequence Alignment Method (SAM). For each cluster of ATMs, four neural networks viz., general regression neural network (GRNN), multi layer feed forward neural network (MLFF), group method of data handling (GMDH) and wavelet neural network (WNN) are built to predict an ATM center’s cash demand. The proposed methodology is applied on the NN5 competition dataset. We observed that GRNN yielded the best result of 18.44% symmetric mean absolute percentage error (SMAPE), which is better than the result of Andrawis, Atiya, and El-Shishiny (2011). This is due to clustering followed by a forecasting phase. Further, the proposed approach yielded much smaller SMAPE values than the approach of direct prediction on the entire sample without clustering. From a managerial perspective, the clusterwise cash demand forecast helps the bank’s top management to design similar cash replenishment plans for all the ATMs in the same cluster. This cluster-level replenishment plans could result in saving huge operational costs for ATMs operating in a similar geographical region.

Mining Innovative Ideas to Support New Product Research and Development

Here, we present an approach for automatically identifying the innovative potential of new technological ideas extracted from textual information. The starting point of each innovation is a good and new idea. Unfortunately, a high percentage of innovations fail, which means many ideas do not have the potential to become an innovation in future. The innovation process from a new idea as starting point via research, development, and production activities through to an innovative product is very cost- and time-consuming. Therefore, the aim of our work is to identify the innovative potential of new technological ideas to improve the performance of the innovation process. We extract new technological ideas from provided textual information. We also identify innovative technology fields by analysing relationships among technologies. All identified ideas are assigned to innovative technology fields by using text mining and text classification methods. Technological ideas in these fields are presented to the user as innovative ideas and might be used as starting point for new product research and development divisions.

Extracting Consumers Needs for New Products - A Web Mining Approach

Here we introduce a web mining approach for automatically identifying new product ideas extracted from web logs. A web log - also known as blog - is a web site that provides commentary, news, and further information on a subject written by individual persons. We can find a large amount of web logs for nearly each topic where consumers present their needs for new products. These new product ideas probably are valuable for producers as well as for researchers and developers. This is because they can lead to a new product development process. Finding these new product ideas is a well-known task in marketing. Therefore, with this automatic approach we support marketing activities by extracting new and useful product ideas from textual information in internet logs. This approach is implemented by a web-based application named Product Idea Web Log Miner where users from the marketing department provide descriptions of existing products. As a result, new product ideas are extracted from the web logs and presented to the users.

Random multiclass classification: generalizing random forests to random MNL and random NB

Random Forests (RF) is a successful classifier exhibiting performance comparable to Adaboost, but is more robust. The exploitation of two sources of randomness, random inputs (bagging) and random features, make RF accurate classifiers in several domains. We hypothesize that methods other than classification or regression trees could also benefit from injecting randomness. This paper generalizes the RF framework to other multiclass classification algorithms like the well-established MultiNomial Logit (MNL) and Naive Bayes (NB). We propose Random MNL (RMNL) as a new bagged classifier combining a forest of MNLs estimated with randomly selected features. Analogously, we introduce Random Naive Bayes (RNB). We benchmark the predictive performance of RF, RMNL and RNB against state-of-the-art SVM classifiers. RF, RMNL and RNB outperform SVM. Moreover, generalizing RF seems promising as reflected by the improved predictive performance of RMNL.

Modeling complex longitudinal consumer behavior with Dynamic Bayesian networks: an Acquisition Pattern Analysis application

Longitudinal consumer behavior has been modeled by sequence analysis. A popular application involves Acquisition Pattern Analysis exploiting typical acquisition patterns to predict a customer’s next purchase. Typically, the acquisition process is represented by an extensional, unidimensional sequence taking values from a symbolic alphabet. Given complex product structures, the extensional state representation rapidly evokes the state-space explosion problem. Consequently, most authors simplify the decision problem to the prediction of acquisitions for selected products or within product categories. This paper advocates the use of intensional state definitions representing the state by a set of variables thereby exploiting structure and allowing to model complex, possibly coupled sequential phenomena. The advantages of this intensional state space representation are demonstrated on a financial-services cross-sell application. A Dynamic Bayesian Network (DBN) models longitudinal customer behavior as represented by acquisition, product ownership and covariate variables. The DBN provides insight in the longitudinal interaction between a household’s portfolio maintenance behavior and acquisition behavior. Moreover, it exhibits adequate predictive performance to support the financial-services provider’s cross-sell strategy comparable to decision trees but superior to MulltiLayer Perceptron neural networks.