David S. Vogel

COMPUTER GAMING AND INTERACTIVE SIMULATIONS FOR LEARNING: A META-ANALYSIS

Substantial disagreement exists in the literature regarding which educational technology results in the highest cognitive gain for learners. In an attempt to resolve this dispute, we conducted a meta-analysis to decipher which teaching method, games and interactive simulations or traditional, truly dominates and under what circumstances. It was found that across people and situations, games and interactive simulations are more dominant for cognitive gain outcomes. However, consideration of specific moderator variables yielded a more complex picture. For example, males showed no preference while females showed a preference for the game and interactive simulation programs. Also, when students navigated through the programs themselves, there was a significant preference for games and interactive simulations. However, when teachers controlled the programs, no significant advantage was found. Further, when the computer dictated the sequence of the program, results favored those in the traditional teaching method over the games and interactive simulations. These findings are discussed in terms of their implications for exiting theoretical positions as well as future empirical research.

Cerebral lateralization of spatial abilities: A meta-analysis

There is a substantial disagreement in the existing literature regarding which hemisphere of the brain controls spatial abilities. In an attempt to resolve this dispute, we conducted a meta-analysis to decipher which hemisphere truly dominates and under what circumstances. It was found that across people and situations, the right hemisphere is the more dominant for spatial processing. However, consideration of specific moderator variables yielded a more complex picture. For example, females showed no hemisphere preference while males showed a right hemisphere advantage. Also, no hemisphere preference was indicated for spatial visualization tasks while subjects performing spatial orientation and manual manipulation tasks displayed a predictable right hemisphere preference. These findings are discussed in terms of their implications for exiting theoretical positions as well as future empirical research.

Classifying search engine queries using the web as background knowledge

The performance of search engines crucially depends on their ability to capture the meaning of a query most likely intended by the user. We study the problem of mapping a search engine query to those nodes of a given subject taxonomy that characterize its most likely meanings. We describe the architecture of a classification system that uses a web directory to identify the subject context that the query terms are frequently used in. Based on its performance on the classification of 800,000 example queries recorded from MSN search, the system received the Runner-Up Award for Query Categorization Performance of the KDD Cup 2005.

Predictive Modeling in Health Plans

Predictive modeling in healthcare has been gaining more interest and utilization in recent years. The tools for doing this have become more sophisticated with increasingly higher accuracy. We present a case study of how artificial intelligence (AI) can be used for a high quality predictive modeling process, and how this process is used to improve the quality and efficiency of healthcare. In this case study, MEDai, Inc. provides the analytical tools for the predictive modeling, and Sentara Healthcare uses these predictions to determine which members can be helped the most by actively looking for ways to prevent future severe outcomes. Most predictive methodologies implement rule-based systems or regression techniques. There are many pitfalls of these techniques when applied to medical data, where many variables and many interactive variable combinations exist necessitating modeling with AI. When comparing the R2 statistic (the commonly accepted measurement of how accurate a predictive model is) of traditional techniques versus AI techniques, the resulting accuracy more than doubles. The cited publications show a range of raw R2 values from 0.10 to 0.15. In contrast, the R2 value obtained from AI techniques implemented at Sentara is 0.34. Once the predictions are generated, data are displayed and analytical programs utilized for data mining and analysis. With this tool, it is possible to examine sub-groups of the data, or data mine to the member level. Risk factors can be determined and individual members/member groups can be analyzed to help make the decisions of what changes can be made to improve the level of medical care that people receive.

Scalable look-ahead linear regression trees

Most decision tree algorithms base their splitting decisions on a piecewise constant model. Often these splitting algorithms are extrapolated to trees with non-constant models at the leaf nodes. The motivation behind Look-ahead Linear Regression Trees (LLRT) is that out of all the methods proposed to date, there has been no scalable approach to exhaustively evaluate all possible models in the leaf nodes in order to obtain an optimal split. Using several optimizations, LLRT is able to generate and evaluate thousands of linear regression models per second. This allows for a near-exhaustive evaluation of all possible splits in a node, based on the quality of fit of linear regression models in the resulting branches. We decompose the calculation of the Residual Sum of Squares in such a way that a large part of it is pre-computed. The resulting method is highly scalable. We observe it to obtain high predictive accuracy for problems with strong mutual dependencies between attributes. We report on experiments with two simulated and seven real data sets.

Design and analysis of the KDD cup 2009: fast scoring on a large orange customer database

We organized the KDD cup 2009 around a marketing problem with the goal of identifying data mining techniques capable of rapidly building predictive models and scoring new entries on a large database. Customer Relationship Management (CRM) is a key element of modern marketing strategies. The KDD Cup 2009 offered to participants an opportunity to work on a large marketing database from the French Telecom company Orange. The tasks were to predict the propensity of customers to switch provider (churn), buy new products or services (appetency), or buy upgrades/addons proposed to them to make the sale more profitable (upselling). The challenge, which lasted from March 10 to May 11, 2009, attracted over 450 participants from 46 countries. We attribute its popularity to several factors: (1) A generic problem relevant to the Industry (a classification problem), but presenting a number of scientific and technical challenges, including many missing values (about 60%), a large number of features (15000) and a large number of training examples (50000), unbalanced class proportions (fewer than 10% of the examples of the positive class), noisy data, and the presence of categorical variables with many different values. (2) Prizes (Orange offers 10000 Euros in prizes). (3) A well designed protocol and web site (we benefitted from past experience). (4) An effective advertising campaign using mailings and a teleconference to answer potential participants questions. The results of the challenge were discussed at the KDD conference (June 28, 2009). The principal conclusions are that ensemble methods are very effective and that ensemble of decision trees offer off-the-shelf solutions to problems with large numbers of samples and attributes, mixed types of variables, and lots of missing values. The data and the platform of the challenge remain available for research and educational purposes at http://www.kddcup-orange.com/.

Predicting the effects of gene deletion

In this paper, we describe techniques that can be used to predict the effects of gene deletion. We will focus mainly on the creation of predictive variables, and then briefly discuss different modeling techniques that have been used successfully on this data.

Protein matching with custom neural network objective functions

This 2004 KDD Cup presents a perfect case where the usual neural network objective functions do not apply. While the contest problem consisted of 4 different entries with 4 different objective functions, this paper will focus on the solution optimizing GRMSE (Grouped Root Mean Squared Error). It will be shown that the more typical objective functions (including RMSE) cannot be as effective at meeting this criteria. While this objective function may be specific to this problem, and the reader may never see this exact function again in his/her lifetime, the idea behind this paper is applicable in many situations. Too often neural networks are used to minimize SSE (sum of the squares of the errors) or Cross Entropy, when the true measure of success for the model may require a small coding change to the Neural Network objective function. It is shown in this paper that a few small coding changes can make a big difference on a models performance.

1-dimensional splines as building blocks for improving accuracy of risk outcomes models

Transformation of both the response variable and the predictors is commonly used in fitting regression models. However, these transformation methods do not always provide the maximum linear correlation between the response variable and the predictors, especially when there are non-linear relationships between predictors and the response such as the medical data set used in this study. A spline based transformation method is proposed that is second order smooth, continuous, and minimizes the mean squared error between the response and each predictor. Since the computation time for generating this spline is O(n), the processing time is reasonable with massive data sets. In contrast to cubic smoothing splines, the resulting transformation equations also display a high level of efficiency for scoring. Data used for predicting health outcomes contains an abundance of non-linear relationships between predictors and the outcomes requiring an algorithm for modeling them accurately. Thus, a transformation that fits an adaptive cubic spline to each of a set of variables is proposed. These curves are used as a set of transformation functions on the predictors. A case study of how the transformed variables can be fed into a simple linear regression model to predict risk outcomes is presented. The results show significant improvement over the performance of the original variables in both linear and non-linear models.