Arie Ben-David

Monotonicity Maintenance in Information-Theoretic Machine Learning Algorithms

Decision trees that are based on information-theory are useful paradigms for learning from examples. However, in some real-world applications, known information-theoretic methods frequently generate nonmonotonic decision trees, in which objects with better attribute values are sometimes classified to lower classes than objects with inferior values. This property is undesirable for problem solving in many application domains, such as credit scoring and insurance premium determination, where monotonicity of subsequent classifications is important. An attribute-selection metric is proposed here that takes both the error as well as monotonicity into account while building decision trees. The metric is empirically shown capable of significantly reducing the degree of non-monotonicity of decision trees without sacrificing their inductive accuracy.

Comparison of classification accuracy using Cohen's Weighted Kappa

Many expert systems solve classification problems. While comparing the accuracy of such classifiers, the cost of error must frequently be taken into account. In such cost-sensitive applications just using the percentage of misses as the sole meter for accuracy can be misleading. Typical examples of such problems are medical and military applications, as well as data sets with ordinal (i.e., ordered) class. A new methodology is proposed here for assessing classifiers accuracy. The approach taken is based on Cohens Kappa statistic. It compensates for classifications that may be due to chance. The use of Kappa is proposed as a standard meter for measuring the accuracy of all multi-valued classification problems. The use of Weighted Kappa enables to effectively deal with cost-sensitive classification. When the cost of error is unknown and can only be roughly estimated, the use of sensitivity analysis with Weighted Kappa is highly recommended.

Control of properties in injection molding by neural networks

Abstract Adequate control of product properties in injection molded plastics requires very accurate predictions. The problem is that the mechanical properties of these plastics, such as tensile modulus, are highly non-linear with the process variables, hence they are tough to predict. Consequently, up to date, injection molding machines include only closed loop control of process variables. Control of product properties is virtually non-existent. We show here for the first time, that mechanical properties, such as tensile modulus values, can be predicted using Artificial Neural Networks quite accurately within a reasonable time. This is a major step towards an integrated self-taught control mechanism for the injection molded plastics industry.

Critical managerial factors affecting defense projects success: A comparison between neural network and regression analysis

Abstract A comparison between neural networks and linear regression analysis is used for identifying critical managerial factors affecting the success of high-tech defense projects. The study shows that neural networks have better explanatory and prediction power, and it enables the exploration of relationships among the data that are difficult to arrive at by traditional statistical methods. The study yielded some new results: The chances to success of a project that was acknowledged by its prospected customers as essential for improving their performance are much higher than other projects. Furthermore, organizational learning and social cohesion of the development team are of extreme importance for success.

Adding monotonicity to learning algorithms may impair their accuracy

Ordinal (i.e., ordered) classifiers are used to make judgments that we make on a regular basis, both at work and at home. Perhaps surprisingly, there have been no comprehensive studies in the scientific literature comparing the various ordinal classifiers. This paper compares the accuracy of five ordinal and three non-ordinal classifiers on a benchmark of fifteen real-world datasets. The results show that the ordinal classifiers that were tested had no meaningful statistical advantage over the corresponding non-ordinal classifiers. Furthermore, the ordinal classifiers that guaranteed monotonic classifications showed no meaningful statistical advantage over a majority-based classifier. We suggest that the tested ordinal classifiers did not properly utilize the order information in the presence of non-monotonic noise.

Generating rules from examples of human multiattribute decision making should be simple

Abstract How many prototypes or clusters are needed to predict real world human multiattribute subjective decision making? Although subjective decision making problems occur daily in our life, they have received relatively little attention in artificial intelligence, machine learning and data mining communities. We claim that for most problems, a simple set of rules derived by a nearest neighbor algorithm is the appropriate approach. A simple version of a nearest neighbor model is tested and compared with two other well-established classification methods: neural networks and classifications and regression trees (CART). The results of the experiments show that the simple nearest neighbor method provides very accurate predictions while using very few prototypes or clusters. Although not always the best in accuracy, the differences are sufficiently slight to not warrant greater complexity in deriving rules. Our research on the effectiveness of parsimonious rule sets suggests that decision trees with more than 7–10 branches are not needed for capturing most human multiattribute decision-making problems, and minimal time or memory resources should be used to generate decision making rules.

Two algorithms for generating structured and unstructured monotone ordinal data sets

Monotone constraints are very common while dealing with multi-attribute ordinal problems. Grinding wheels hardness selection, timely replacements of costly laser sensors in silicon wafer manufacturing, and the selection of the right personnel for sensitive production facilities, are just a few examples of ordinal problems where monotonicity makes sense. In order to evaluate the performance of various ordinal classifiers one needs both artificially generated as well as real world data sets. Two algorithms are presented for generating monotone ordinal data sets. The first can be used for generating random monotone ordinal data sets without an underlying structure. The second algorithm, which is the main contribution of this paper, describes for the first time how structured monotone data sets can be generated.

Self-improving expert systems: an architecture and implementation

Abstract Self-improving expert systems that are based upon learning-by-example have drawn much attention in recent years. A methodology is presented which assists in the use of a learning-by-example paradigm for expert systems applications. The architecture is based upon a hybrid of neural networks and rule-based models. Practitioners may use a similar approach to construct self- improving expert systems faster and more efficiently than has been possible with pure rule-based systems. The ideas are illustrated through an actual expert system that assists experts during the planning stage of a chemical product that has given properties and composition. A description of the application and a discussion of some interesting implementation issues are presented.

Rule effectiveness in rule-based systems: A credit scoring case study

Rule-based systems may sometimes grow very large, making their acceptance by users and their maintenance quite problematic. One therefore needs to make rule-bases as compact as possible. The classical definition of rule redundancy in the literature is based upon logic and graph theory. Another, complementary, view of redundancy is proposed here. The suggested approach is based on the contribution of individual rules to the overall systems accuracy. It is shown here, though an analysis of a real-world credit scoring rule-based system, that by taking into account systems accuracy, one can sometimes significantly reduce the size of a rule-base; even one which is already free from logic-related abnormalities. The approach taken here is not proposed as a substitution to classical logic and graph-based methods. Rather, it complements them.

Classification Accuracy: Machine Learning vs. Explicit Knowledge Acquisition

This empirical study provides evidence that machine learning models can provide better classification accuracy than explicit knowledge acquisition techniques. The findings suggest that the main contribution of machine learning to expert systems is not just cost reduction, but rather the provision of tools for the development of better expert systems.This empirical study provides evidence that machine learning models can provide better classification accuracy than explicit knowledge acquisition techniques. The findings suggest that the main contribution of machine learning to expert systems is not just cost reduction, but rather the provision of tools for the development of better expert systems.