Barbara Marszał-Paszek

Comparison of some classification algorithms based on deterministic and nondeterministic decision rules

We discuss two, in a sense extreme, kinds of nondeterministic rules in decision tables. The first kind of rules, called as inhibitory rules, are blocking only one decision value (i.e., they have all but one decisions from all possible decisions on their right hand sides). Contrary to this, any rule of the second kind, called as a bounded nondeterministic rule, can have on the right hand side only a few decisions. We show that both kinds of rules can be used for improving the quality of classification. In the paper, two lazy classification algorithms of polynomial time complexity are considered. These algorithms are based on deterministic and inhibitory decision rules, but the direct generation of rules is not required. Instead of this, for any new object the considered algorithms extract from a given decision table efficiently some information about the set of rules. Next, this information is used by a decision-making procedure. The reported results of experiments show that the algorithms based on inhibitory decision rules are often better than those based on deterministic decision rules. We also present an application of bounded nondeterministic rules in construction of rule based classifiers. We include the results of experiments showing that by combining rule based classifiers based on minimal decision rules with bounded nondeterministic rules having confidence close to 1 and sufficiently large support, it is possible to improve the classification quality.

Minimal Templates and Knowledge Discovery

In this paper the dependences between the Dempster-Shafer theory and rough set theory have been used to find a minimal template in a given decision table. The Dempster-Shafer theory [5] is called a mathematical theory of evidence. This theory is based on belief functions and plausible reasoning is used to combine separate pieces of information (evidence) to calculate the probability of an event. Rough set theory was proposed by Pawlak in 1982 [3] as a mathematical tool for describing the uncertain knowledge. In 1987 [1] and 1991 [6] the basic functions of the evidence theory were defined, based on the notation from rough set theory. These definitions allow finding interesting dependences in decision tables.

Evidence Theory and VPRS model

Abstract The Rough Set Theory (RST) was proposed by Pawlak [4] as a new mathematical approach to deal with uncertain knowledge in expert systems. In 1991 Ziarko [11] proposed the Variable Precision Rough Set Model (VPRSM) as a certain extension of the rough set theory. VPRSM approach makes it possible to use a certain level of misclassification. The aim of this paper is to introduce belief and plausibility functions defined by the β-approximation regions. On the basis of the β-approximation regions, the β-basic probability assignment is defined and the Dempsters combination rule for product of two decision tables is constructed. This entire approach is illustrated by examples.

Nondeterministic Decision Rules in Classification Process

In the paper, we discuss nondeterministic rules in decision tables, called the truncated nondeterministic rules. These rules have on the right hand side a few decisions. We show that the truncated nondeterministic rules can be used for improving the quality of classification.

Classification Algorithms Based on Template’s Decision Rules

In the paper, classification algorithms are presented. These algorithms are based on nondeterministic decision rules that are called template’s decision rules. The conditional part of these rules is a template and the decision part is satisfactorily small set of decisions. Only rules with suficiently large support are used. The proposed classification algorithms were tested on the group of decision tables from the UCI Machine Learning Repository. Results of experiments show that the classification algorithms based on template’s decision rules are often better than the algorithms based on deterministic decision rules.

Classifiers Based on Nondeterministic Decision Rules

In the chapter, we discuss classifiers based on rough set theory and nondeterministic decision rules. We used two kinds of nondeterministic rules called the first and second type. These rules have a few decision values but the rules of the second type can have on the left-hand side one generalized descriptor. i.e., a condition of the form a ∈ V, where V is a two-element subset of the attribute value set V a . We show that these kinds of rules can be used for improving the quality of classification and we propose classifications algorithms based on nondeterministic (first and second type) rules. These algorithms are using not only nondeterministic rules but also minimal rules in the sense of rough sets. In the chapter, these classifiers were tested on several data sets from the UCI Machine Learning Repository and the results were compared. The reported results of experiments show that the proposed classifiers based on nondeterministic rules can improve the classification quality but it requires tuning some of their parameters relative to analyzed data.

Minimal Templates Problem

In a 1976 Dempster and Shafer have created a mathematical theory of evidence called Dempster-Shafer theory. This theory is based on belief functions and plausible reasoning, which is used to combine separate pieces of information (evidence) to calculate the probability of an event. In 1982 Pawlak has created the rough set theory as an innovative mathematical tool to describing the knowledge, including also the uncertain and inexact knowledge. In 1994 the basic functions of the evidence theory have been defined, based on the notion from the rough set theory. This dependence between these theories has allowed further research on their practical usage. In this paper the above-mentioned dependences have been used to find minimal template in a given decision table. The problem of finding such templates is NP- hard. Therefore, some heuristics based on genetic algorithms have been proposed.

Extracting Minimal Templates in a Decision Table

In 1991 there were defined basic functions of the evidence theory based on the concepts of the rough set theory. In this paper we use these functions in specifying minimal templates of decision tables. The problem of finding such templates is NP-hard. Hence, we propose some heuristics based on genetic algorithms.

Nondeterministic Decision Rules in Rule-Based Classifier

In the paper is discussed the truncated nondeterministic rules and their role in an evaluation of classification model. The nondeterministic rules are created as the result of shorting deterministic rules in accordance with the principle of minimum description length (MDL). As deterministic rules in database we treat the full objects description in a meaning of descriptors conjunction. The nondeterministic rules are calculated in polynomial time by using greedy strategy.