Beata Zielosko

Dynamic programming approach to optimization of approximate decision rules

This paper is devoted to the study of an extension of dynamic programming approach which allows sequential optimization of approximate decision rules relative to the length and coverage. We introduce an uncertainty measure R(T) which is the number of unordered pairs of rows with different decisions in the decision table T. For a nonnegative real number @b, we consider @b-decision rules that localize rows in subtables of T with uncertainty at most @b. Our algorithm constructs a directed acyclic graph @D@b(T) which nodes are subtables of the decision table T given by systems of equations of the kind attribute=value. This algorithm finishes the partitioning of a subtable when its uncertainty is at most @b. The graph @D@b(T) allows us to describe the whole set of so-called irredundant @b-decision rules. We can describe all irredundant @b-decision rules with minimum length, and after that among these rules describe all rules with maximum coverage. We can also change the order of optimization. The consideration of irredundant rules only does not change the results of optimization. This paper contains also results of experiments with decision tables from UCI Machine Learning Repository.

Combinatorial Machine Learning

Decision trees and decision rule systems are widely used in different applicationsas algorithms for problem solving, as predictors, and as a way forknowledge representation. Reducts play key role in the problem of attribute(feature) selection. The aims of this book are (i) the consideration of the setsof decision trees, rules and reducts; (ii) study of relationships among theseobjects; (iii) design of algorithms for construction of trees, rules and reducts;and (iv) obtaining bounds on their complexity. Applications for supervisedmachine learning, discrete optimization, analysis of acyclic programs, faultdiagnosis, and pattern recognition are considered also. This is a mixture ofresearch monograph and lecture notes. It contains many unpublished results.However, proofs are carefully selected to be understandable for students.The results considered in this book can be useful for researchers in machinelearning, data mining and knowledge discovery, especially for those who areworking in rough set theory, test theory and logical analysis of data. The bookcan be used in the creation of courses for graduate students.

Dynamic Programming Approach for Exact Decision Rule Optimization

This chapter is devoted to the study of an extension of dynamic programming approach that allows sequential optimization of exact decision rules relative to the length and coverage. It contains also results of experiments with decision tables from UCI Machine Learning Repository.

Three approaches to data analysis

In this book, the following three approaches to data analysis are presented: - Test Theory, founded by Sergei V. Yablonskii (1924-1998); the first publications appeared in 1955 and 1958, - Rough Sets, founded by Zdzisaw I. Pawlak (1926-2006); the first publications appeared in 1981 and 1982,- Logical Analysis of Data, founded by Peter L. Hammer (1936-2006); the first publications appeared in 1986 and 1988. These three approaches have much in common, but researchers active in one of these areas often have a limited knowledge about the results and methods developed in the other two. On the other hand, each of the approaches shows some originality and we believe that the exchange of knowledge can stimulate further development of each of them. This can lead to new theoretical results and real-life applications and, in particular, new results based on combination of these three data analysis approaches can be expected. - Logical Analysis of Data, founded by Peter L. Hammer (1936-2006); the first publications appeared in 1986 and 1988. These three approaches have much in common, but researchers active in one of these areas often have a limited knowledge about the results and methods developed in the other two. On the other hand, each of the approaches shows some originality and we believe that the exchange of knowledge can stimulate further development of each of them. This can lead to new theoretical results and real-life applications and, in particular, new results based on combination of these three data analysis approaches can be expected. These three approaches have much in common, but researchers active in one of these areas often have a limited knowledge about the results and methods developed in the other two. On the other hand, each of the approaches shows some originality and we believe that the exchange of knowledge can stimulate further development of each of them. This can lead to new theoretical results and real-life applications and, in particular, new results based on combination of these three data analysis approaches can be expected.

Optimization and analysis of decision trees and rules: dynamic programming approach

Abstract This paper is devoted to the consideration of software system Dagger created in KAUST. This system is based on extensions of dynamic programming. It allows sequential optimization of decision trees and rules relative to different cost functions, derivation of relationships between two cost functions (in particular, between number of misclassifications and depth of decision trees), and between cost and uncertainty of decision trees. We describe features of Dagger and consider examples of this system’s work on decision tables from UCI Machine Learning Repository. We also use Dagger to compare 16 different greedy algorithms for decision tree construction.

Optimization of Decision Rules Based on Dynamic Programming Approach

This chapter is devoted to the study of an extension of dynamic programming approach which allows optimization of approximate decision rules relative to the length and coverage. We introduce an uncertainty measure that is the difference between number of rows in a given decision table and the number of rows labeled with the most common decision for this table divided by the number of rows in the decision table. We fix a threshold γ, such that 0 ≤ γ < 1, and study so-called γ-decision rules (approximate decision rules) that localize rows in subtables which uncertainty is at most γ. Presented algorithm constructs a directed acyclic graph Δγ T which nodes are subtables of the decision table T given by pairs “attribute = value”. The algorithm finishes the partitioning of a subtable when its uncertainty is at most γ. The chapter contains also results of experiments with decision tables from UCI Machine Learning Repository.

Relationships Between Length and Coverage of Decision Rules

The paper describes a new tool for study relationships between length and coverage of exact decision rules. This tool is based on dynamic programming approach. We also present results of experiments with decision tables from UCI Machine Learning Repository.

Logical Analysis of Data: Theory, Methodology and Applications

Logical analysis of data (LAD) is a data analysis methodology which combines ideas and concepts from optimization, combinatorics and Boolean functions. The idea of LAD was first described by Peter L. Hammer in a lecture given in 1986 at the International Conference on Multi-attribute Decision Making via OR-based Expert Systems [41] and was later expanded and developed in [32]. That first publication was followed by a stream of research studies many of which can be found in the list of references. In early publications the focus of research was on theoretical developments and on computational implementation. In recent years attention was concentrated on practical applications varying from medicine to credit risk ratings. The purpose of the present chapter is to provide an overview of the theoretical foundations of this methodology, to discuss various aspects of its implementation and to survey some of its numerous applications. We start with an introductory example proposed in [32].

Optimization of inhibitory decision rules relative to length and coverage

The paper is devoted to the study of algorithms for optimization of inhibitory rules relative to the length and coverage. In contrast with usual rules that have on the right-hand side a relation attribute = value, inhibitory rules have a relation attribute ≠ value on the right-hand side. The considered algorithms are based on extensions of dynamic programming.

Construction of α-decision trees for tables with many-valued decisions

In the paper, we study a greedy algorithm for construction of approximate decision trees. This algorithm is applicable to decision tables with many-valued decisions where each row is labeled with a set of decisions. For a given row, we should find a decision from the set attached to this row. We use an uncertainty measure which is the number of boundary subtables. We present also experimental results for data sets from UCI Machine Learning Repository for proposed approach and approach based on generalized decision.