Jiye Liang

Positive approximation: An accelerator for attribute reduction in rough set theory

Feature selection is a challenging problem in areas such as pattern recognition, machine learning and data mining. Considering a consistency measure introduced in rough set theory, the problem of feature selection, also called attribute reduction, aims to retain the discriminatory power of original features. Many heuristic attribute reduction algorithms have been proposed however, quite often, these methods are computationally time-consuming. To overcome this shortcoming, we introduce a theoretic framework based on rough set theory, called positive approximation, which can be used to accelerate a heuristic process of attribute reduction. Based on the proposed accelerator, a general attribute reduction algorithm is designed. Through the use of the accelerator, several representative heuristic attribute reduction algorithms in rough set theory have been enhanced. Note that each of the modified algorithms can choose the same attribute reduct as its original version, and hence possesses the same classification accuracy. Experiments show that these modified algorithms outperform their original counterparts. It is worth noting that the performance of the modified algorithms becomes more visible when dealing with larger data sets.

THE INFORMATION ENTROPY, ROUGH ENTROPY AND KNOWLEDGE GRANULATION IN ROUGH SET THEORY

Rough set theory is a relatively new mathematical tool for use in computer applications in circumstances which are characterized by vagueness and uncertainty. In this paper, we introduce the concepts of information entropy, rough entropy and knowledge granulation in rough set theory, and establish the relationships among those concepts. These results will be very helpful for understanding the essence of concept approximation and establishing granular computing in rough set theory.

A NEW METHOD FOR MEASURING UNCERTAINTY AND FUZZINESS IN ROUGH SET THEORY

Based on the complement behavior of information gain, a new definition of information entropy is proposed along with its justification in rough set theory. Some properties of this definition imply those of Shannons entropy. Based on the new information entropy, conditional entropy and mutual information are then introduced and applied to knowledge bases. The new information entropy is proved to also be a fuzzy entropy.

Incomplete Multigranulation Rough Set

The original rough-set model is primarily concerned with the approximations of sets described by a single equivalence relation on a given universe. With granular computing point of view, the classical rough-set theory is based on a single granulation. This correspondence paper first extends the rough-set model based on a tolerance relation to an incomplete rough-set model based on multigranulations, where set approximations are defined through using multiple tolerance relations on the universe. Then, several elementary measures are proposed for this rough-set framework, and a concept of approximation reduct is introduced to characterize the smallest attribute subset that preserves the lower approximation and upper approximation of all decision classes in this rough-set model. Finally, several key algorithms are designed for finding an approximation reduct.

Multigranulation decision-theoretic rough sets

The Bayesian decision-theoretic rough sets propose a framework for studying rough set approximations using probabilistic theory, which can interprete the parameters from existing forms of probabilistic approaches to rough sets. Exploring rough sets in the viewpoint of multigranulation is becoming one of desirable directions in rough set theory, in which lower/upper approximations are approximated by granular structures induced by multiple binary relations. Through combining these two ideas, the objective of this study is to develop a new multigranulation rough set model, called a multigranulation decision-theoretic rough set. Many existing multigranulation rough set models can be derived from the multigranulation decision-theoretic rough set framework.

Information entropy, rough entropy and knowledge granulation in incomplete information systems

Rough set theory is a relatively new mathematical tool for use in computer applications in circumstances that are characterized by vagueness and uncertainty. Rough set theory uses a table called an information system, and knowledge is defined as classifications of an information system. In this paper, we introduce the concepts of information entropy, rough entropy, knowledge granulation and granularity measure in incomplete information systems, their important properties are given, and the relationships among these concepts are established. The relationship between the information entropy E(A) and the knowledge granulation GK(A) of knowledge A can be expressed as E(A)+GK(A) = 1, the relationship between the granularity measure G(A) and the rough entropy E r(A) of knowledge A can be expressed as G(A)+E r(A) = log2|U|. The conclusions in Liang and Shi (2004) are special instances in this paper. Furthermore, two inequalities − log2 GK(A) ≤ G(A) and E r(A) ≤ log2(|U|(1 − E(A))) about the measures GK, G, E and E r are obtained. These results will be very helpful for understanding the essence of uncertainty measurement, the significance of an attribute, constructing the heuristic function in a heuristic reduct algorithm and measuring the quality of a decision rule in incomplete information systems.

The algorithm on knowledge reduction in incomplete information systems

Rough set theory is emerging as a powerful tool for reasoning about data, knowledge reduction is one of the important topics in the research on rough set theory. It has been proven that finding the minimal reduct of an information system is a NP-hard problem, so is finding the minimal reduct of an incomplete information system. Main reason of causing NP-hard is combination problem of attributes. In this paper, knowledge reduction is defined from the view of information, a heuristic algorithm based on rough entropy for knowledge reduction is proposed in incomplete information systems, the time complexity of this algorithm is O(|A|2|U|). An illustrative example is provided that shows the application potential of the algorithm.

A Group Incremental Approach to Feature Selection Applying Rough Set Technique

Many real data increase dynamically in size. This phenomenon occurs in several fields including economics, population studies, and medical research. As an effective and efficient mechanism to deal with such data, incremental technique has been proposed in the literature and attracted much attention, which stimulates the result in this paper. When a group of objects are added to a decision table, we first introduce incremental mechanisms for three representative information entropies and then develop a group incremental rough feature selection algorithm based on information entropy. When multiple objects are added to a decision table, the algorithm aims to find the new feature subset in a much shorter time. Experiments have been carried out on eight UCI data sets and the experimental results show that the algorithm is effective and efficient.

An efficient accelerator for attribute reduction from incomplete data in rough set framework

Feature selection (attribute reduction) from large-scale incomplete data is a challenging problem in areas such as pattern recognition, machine learning and data mining. In rough set theory, feature selection from incomplete data aims to retain the discriminatory power of original features. To address this issue, many feature selection algorithms have been proposed, however, these algorithms are often computationally time-consuming. To overcome this shortcoming, we introduce in this paper a theoretic framework based on rough set theory, which is called positive approximation and can be used to accelerate a heuristic process for feature selection from incomplete data. As an application of the proposed accelerator, a general feature selection algorithm is designed. By integrating the accelerator into a heuristic algorithm, we obtain several modified representative heuristic feature selection algorithms in rough set theory. Experiments show that these modified algorithms outperform their original counterparts. It is worth noting that the performance of the modified algorithms becomes more visible when dealing with larger data sets.

Knowledge structure, knowledge granulation and knowledge distance in a knowledge base

One of the strengths of rough set theory is the fact that an unknown target concept can be approximately characterized by existing knowledge structures in a knowledge base. Knowledge structures in knowledge bases have two categories: complete and incomplete. In this paper, through uniformly expressing these two kinds of knowledge structures, we first address four operators on a knowledge base, which are adequate for generating new knowledge structures through using known knowledge structures. Then, an axiom definition of knowledge granulation in knowledge bases is presented, under which some existing knowledge granulations become its special forms. Finally, we introduce the concept of a knowledge distance for calculating the difference between two knowledge structures in the same knowledge base. Noting that the knowledge distance satisfies the three properties of a distance space on all knowledge structures induced by a given universe. These results will be very helpful for knowledge discovery from knowledge bases and significant for establishing a framework of granular computing in knowledge bases.