Wai Lam

A multilevel approach to intelligent information filtering: model, system, and evaluation

In information-filtering environments, uncertainties associated with changing interests of the user and the dynamic document stream must be handled efficiently. In this article, a filtering model is proposed that decomposes the overall task into subsystem functionalities and highlights the need for multiple adaptation techniques to cope with uncertainties. A filtering system, SIFTER, has been implemented based on the model, using established techniques in information retrieval and artificial intelligence. These techniques include document representation by a vector-space model, document classification by unsupervised learning, and user modeling by reinforcement learning. The system can filter information based on content and a users specific interests. The users interests are automatically learned with only limited user intervention in the form of optional relevance feedback for documents. We also describe experimental studies conducted with SIFTER to filter computer and information science documents collected from the Internet and commercial database services. The experimental results demonstrate that the system performs very well in filtering documents in a realistic problem setting.

Fuzzy concepts in expert systems

The authors present a comprehensive expert-system building tool, called System Z-II, that can deal with exact, fuzzy (or inexact), and combined reasoning, allowing fuzzy and normal terms to be freely mixed in the rules and facts of an expert system. This fully implemented tool has been used to build several expert systems in the fields of student curriculum advisement, medical diagnosis, psychoanalysis, and risk analysis. System Z-II is a rule-based system that uses fuzzy logic and fuzzy numbers for its inexact reasoning. It uses two basic inexact concepts, fuzziness and uncertainty, which are distinct from each other in the system.<<ETX>>

Automatic text categorization and its application to text retrieval

We develop an automatic text categorization approach and investigate its application to text retrieval. The categorization approach is derived from a combination of a learning paradigm known as instance-based learning and an advanced document retrieval technique known as retrieval feedback. We demonstrate the effectiveness of our categorization approach using two real-world document collections from the MEDLINE database. Next, we investigate the application of automatic categorization to text retrieval. Our experiments clearly indicate that automatic categorization improves the retrieval performance compared with no categorization. We also demonstrate that the retrieval performance using automatic categorization achieves the same retrieval quality as the performance using manual categorization. Furthermore, detailed analysis of the retrieval performance on each individual test query is provided.

Using evolutionary programming and minimum description length principle for data mining of Bayesian networks

We have developed a new approach to learning Bayesian network structures based on the minimum description length (MDL) principle and evolutionary programming. It employs a MDL metric, which is founded on information theory, and integrates a knowledge-guided genetic operator for the optimization in the search process.

Automatic document classification based on probabilistic reasoning: model and performance analysis

We develop a new approach to test classification based on automatic feature extraction and probabilistic reasoning. The knowledge representation used to perform such task is known as Bayesian inference networks. A Bayesian network text classifier is automatically constructed from a set of training test documents. We have conducted a series of experiments on two text document corpus, namely the CACM and Reuters, to analyze the performance of our approach, which are described in the paper.

Extracting discriminative concepts for domain adaptation in text mining

One common predictive modeling challenge occurs in text mining problems is that the training data and the operational (testing) data are drawn from different underlying distributions. This poses a great difficulty for many statistical learning methods. However, when the distribution in the source domain and the target domain are not identical but related, there may exist a shared concept space to preserve the relation. Consequently a good feature representation can encode this concept space and minimize the distribution gap. To formalize this intuition, we propose a domain adaptation method that parameterizes this concept space by linear transformation under which we explicitly minimize the distribution difference between the source domain with sufficient labeled data and target domains with only unlabeled data, while at the same time minimizing the empirical loss on the labeled data in the source domain. Another characteristic of our method is its capability for considering multiple classes and their interactions simultaneously. We have conducted extensive experiments on two common text mining problems, namely, information extraction and document classification to demonstrate the effectiveness of our proposed method.

Bayesian network refinement via machine learning approach

An approach to refining Bayesian network structures from new data is developed. Most previous work has only considered the refinement of the networks conditional probability parameters and has not addressed the issue of refining the networks structure. We tackle this problem by a machine learning approach based on a formalism known as the minimum description length (MDL) principle. The MDL principle is well suited to this task since it can perform tradeoffs between the accuracy, simplicity, and closeness to the existent structure. Another salient feature of this refinement approach is the capability of refining a network structure using partially specified data. Moreover, a localization scheme is developed for efficient computation of the description lengths since direct evaluation involves exponential time resources.

Discovering knowledge from medical databases using evolutionory algorithms

Discusses learning roles and causal structures for capturing patterns and causality relationships. The authors present their approach for knowledge discovery from two specific medical databases. First, rules are learned to represent the interesting patterns of the data. Second, Bayesian networks are induced to act as causality relationship models among the attributes. The Bayesian network learning process is divided into two phases. In the first phase, a discretization policy is learned to discretize the continuous variables, and then Bayesian network structures are induced in the second phase. The authors employ advanced evolutionary algorithms such as generic genetic programming, evolutionary programming, and genetic algorithms to conduct the learning tasks. From the fracture database, they discovered knowledge about the patterns of child fractures. From the scoliosis database, they discovered knowledge about the classification of scoliosis. They also found unexpected rules that led to discovery of errors in the database. These results demonstrate that the knowledge discovery process can find interesting knowledge about the data, which can provide novel clinical knowledge as well as suggest refinements of the existing knowledge.

Learning to Adapt Web Information Extraction Knowledge and Discovering New Attributes via a Bayesian Approach

This paper presents a Bayesian learning framework for adapting information extraction wrappers with new attribute discovery, reducing human effort in extracting precise information from unseen Web sites. Our approach aims at automatically adapting the information extraction knowledge previously learned from a source Web site to a new unseen site, at the same time, discovering previously unseen attributes. Two kinds of text-related clues from the source Web site are considered. The first kind of clue is obtained from the extraction pattern contained in the previously learned wrapper. The second kind of clue is derived from the previously extracted or collected items. A generative model for the generation of the site-independent content information and the site-dependent layout format of the text fragments related to attribute values contained in a Web page is designed to harness the uncertainty involved. Bayesian learning and expectation-maximization (EM) techniques are developed under the proposed generative model for identifying new training data for learning the new wrapper for new unseen sites. Previously unseen attributes together with their semantic labels can also be discovered via another EM-based Bayesian learning based on the generative model. We have conducted extensive experiments from more than 30 real-world Web sites in three different domains to demonstrate the effectiveness of our framework.

Learning nonlinear multiregression networks based on evolutionary computation

This paper describes a novel knowledge discovery and data mining framework dealing with nonlinear interactions among domain attributes. Our network-based model provides an effective and efficient reasoning procedure to perform prediction and decision making. Unlike many existing paradigms based on linear models, the attribute relationship in our framework is represented by nonlinear nonnegative multiregressions based on the Choquet integral. This kind of multiregression is able to model a rich set of nonlinear interactions directly. Our framework involves two layers. The outer layer is a network structure consisting of network elements as its components, while the inner layer is concerned with a particular network element modeled by Choquet integrals. We develop a fast double optimization algorithm (FDOA) for learning the multiregression coefficients of a single network element. Using this local learning component and multiregression-residual-cost evolutionary programming (MRCEP), we propose a global learning algorithm, called MRCEP-FDOA, for discovering the network structures and their elements from databases. We have conducted a series of experiments to assess the effectiveness of our algorithm and investigate the performance under different parameter combinations, as well as sizes of the training data sets. The empirical results demonstrate that our framework can successfully discover the target network structure and the regression coefficients.