Qigang Gao

Curve detection based on perceptual organization

Abstract A curve detection method is described based on the perceptual organization of descriptive curve features. A set of curve partitioning and grouping rules is derived for detecting image curves. With these rules, this method is capable of tracking curve segments and joining them into an appropriate form of curve structure according to its topological and geometric properties. Experimental results demonstrating the effectiveness of this technique are included in the presentation.

Using controlled query generation to evaluate blind relevance feedback algorithms

Currently in document retrieval there are many algorithms each with different strengths and weakness. There is some difficulty, however, in evaluating the impact of the test query set on retrieval results. The traditional evaluation process, the Cranfield evaluation paradigm, which uses a corpus and a set of user queries, focuses on making the queries as realistic as possible. Unfortunately such query sets lack the fine grained control necessary to test algorithm properties. We present an approach called controlled query generation (CQG) that creates query sets from documents in the corpus in a way that regulates the theoretic information quality of each query. This allows us to generate reproducible and well defined sets of queries of varying length and term specificity. Imposing this level of control over the query sets used for testing retrieval algorithms enables the rigorous simulation of different query environments to identify specific algorithm properties before introducing user queries. In this work, we demonstrate the usefulness of CQG by generating three different query environments to investigate characteristics of two blind relevance feedback approaches

Detecting anomalies from big network traffic data using an adaptive detection approach

The unprecedented explosion of real-life big data sets have sparked a lot of research interests in data mining in recent years. Many of these big data sets are generated in network environment and are characterized by a dauntingly large size and high dimensionality which pose great challenges for detecting useful knowledge and patterns, such as network traffic anomalies, from them. In this paper, we study the problem of anomaly detection in big network connection data sets and propose an outlier detection technique, called Adaptive Stream Projected Outlier deTector (A-SPOT), to detect anomalies from large data sets using a novel adaptive subspace analysis approach. A case study of A-SPOT is conducted in this paper by deploying it to the 1999 KDD CUP anomaly detection application. Innovative approaches for training data generation, anomaly classification and false positive reduction are proposed in this paper as well to better tailor A-SPOT to deal with the case study. Experimental results demonstrate that A-SPOT is effective and efficient in detecting anomalies from network data sets and outperforms existing detection methods.

Perceptual motion tracking from image sequences

This paper presents a method based on perceptual organization for tracking object motion within image sequences (or video). To achieve the characteristics of real-time, efficiency and robustness, a perceptual edge tracker was developed to extract generic edge tokens (GETS) on the fly. The GETS are defined qualitatively according to their descriptive properties. Object motion is tracked within the image sequence by computing the difference between GET streams in consecutive frames. Multiple moving objects are grouped by forming clusters of motion GET. Experimental results are provided.

Integrating web content clustering into web log association rule mining

One of the effects of the general Internet growth is an immense number of user accesses to WWW resources These accesses are recorded in the web server log files, which are a rich data resource for finding useful patterns and rules of user browsing behavior, and they caused the rise of technologies for Web usage mining Current Web usage mining applications rely exclusively on the web server log files The main hypothesis discussed in this paper is that Web content analysis can be used to improve Web usage mining results We propose a system that integrates Web page clustering into log file association mining and uses the cluster labels as Web page content indicators It is demonstrated that novel and interesting association rules can be mined from the combined data source The rules can be used further in various applications, including Web user profiling and Web site construction We experiment with several approaches to content clustering, relying on keyword and character n-gram based clustering with different distance measures and parameter settings Evaluation shows that character n-gram based clustering performs better than word-based clustering in terms of an internal quality measure (about 3 times better) On the other hand, word-based cluster profiles are easier to manually summarize Furthermore, it is demonstrated that high-quality rules are extracted from the combined dataset.

A Novel Method for Detecting Outlying Subspaces in High-dimensional Databases Using Genetic Algorithm

Detecting outlying subspaces is a relatively new research problem in outlier-ness analysis for high-dimensional data. An outlying subspace for a given data point p is the sub- space in which p is an outlier. Outlying subspace detection can facilitate a better characterization process for the detected outliers. It can also enable outlier mining for high- dimensional data to be performed more accurately and efficiently. In this paper, we proposed a new method using genetic algorithm paradigm for searching outlying subspaces efficiently. We developed a technique for efficiently computing the lower and upper bounds of the distance between a given point and its kth nearest neighbor in each possible subspace. These bounds are used to speed up the fitness evaluation of the designed genetic algorithm for outlying subspace detection. We also proposed a random sampling technique to further reduce the computation of the genetic algorithm. The optimal number of sampling data is specified to ensure the accuracy of the result. We show that the proposed method is efficient and effective in handling outlying subspace detection problem by a set of experiments conducted on both synthetic and real-life datasets.

A genetic-based subspace analysis method for improving Error-Correcting Output Coding

Two key factors affecting the performance of Error Correcting Output Codes (ECOC) in multiclass classification problems are the independence of binary classifiers and the problem-dependent coding design. In this paper, we propose an evolutionary algorithm-based approach to the design of an application-dependent codematrix in the ECOC framework. The central idea of this work is to design a three-dimensional codematrix, where the third dimension is the feature space of the problem domain. In order to do that, we consider the feature space in the design process of the codematrix with the aim of improving the independence and accuracy of binary classifiers. The proposed method takes advantage of some basic concepts of ensemble classification, such as diversity of classifiers, and also benefits from the evolutionary approach for optimizing the three-dimensional codematrix, taking into account the problem domain. We provide a set of experimental results using a set of benchmark datasets from the UCI Machine Learning Repository, as well as two real multiclass Computer Vision problems. Both sets of experiments are conducted using two different base learners: Neural Networks and Decision Trees. The results show that the proposed method increases the classification accuracy in comparison with the state-of-the-art ECOC coding techniques.

SPOT: A System for Detecting Projected Outliers From High-dimensional Data Streams

In this paper, we present a new technique, called stream projected ouliter detector (SPOT), to deal with outlier detection problem in high-dimensional data streams. SPOT is unique in a number of aspects. First, SPOT employs a novel window-based time model and decaying cell summaries to capture statistics from the data stream. Second, sparse subspace template (SST), a set of top sparse subspaces obtained by unsupervised and/or supervised learning processes, is constructed in SPOT to detect projected outliers effectively. Multi-Objective genetic algorithm (MOGA) is employed as an effective search method in unsupervised learning for finding outlying subspaces from training data. Finally, SST is able to carry out online self- evolution to cope with dynamics of data streams. This paper provides details on the motivation and technical challenges of detecting outliers from high-dimensional data streams, present an overview of SPOT, and give the plans for system demonstration of SPOT.

Detecting Projected Outliers in High-Dimensional Data Streams

In this paper, we study the problem of projected outlier detection in high dimensional data streams and propose a new technique, called Stream Projected Ouliter deTector (SPOT), to identify outliers embedded in subspaces. Sparse Subspace Template (SST), a set of subspaces obtained by unsupervised and/or supervised learning processes, is constructed in SPOT to detect projected outliers effectively. Multi-Objective Genetic Algorithm (MOGA) is employed as an effective search method for finding outlying subspaces from training data to construct SST. SST is able to carry out online self-evolution in the detection stage to cope with dynamics of data streams. The experimental results demonstrate the efficiency and effectiveness of SPOT in detecting outliers in high-dimensional data streams.

Perceptual Shape-Based Natural Image Representation and Retrieval

Human visual recognition is based largely on shape, yet effectively using shapes in natural image retrieval is a challenging task. Most existing methods are based on the geometric equations of curves computed from processing an entire image. These processes are computationally intensive, lack flexibility and do not take advantage or with minimum use of the Gestalt rules of human vision. By applying certain mechanisms based on the human visual perception process, our methods extract generic shape features from real world images. We extract and group perceptually significant segments and use their properties to create a Euclidean distance matrix for image retrieval. As all the computing is based on simple calculation and one pixel width edges instead of the whole image, this method provides a novel and efficient image feature representation. Testing on standard benchmark datasets and comparison with other well-known methods show this shape analysis method using only compact feature vectors performs well and robustly for real world images.