Zuhair Bandar

An approach for measuring semantic similarity between words using multiple information sources

Semantic similarity between words is becoming a generic problem for many applications of computational linguistics and artificial intelligence. This paper explores the determination of semantic similarity by a number of information sources, which consist of structural semantic information from a lexical taxonomy and information content from a corpus. To investigate how information sources could be used effectively, a variety of strategies for using various possible information sources are implemented. A new measure is then proposed which combines information sources nonlinearly. Experimental evaluation against a benchmark set of human similarity ratings demonstrates that the proposed measure significantly outperforms traditional similarity measures.

Sentence similarity based on semantic nets and corpus statistics

Sentence similarity measures play an increasingly important role in text-related research and applications in areas such as text mining, Web page retrieval, and dialogue systems. Existing methods for computing sentence similarity have been adopted from approaches used for long text documents. These methods process sentences in a very high-dimensional space and are consequently inefficient, require human input, and are not adaptable to some application domains. This paper focuses directly on computing the similarity between very short texts of sentence length. It presents an algorithm that takes account of semantic information and word order information implied in the sentences. The semantic similarity of two sentences is calculated using information from a structured lexical database and from corpus statistics. The use of a lexical database enables our method to model human common sense knowledge and the incorporation of corpus statistics allows our method to be adaptable to different domains. The proposed method can be used in a variety of applications that involve text knowledge representation and discovery. Experiments on two sets of selected sentence pairs demonstrate that the proposed method provides a similarity measure that shows a significant correlation to human intuition

A global-local artificial neural network with application to wave overtopping prediction

We present a hybrid Radial Basis Function (RBF) - sigmoid neural network with a three-step training algorithm that utilises both global search and gradient descent training. We test the effectiveness of our method using four synthetic datasets and demonstrate its use in wave overtopping prediction. It is shown that the hybrid architecture is often superior to architectures containing neurons of a single type in several ways: lower errors are often achievable using fewer hidden neurons and with less need for regularisation. Our Global-Local Artificial Neural Network (GL-ANN) is also seen to compare favourably with both Perceptron Radial Basis Net (PRBFN) and Regression Tree RBFs.

On constructing a fuzzy inference framework using crisp decision trees

This paper proposes a framework which consists of a novel fuzzy inference algorithm to generate fuzzy decision trees from induced crisp decision trees. Fuzzy theoretical techniques are used to fuzzify crisp decision trees in order to soften sharp decision boundaries at decision nodes inherent in this type of trees. A framework for the investigation of various types of membership functions and fuzzy inference techniques is proposed. Once the decision tree has been fuzzified, all branches throughout the tree will fire, resulting in a membership grade being generated at each branch. Five different fuzzy inference mechanisms are used to investigate the degree of interaction between membership grades on each path in the decision tree, which ultimately leads to a final crisp classification. A genetic algorithm is used to optimize and automatically determine the set of fuzzy regions for all branches and simultaneously the degree in which the inference parameters will be applied. Comparisons between crisp trees and the fuzzified trees suggest that the later fuzzy tree is significantly more robust and produces a more balanced classification. In addition, the results obtained from five real-world data sets show that there is a significant improvement in the accuracy of the fuzzy trees when compared with crisp trees.

On global-local artificial neural networks for function approximation

We present a hybrid radial basis function (RBF) sigmoid neural network with a three-step training algorithm that utilizes both global search and gradient descent training. The algorithm used is intended to identify global features of an input-output relationship before adding local detail to the approximating function. It aims to achieve efficient function approximation through the separate identification of aspects of a relationship that are expressed universally from those that vary only within particular regions of the input space. We test the effectiveness of our method using five regression tasks; four use synthetic datasets while the last problem uses real-world data on the wave overtopping of seawalls. It is shown that the hybrid architecture is often superior to architectures containing neurons of a single type in several ways: lower mean square errors are often achievable using fewer hidden neurons and with less need for regularization. Our global-local artificial neural network (GL-ANN) is also seen to compare favorably with both perceptron radial basis net and regression tree derived RBFs. A number of issues concerning the training of GL-ANNs are discussed: the use of regularization, the inclusion of a gradient descent optimization step, the choice of RBF spreads, model selection, and the development of appropriate stopping criteria.

Conversation-Based Natural Language Interface to Relational Databases

This paper proposes a new approach for creating conversation-based natural language interfaces to relational databases by combining goal oriented conversational agents and knowledge trees. Goal oriented conversational agents have proven their capability to disambiguate the users needs and to converse within a context (i.e. specific domain). Knowledge trees used to overcome the lacking of connectivity between the conversational agent and the relational database, through organizing the domain knowledge in knowledge trees. Knowledge trees also work as a road map for the conversational agent dialogue flow. The proposed framework makes it easier for knowledge engineers to develop a reliable conversation-based NLI-RDB. The developed prototype system shows excellent performance on common queries (i. e. queries extracted from expert by a knowledge engineer). The user will have a friendly interface that can converse with the relational database.

Genetic tuning of fuzzy inference within fuzzy classifier systems

In generating a suitable fuzzy classifier system, significant effort is often placed on the determination and the fine tuning of the fuzzy sets. However, in such systems little thought is given to the way in which membership functions are combined within the fuzzy rules. Often traditional fuzzy inference strategies are used which consequently provide no control over how strongly or weakly the inference is applied within these rules. Furthermore such strategies will allow no interaction between grades of membership. A number of theoretical fuzzy inference operators have been proposed for both regression and classification problems but they have not been investigated in the context of real-world applications. In this paper we propose a novel genetic algorithm framework for optimizing the strength of fuzzy inference operators concurrently with the tuning of membership functions for a given fuzzy classifier system. Each fuzzy system is generated using two well-established decision tree algorithms: C4.5 and CHAID. This will enable both classification and regression problems to be addressed within the framework. Each solution generated by the genetic algorithm will produce a set of fuzzy membership functions and also determine how strongly the inference will be applied within each fuzzy rule. We investigate several theoretical proven fuzzy inference techniques (T-norms) in the context of both classification and regression problems. The methodology proposed is applied to a number of real-world data sets in order to determine the effects of the simultaneous tuning of membership functions and inference parameters on the accuracy and robustness of fuzzy classifiers.

Growing a fuzzy decision forest

The creation of multiple decision trees is a relatively new concept, which aims to improve the predictive power of a single decision tree. The approach is based on the induction of more than one C4.5-type decision tree from the same training sample where each decision tree represents a different view of the same domain resulting in a network of decision tree models. The utilization of multiple decision trees has been shown to lead to an improved performance by combining multiple perspectives of the same domain thus increasing the information content whereas, in comparison, a single decision tree can only represent one restricted view of the domain. One predominant weakness in creating a single tree is the generation of sharp decision boundaries at every node within the tree, which results in small changes in attribute values giving radically different classifications. This problem becomes more apparent with the generation of multiple trees. This paper presents a novel approach of overcoming this weakness through the use of fuzzy decision forests. The approach is based upon the induction of multiple fuzzy decision trees from one training sample, where each tree represents a different view of the data domain. A genetic algorithm (GA) is used to select a series of high performance membership functions, which are then applied to branches within all decision trees in the forest. The GA will in addition optimise a pre-selected fuzzy inference technique, which will assign a degree of strength to the conjunction and disjunction of membership grades within the tree. Considerable improvements in classification accuracy over original single C4.5 (crisp) trees were obtained using two real world data sets.

On the Optimization of T-norm parameters within Fuzzy Decision Trees

The success of fuzzy decision trees when applied to classification problems is usually attributed to the selection and tuning of fuzzy sets to represent the problem domain. The impact of fuzzy inference in combining grades of membership throughout fuzzy trees has not been considered in-depth. A number of parameterized fuzzy operators based on the T-norm model have been proposed but not exploited in practical applications. This paper presents a comparative study which examines a number of T-norm and T-conorms and their application within Fuzzy Decision Trees. The methodology uses a Genetic Algorithm to tune the weights of T-norm operators and optimize fuzzy membership functions simultaneously in fuzzy trees. The paper applies the methodology to two Fuzzy Decision Tree algorithms known as FIA and Fuzzy CHAIRS. Six different T-norm models are investigated across five real world datasets. Experimental results indicate that significant improvements can be made in the performance of fuzzy trees when the most appropriate T-norm is optimised for a specific domain.

A semantic-based conversational agent framework

This paper focuses on the implementation of a novel semantic-based Conversational Agent (CA) framework. Traditional CA frameworks interpret scripts consisting of structural patterns of sentences. User input is matched against such patterns and an associated response is sent as output. This technique, which takes into account solely surface information, that is, the structural form of a sentence, requires the scripter to anticipate the inordinate ways that a user may send input. This is a tiresome and time-consuming process. As such, a semantic-based CA that interprets scripts consisting of natural language sentences will alleviate such burden. Using a pre-determined, domain-specific scenario, the CA was evaluated by participants indicating promising results.