Dewan Md. Farid

Hybrid decision tree and naïve Bayes classifiers for multi-class classification tasks

In this paper, we introduce two independent hybrid mining algorithms to improve the classification accuracy rates of decision tree (DT) and naive Bayes (NB) classifiers for the classification of multi-class problems. Both DT and NB classifiers are useful, efficient and commonly used for solving classification problems in data mining. Since the presence of noisy contradictory instances in the training set may cause the generated decision tree suffers from overfitting and its accuracy may decrease, in our first proposed hybrid DT algorithm, we employ a naive Bayes (NB) classifier to remove the noisy troublesome instances from the training set before the DT induction. Moreover, it is extremely computationally expensive for a NB classifier to compute class conditional independence for a dataset with high dimensional attributes. Thus, in the second proposed hybrid NB classifier, we employ a DT induction to select a comparatively more important subset of attributes for the production of naive assumption of class conditional independence. We tested the performances of the two proposed hybrid algorithms against those of the existing DT and NB classifiers respectively using the classification accuracy, precision, sensitivity-specificity analysis, and 10-fold cross validation on 10 real benchmark datasets from UCI (University of California, Irvine) machine learning repository. The experimental results indicate that the proposed methods have produced impressive results in the classification of real life challenging multi-class problems. They are also able to automatically extract the most valuable training datasets and identify the most effective attributes for the description of instances from noisy complex training databases with large dimensions of attributes.

An adaptive ensemble classifier for mining concept drifting data streams

It is challenging to use traditional data mining techniques to deal with real-time data stream classifications. Existing mining classifiers need to be updated frequently to adapt to the changes in data streams. To address this issue, in this paper we propose an adaptive ensemble approach for classification and novel class detection in concept drifting data streams. The proposed approach uses traditional mining classifiers and updates the ensemble model automatically so that it represents the most recent concepts in data streams. For novel class detection we consider the idea that data points belonging to the same class should be closer to each other and should be far apart from the data points belonging to other classes. If a data point is well separated from the existing data clusters, it is identified as a novel class instance. We tested the performance of this proposed stream classification model against that of existing mining algorithms using real benchmark datasets from UCI (University of California, Irvine) machine learning repository. The experimental results prove that our approach shows great flexibility and robustness in novel class detection in concept drifting and outperforms traditional classification models in challenging real-life data stream applications.

COMBINING NAIVE BAYES AND DECISION TREE FOR ADAPTIVE INTRUSION DETECTION

In this paper, a new learning algorithm for adaptive network intrusion detection using naive Bayesian classifier and decision tree is presented, which performs balance detections and keeps false positives at acceptable level for different types of network attacks, and eliminates redundant attributes as well as contradictory examples from training data that make the detection model complex. The proposed algorithm also addresses some difficulties of data mining such as handling continuous attribute, dealing with missing attribute values, and reducing noise in training data. Due to the large volumes of security audit data as well as the complex and dynamic properties of intrusion behaviours, several data miningbased intrusion detection techniques have been applied to network-based traffic data and host-based data in the last decades. However, there remain various issues needed to be examined towards current intrusion detection systems (IDS). We tested the performance of our proposed algorithm with existing learning algorithms by employing on the KDD99 benchmark intrusion detection dataset. The experimental results prove that the proposed algorithm achieved high detection rates (DR) and significant reduce false positives (FP) for different types of network intrusions using limited computational resources.

Intelligent facial emotion recognition and semantic-based topic detection for a humanoid robot

Automatic perception of human affective behaviour from facial expressions and recognition of intentions and social goals from dialogue contexts would greatly enhance natural human robot interaction. This research concentrates on intelligent neural network based facial emotion recognition and Latent Semantic Analysis based topic detection for a humanoid robot. The work has first of all incorporated Facial Action Coding System describing physical cues and anatomical knowledge of facial behaviour for the detection of neutral and six basic emotions from real-time posed facial expressions. Feedforward neural networks (NN) are used to respectively implement both upper and lower facial Action Units (AU) analysers to recognise six upper and 11 lower facial actions including Inner and Outer Brow Raiser, Lid Tightener, Lip Corner Puller, Upper Lip Raiser, Nose Wrinkler, Mouth Stretch etc. An artificial neural network based facial emotion recogniser is subsequently used to accept the derived 17 Action Units as inputs to decode neutral and six basic emotions from facial expressions. Moreover, in order to advise the robot to make appropriate responses based on the detected affective facial behaviours, Latent Semantic Analysis is used to focus on underlying semantic structures of the data and go beyond linguistic restrictions to identify topics embedded in the users’ conversations. The overall development is integrated with a modern humanoid robot platform under its Linux C++ SDKs. The work presented here shows great potential in developing personalised intelligent agents/robots with emotion and social intelligence.

Anomaly Network Intrusion Detection Based on Improved Self Adaptive Bayesian Algorithm

Recently, research on intrusion detection in computer systems has received much attention to the computational intelligence society. Many intelligence learning algorithms applied to the huge volume of complex and dynamic dataset for the construction of efficient intrusion detection systems (IDSs). Despite of many advances that have been achieved in existing IDSs, there are still some difficulties, such as correct classification of large intrusion detection dataset, unbalanced detection accuracy in the high speed network traffic, and reduce false positives. This paper presents a new approach to the alert classification to reduce false positives in intrusion detection using improved self adaptive Bayesian algorithm (ISABA). The proposed approach applied to the security domain of anomaly based network intrusion detection, which correctly classifies different types of attacks of KDD99 benchmark dataset with high classification rates in short response time and reduce false positives using limited computational resources.

Adaptive Intrusion Detection based on Boosting and Naïve Bayesian Classifier

In this paper, we introduce a new learning algorithm for adaptive intrusion detection using boosting and naive Bayesian classifier, which considers a series of classifiers and combines the votes of each individual classifier for classifying an unknown or known example. The proposed algorithm generates the probability set for each round using naive Bayesian classifier and updates the weights of training examples based on the misclassification error rate that produced by the training examples in each round. This algorithm addresses the problem of classifying the large intrusion detection dataset, which improves the detection rates (DR) and reduces the false positives (FP) at acceptable level in intrusion detection. We tested the performance of the proposed algorithm with existing data mining algorithms by employing on the KDD99 benchmark intrusion detection dataset, and the experimental results proved that the proposed algorithm achieved high detection rates and significantly reduced the number of false positives for different types of network intrusions.

Novel class detection in concept-drifting data stream mining employing decision tree

In this paper, we propose a new approach for detecting novel class in data stream mining using decision tree classifier that can determine whether an unseen or new instance belongs to a novel class. Most existing data mining classifiers can not detect and classify the novel class instances in real-time data stream mining problems like weather conditions, economical changes, astronomical, and intrusion detection etc, untill the classification models are trained with the labeled instances of the novel class. Arrival of a novel class in concept-drift occurs in data stream mining when new data introduce the new concept classes or remove the old ones. The proposed approach for incremental learning of concept drift considers mining, where the streaming data distributions change over time. It build a decision tree model from training dataset, which continuously updates so that the tree represents the most recent concept in data stream. The experiments on real benchmark data evaluate the efficiency of the proposed approach in both detecting the novel class and classification accuracy with comparisons of traditional data mining classifiers.

Learning intrusion detection based on adaptive bayesian algorithm

Recent intrusion detection have emerged an important technique for information security systems. Its very important that the security mechanisms for an information system should be designed to prevent unauthorized access of system resources and data. Last few years, many intelligent learning techniques of machine learning applied to the large volumes of complex and dynamic audit data for the construction of efficient intrusion detection systems (IDS). This paper presents, theoretical overview of intrusion detection and a new approach for intrusion detection based on adaptive Bayesian algorithm. This algorithm correctly classify different types of attack of KDD99 benchmark intrusion detection dataset with high detection accuracy in short response time. The experimental result also shows that, this algorithm maximize the detection rate (DR) and minimized the false positive rate (FPR) for intrusion detection.

Feature selection and intrusion classification in NSL-KDD cup 99 dataset employing SVMs

Intrusion is the violation of information security policy by malicious activities. Intrusion detection (ID) is a series of actions for detecting and recognising suspicious actions that make the expedient acceptance of standards of confidentiality, quality, consistency, and availability of a computer based network system. In this paper, we present a new approach consists with merging of feature selection and classification for multiple class NSL-KDD cup 99 intrusion detection dataset employing support vector machine (SVM). The objective is to improve the competence of intrusion classification with a significantly reduced set of input features from the training data. In supervised learning, feature selection is the process of selecting the important input training features and removing the irrelevant input training features, with the objective of obtaining a feature subset that produces higher classification accuracy. In the experiment, we have applied SVM classifier on several input feature subsets of training dataset of NSL-KDD cup 99 dataset. The experimental results obtained showed the proposed method successfully bring 91% classification accuracy using only three features and 99% classification accuracy using 36 features, while all 41 training features achieved 99% classification accuracy.

An Ensemble Approach to Classifier Construction based on Bootstrap Aggregation

paper, we introduce a new approach to the classification of streaming data based on bootstrap aggregation (bagging). The proposed approach creates an ensemble model by using ID3 classifier, naive Bayesian classifier, and k-Nearest-Neighbor classifier for a learning scheme where each classifier gives the weighted prediction. ID3, naive Bayesian, and k-Nearest- Neighbor classifiers are very well known data mining methods, which have been already used in many real life classification problems. The proposed approach addresses the practical problems of the classification of streaming data and successfully tested on a number of benchmark problems including large intrusion detection dataset from the UCI machine learning repository to produce a comparison with the established approaches. The experimental results demonstrate that the proposed ensemble classifier achieved high classification rates and generated very low misclassification error.