Leonard E. Trigg

Data mining in bioinformatics using Weka

UNLABELLED The Weka machine learning workbench provides a general-purpose environment for automatic classification, regression, clustering and feature selection-common data mining problems in bioinformatics research. It contains an extensive collection of machine learning algorithms and data pre-processing methods complemented by graphical user interfaces for data exploration and the experimental comparison of different machine learning techniques on the same problem. Weka can process data given in the form of a single relational table. Its main objectives are to (a) assist users in extracting useful information from data and (b) enable them to easily identify a suitable algorithm for generating an accurate predictive model from it. AVAILABILITY http://www.cs.waikato.ac.nz/ml/weka.

Jumble Java Byte Code to Measure the Effectiveness of Unit Tests

Jumble is a byte code level mutation testing tool for Java which inter-operates with JUnit. It has been designed to operate in an industrial setting with large projects. Heuristics have been included to speed the checking of mutations, for example, noting which test fails for each mutation and running this first in subsequent mutation checks. Significant effort has been put into ensuring that it can test code which uses custom class loading and reflection. This requires careful attention to class path handling and coexistence with foreign class-loaders. Jumble is currently used on a continuous basis within an agile programming environment with approximately 370,000 lines of Java code under source control. This checks out project code every fifteen minutes and runs an incremental set of unit tests and mutation tests for modified classes. Jumble is being made available as open source.

A diagnostic tool for tree based supervised classification learning algorithms

The process of developing applications of machine learning and data mining that employ supervised classification algorithms includes the important step of knowledge verification. Interpretable output is presented to a user so that they can verify that the knowledge contained in the output makes sense for the given application. As the development of an application is an iterative process it is quite likely that a user would wish to compare models constructed at various times or stages. One crucial stage where comparison of models is important is when the accuracy of a model is being estimated, typically using some form of cross-validation. This stage is used to establish an estimate of how well a model will perform on unseen data. This is vital information to present to a user, but it is also important to show the degree of variation between models obtained from the entire dataset and models obtained during cross-validation. In this way it can be verified that the cross-validation models are at least structurally aligned with the model garnered from the entire dataset. This paper presents a diagnostic tool for the comparison of tree-based supervised classification models. The method is adapted from work on approximate tree matching and applied to decision trees. The tool is described together with experimental results on standard datasets.

Experiences with a weighted decision tree learner

Machine learning algorithms for inferring decision trees typically choose a single “best” tree to describe the training data. Recent research has shown that classification performance can be significantly improved by voting predictions of multiple, independently produced decision trees. This paper describes an algorithm, OB1, that produces a weighted sum over many possible models. Model weights are determined by the prior probability of the model, as well as the performance of the model during training. We describe an implementation of OBI that includes all possible decision trees as well as naive Bayesian models within a single option tree. Constructing all possible decision trees is very expensive, growing exponentially in the number of attributes. However it is possible to use the internal structure of the option tree to avoid recomputing values. In addition, the current implementation allows the option tree to be depth bounded.

Objective measurement of mushroom quality relative to industry inspectors

A machine learning classification system has been developed to sort mushrooms into similar quality grades to those used by human inspectors. The attributes considered for the algorithm included weight, firmness, image features and some subjective scales of the common cultivated button mushroom. Two grading systems were tested; one involving three broad quality grades and a more detailed five-grade system. Two machine learning methods were used to build quality prediction models, relative to the mushroom quality grade criteria specified by the inspectors. A head inspector was used as a reference grading expert and the mis-classification error of the models based on his/her grading for both three and five grades varied from 17 to 22%. The accuracy of the quality prediction models is at the upper level of the variation measured between mushroom industry inspectors. This machine learning classification system provided insights into the subjective decision-making regarding mushroom quality.