Ljupčo Todorovski

Knowledge Discovery in Databases: PKDD 2003

This paper describes the Robosail project. It started in 1997 with the aim to build a self-learning auto pilot for a single handed sailing yacht. The goal was to make an adaptive system that would help a single handed sailor to go faster on average in a race. Presently, after five years of development and a number of sea trials, we have a commercial system available (www.robosail.com). It is a hybrid system using agent technology, machine learning, data mining and rule-based reasoning. Apart from describing the system we try to generalize our findings, and argue that sailing is an interesting paradigm for a class of hybrid systems that one could call Skill-based Systems.

Machine Learning: ECML 2003

This paper describes the Robosail project. It started in 1997 with the aim to build a self-learning auto pilot for a single handed sailing yacht. The goal was to make an adaptive system that would help a single handed sailor to go faster on average in a race. Presently, after five years of development and a number of sea trials, we have a commercial system available (www.robosail.com). It is a hybrid system using agent technology, machine learning, data mining and rule-based reasoning. Apart from describing the system we try to generalize our findings, and argue that sailing is an interesting paradigm for a class of hybrid systems that one could call Skill-based Systems.

Discovering dynamics: from inductive logic programming to machine discovery

Machine discovery systems help humans to find natural laws from collections of experimentally collected data. Most of the laws found by existing machine discovery systems describe static situations, where a physical system has reached equilibrium. In this paper, we consider the problem of discovering laws that govern the behavior of dynamical systems, i.e., systems that change their state over time. Based on ideas from inductive logic programming and machine discovery, we present two systems, QMN and LAGRANGE, for discovery of qualitative and quantitative laws from quantitative (numerical) descriptions of dynamical system behavior. We illustrate their use by generating a variety of dynamical system models from example behaviors.

Predictive Performance of Weghted Relative Accuracy

Weighted relative accuracy was proposed in [4] as an alternative to classification accuracy typically used in inductive rule learners. Weighted relative accuracy takes into account the improvement of the accuracy relative to the default rule (i.e., the rule stating that the same class should be assigned to all examples), and also explicitly incorporates the generality of a rule (i.e., the number of examples covered). In order to measure the predictive performance of weighted relative accuracy, we implemented it in the rule induction algorithm CN2. Our main results are that weighted relative accuracy dramatically reduces the size of the rule sets induced with CN2 (on average by a factor 9 on the 23 datasets we used), at the expense of only a small average drop in classification accuracy.

Inductive process modeling

Abstract In this paper, we pose a novel research problem for machine learning that involves constructing a process model from continuous data. We claim that casting learned knowledge in terms of processes with associated equations is desirable for scientific and engineering domains, where such notations are commonly used. We also argue that existing induction methods are not well suited to this task, although some techniques hold partial solutions. In response, we describe an approach to learning process models from time-series data and illustrate its behavior in three domains. In closing, we describe open issues in process model induction and encourage other researchers to tackle this important problem.

Modelling and prediction of phytoplankton growth with equation discovery

In contrast with traditional modelling methods, which are used to identify parameter values of a model with known structure, equation discovery systems identify the structure of the model also. The model generated with such systems can give experts a better insight into the measured data and can be also used for predicting future values of the measured variables. This paper presents LAGRAMGE, an equation discovery system that allows the user to define the space of possible model structures and to make use of domain specific expert knowledge in the form of function definitions. We use LAGRAMGE to automate the modelling of phytoplankton growth in lake Glumsoe, Denmark. The structure of the model constructed with LAGRAMGE agrees with human experts’ expectations. The model can be successfully used for long term prediction of phytoplankton concentration during algal blooms.

Computational Discovery of Scientific Knowledge

This chapter introduces the field of computational scientific discovery and provides a brief overview thereof. We first try to be more specific about what scientific discovery is and also place it in the broader context of the scientific enterprise. We discuss the components of scientific behavior, that is, the knowledge structures that arise in science and the processes that manipulate them. We give a brief historical review of research in computational scientific discovery and discuss the lessons learned, especially in relation to work in data mining that has recently received substantial attention. Finally, we discuss the contents of the book and how it fits in the overall framework of computational scientific discovery.

Constructing explanatory process models from biological data and knowledge

OBJECTIVE We address the task of inducing explanatory models from observations and knowledge about candidate biological processes, using the illustrative problem of modeling photosynthesis regulation. METHODS We cast both models and background knowledge in terms of processes that interact to account for behavior. We also describe IPM, an algorithm for inducing quantitative process models from such input. RESULTS We demonstrate IPMs use both on photosynthesis and on a second domain, biochemical kinetics, reporting the models induced and their fit to observations. CONCLUSION We consider the generality of our approach, discuss related research on biological modeling, and suggest directions for future work.

Ranking with predictive clustering trees

A novel class of applications of predictive clustering trees is addressed, namely ranking. Predictive clustering trees, as implemented in CLUS, allow for predicting multiple target variables. This approach makes sense especially if the target variables are not independent of each other. This is typically the case in ranking, where the (relative) performance of several approaches on the same task has to be predicted from a given description of the task. We propose to use predictive clustering trees for ranking. As compared to existing ranking approaches which are instance-based, our approach also allows for an explanation of the predicted rankings. We illustrate our approach on the task of ranking machine learning algorithms, where the (relative) performance of the learning algorithms on a dataset has to be predicted from a given dataset description.

Parameter estimation with bio-inspired meta-heuristic optimization: modeling the dynamics of endocytosis

BackgroundWe address the task of parameter estimation in models of the dynamics of biological systems based on ordinary differential equations (ODEs) from measured data, where the models are typically non-linear and have many parameters, the measurements are imperfect due to noise, and the studied system can often be only partially observed. A representative task is to estimate the parameters in a model of the dynamics of endocytosis, i.e., endosome maturation, reflected in a cut-out switch transition between the Rab5 and Rab7 domain protein concentrations, from experimental measurements of these concentrations. The general parameter estimation task and the specific instance considered here are challenging optimization problems, calling for the use of advanced meta-heuristic optimization methods, such as evolutionary or swarm-based methods.ResultsWe apply three global-search meta-heuristic algorithms for numerical optimization, i.e., differential ant-stigmergy algorithm (DASA), particle-swarm optimization (PSO), and differential evolution (DE), as well as a local-search derivative-based algorithm 717 (A717) to the task of estimating parameters in ODEs. We evaluate their performance on the considered representative task along a number of metrics, including the quality of reconstructing the system output and the complete dynamics, as well as the speed of convergence, both on real-experimental data and on artificial pseudo-experimental data with varying amounts of noise. We compare the four optimization methods under a range of observation scenarios, where data of different completeness and accuracy of interpretation are given as input.ConclusionsOverall, the global meta-heuristic methods (DASA, PSO, and DE) clearly and significantly outperform the local derivative-based method (A717). Among the three meta-heuristics, differential evolution (DE) performs best in terms of the objective function, i.e., reconstructing the output, and in terms of convergence. These results hold for both real and artificial data, for all observability scenarios considered, and for all amounts of noise added to the artificial data. In sum, the meta-heuristic methods considered are suitable for estimating the parameters in the ODE model of the dynamics of endocytosis under a range of conditions: With the model and conditions being representative of parameter estimation tasks in ODE models of biochemical systems, our results clearly highlight the promise of bio-inspired meta-heuristic methods for parameter estimation in dynamic system models within system biology.