Matjaz Kukar

Reliable Classifications with Machine Learning

In the past decades Machine Learning algorithms have been successfully used in numerous classification problems. While they usually significantly outperform domain experts (in terms of classification accuracy or otherwise), they are mostly not being used in practice. A plausible reason for this is that it is difficult to obtain an unbiased estimation of a single classifications reliability. In the paper we propose a general transductive method for estimation of classifications reliability on single examples that is independent of the applied Machine Learning algorithm. We compare our method with existing approaches and discuss its advantages. We perform extensive testing on 14 domains and 6 Machine Learning algorithms and show that our approach can frequently yield more than 100% improvement in reliability estimation performance.

An application of machine learning in the diagnosis of ischaemic heart disease

Ischaemic heart disease is one of the worlds most important causes of mortality, so improvements and rationalization of diagnostic procedures would be very useful. The four diagnostic levels consist of evaluation of signs and symptoms of the disease and ECG (electrocardiogram) at rest, sequential ECG testing during the controlled exercise, myocardial scintigraphy and finally coronary angiography. The diagnostic process is stepwise and the results are interpreted hierarchically, i.e, the next step is necessary only if the results of the former are inconclusive. Because suggestibility is possible, the results of each step are interpreted individually and only the results of the highest step are valid. On the other hand, machine learning methods may be capable of objective interpretation of all available results for the same patient and in this way increase the diagnostic accuracy, sensitivity and specificity of each step. In the usual setting, the machine learning algorithms are tuned to maximize classification accuracy. In our case, the sensitivity and specificity were much more important, so we generalized the algorithms to take in account the variable misclassification costs. The costs can be tuned in order to bias the algorithms towards higher sensitivity or specificity. We conducted many experiments with four learning algorithms and different variations of our dataset (327 patients with completed diagnostic procedures). Our results show that improvements using machine learning techniques are reasonable and might find good use in practice.

Separation of Interleaved Web Sessions with Heuristic Search

We describe a heuristic search-based method for interleaved HTTP (Web) session reconstruction building upon first order Markov models. An interleaved session is generated by a user who is concurrently browsing the same web site in two or more web sessions (browser tabs or windows). In order to assure data quality for subsequent phases in analyzing users browsing behavior, such sessions need to be separated in advance. We propose a separating process based on best-first search and trained first order Markov chains. We develop a testing method based on various measures of reconstructed sessions similarity to original ones. We evaluate the developed method on two real world click stream data sources: a web shop and a university student records information system. Preliminary results show that the proposed method performs well.

Multi-level association rules and directed graphs for spatial data analysis

We propose a methodology that upgrades the methods of the Lagrangian analysis of surface sea-water parcels. This methodology includes data mining with efficient visualization techniques, namely, spatial-temporal association rules and multi-level directed graphs with different levels of space and time granularity. In the resulting multi-level directed graphs we can intertwine knowledge from various disciplines related to oceanography (in our application) and perform the mining of such graphs. We evaluate the proposed methodology on Lagrangian tracking of virtual particles in the velocity field of the numerical model called the Mediterranean Ocean Forecasting Model (MFS). We describe an efficient algorithm based on label propagation clustering, which finds cycles and paths in multi-level directed graphs and reveals how the number and size of the cycles depend on the seasons. In addition, we offer three interesting results of the visualization and mining of such graphs, that is, the 12months periodicity of the exchange of water masses among sea areas, the separation of Mediterranean Sea circulation in summer and winter situations, obtained with the hierarchical clustering of multi-level directed graphs, and finally, with visualization with multi-level directed graphs we confirm the reversal of sea circulation in the Ionian Sea over the last decades. The aforementioned results received a very favorable evaluation from oceanographic experts.

Making Reliable Diagnoses with Machine Learning: A Case Study

In the past decades Machine Learning tools have been successfully used in several medical diagnostic problems. While they often significantly outperform expert physicians (in terms of diagnostic accuracy, sensitivity, and specificity), they are mostly not used in practice. One reason for this is that it is dificult to obtain an unbiased estimation of diagnoses reliability. We propose a general framework for reliability estimation, based on transductive inference. We show that our reliability estimation is closely connected with a general notion of significance tests. We compare our approach with classical stepwise diagnostic process where reliability of diagnose is presented as its post-test probability. The presented approach is evaluated in practice in the problem of clinical diagnosis of coronary artery disease, where significant improvements over existing techniques are achieved.

Reliable diagnostics for coronary artery disease

In the past few decades, machine learning tools have been successfully used in several medical diagnostic problems. While they often significantly outperform expert physicians (in terms of diagnostic accuracy, sensitivity and specificity), they are mostly not being used in practice. One reason for this is that it is difficult to obtain an unbiased estimation of the diagnosiss reliability. We discuss how the reliability of diagnoses is assessed in medical decision-making and propose a general framework for reliability estimation in machine learning, based on transductive inference. We compare our approach with the usual machine-learning probabilistic approach, as well as with classical step-wise diagnostic process, where the reliability of a diagnosis is presented as its post-test probability. The proposed transductive approach is evaluated in a practical problem of the clinical diagnosis of coronary artery disease. Significant improvements over existing techniques are achieved.

An Application of Machine Learning in the Diagnosis of Ischaemic Heart Disease

Ishaemic heart disease is one of the worlds most important causes of mortality, so improvements and rationalization of diagnostic procedures would be very useful. The four diagnostic levels consist of evaluation of signs and symptoms of the disease and ECG (electrocardiogram) at rest, sequential ECG testing during the controlled exercise, myocardial scintigraphy and finally coronary angiography. The diagnostic process is stepwise and the results are interpreted hierarchically, i.e. the next step is necessary only if the results of the former are inconclusive. Because the suggestibility is possible, the results of each step are interpreted individually and only the results of the highest step are valid. On the other hand, Machine Learning methods may be able of objective interpretation of all available results for the same patient and in this way increase the diagnostic accuracy of each step. We conducted many experiments with four learning algorithms and different variations of our dataset (327 patients with completed diagnostic procedures). Our results show that improvements using Machine Learning techniques are reasonable and might find good use in practice.

Dynamic fuzzy paths and cycles in multi-level directed graphs

Abstract In this paper we propose improved algorithms for the discovery of significant paths and cycles that dynamically evolve through time in a series of multi-level directed graphs. First, we search for the most probable paths and combine them into clusters based on similar edges. We combine paths into dynamic fuzzy paths. We also detect cycles in different paths and combine them into dynamic fuzzy cycles. We obtain dynamic fuzzy structures using the hierarchical clustering of individual structures. For paths, the clustering distance depends on common edges, while for cycles we calculate the distance on the basis of common vertices. We apply the developed algorithms to a time series of multi-level directed graphs obtained from the results from the numerical model Mediterranean Ocean Forecasting System during the period 1999–2011. We compare the results with known structures from the oceanographic literature. With our approach we find a high similarity between the resulting dynamic fuzzy paths and cycles and structures found by oceanographic experts. When comparing the cycles, the expert sees our results as a convex hull of the average of individual cycles. On the other hand, the method reveals undiscovered paths and gyres, which can be verified through observation.

Learning from depth sensor data using inductive logic programming

The problem of detecting objects and their movements in sensor data is of crucial importance in providing safe navigation through both indoor and outdoor environments for the visually impaired. In our setting we use depth-sensor data obtained from a simulator and use inductive logic programming (ILP), a subfield of machine learning that deals with learning concept descriptions, to learn how to detect borders, find the border that is nearest to some point of interest, and border correspondence through time. We demonstrate how ILP can be used to tackle this problem in an incremental manner by using previously learned predicates to construct more complex ones. The learned concept descriptions show high (> 90%) accuracy and their natural language interpretation closely matches an intuitive understanding of their meaning.