Matous Cejnek

Learning entropy for novelty detection a cognitive approach for adaptive filters

This paper recalls the practical calculation of Learning Entropy (LE) for novelty detection, extends it for various gradient techniques and discusses its use for multivariate dynamical systems with ability of distinguishing between data perturbations or system-function perturbations. LG has been recently introduced for novelty detection in time series via supervised incremental learning of polynomial filters, i.e. higher-order neural units (HONU). This paper demonstrates LG also on enhanced gradient descent adaptation techniques that are adopted and summarized for HONU. As an aside, LG is proposed as a new performance index of adaptive filters. Then, we discuss Principal Component Analysis and Kernel PCA for HONU as a potential method to suppress detection of data-measurement perturbations and to enforce LG for system-perturbation novelties.

A Fast Neural Network Approach to Predict Lung Tumor Motion during Respiration for Radiation Therapy Applications

During radiotherapy treatment for thoracic and abdomen cancers, for example, lung cancers, respiratory motion moves the target tumor and thus badly affects the accuracy of radiation dose delivery into the target. A real-time image-guided technique can be used to monitor such lung tumor motion for accurate dose delivery, but the system latency up to several hundred milliseconds for repositioning the radiation beam also affects the accuracy. In order to compensate the latency, neural network prediction technique with real-time retraining can be used. We have investigated real-time prediction of 3D time series of lung tumor motion on a classical linear model, perceptron model, and on a class of higher-order neural network model that has more attractive attributes regarding its optimization convergence and computational efficiency. The implemented static feed-forward neural architectures are compared when using gradient descent adaptation and primarily the Levenberg-Marquardt batch algorithm as the ones of the most common and most comprehensible learning algorithms. The proposed technique resulted in fast real-time retraining, so the total computational time on a PC platform was equal to or even less than the real treatment time. For one-second prediction horizon, the proposed techniques achieved accuracy less than one millimeter of 3D mean absolute error in one hundred seconds of total treatment time.

Adaptive classification of EEG for dementia diagnosis

The paper presents new approach to dementia detection in time series of measured EEG. The proposed method introduced in this paper evaluates EEG signal according to included novelty. This novelty is evaluated using prediction error and increment of adaptive weights obtained during adaptive prediction of individual EEG channels. Normalization of learning rate was used for adaptation of predictor. The linear dynamic neuron was used as a predictor with gradient descent adaptation. The method was cross-validated on dataset containing 110 patients suffering with dementia and 110 controls. Best achieved results with our method tested on validation dataset were 90% of specificity and 90% of sensitivity. The achievements and limiting assumptions of this method are discussed as well.

Study of Learning Entropy for Novelty Detection in lung tumor motion prediction for target tracking radiation therapy

This paper presents recently introduced concept of Learning Entropy (LE) for time series and recalls the practical form of its evaluation in real time. Then, a technique that estimates the increased risk of prediction inaccuracy of adaptive predictors in real time using LE is introduced. On simulation examples using artificial signal and real respiratory time series, it is shown that LE can be used to evaluate the actual validity of the adaptive predicting model of time series in real time. The introduced technique is discussed as a potential approach to the improvement of accuracy of lung tumor tracking radiation therapy.

Study of Learning Entropy for onset detection of epileptic seizures in EEG time series

This paper presents a case study of non-Shannon entropy, i.e. Learning Entropy (LE), for instant detection of onset of epileptic seizures in individual EEG time series. Contrary to entropy methods of EEG evaluation that are based on probabilistic computations, we present the LE-based approach that evaluates the conformity of individual samples of data to the contemporary learned governing law of a learning system and thus LE can detect changes of dynamics on individual samples of data. For comparison, the principle and the results are compared to the Sample Entropy approach. The promising results indicate the LE potentials for feature extraction enhancement for early detection of epileptic seizures on individual-data-sample basis.

Adaptive polynomial filters with individual learning rates for computationally efficient lung tumor motion prediction

This paper presents a study of higher-order neural units as polynomial adaptive filters with multiple-learning-rate gradient descent for 3-D lung tumor motion prediction. The method is compared with single-learning rate gradient descent approaches with and without learning rate normalization. Experimental analysis is done with linear and quadratic neural unit. The influence of correct selection of adaptation parameters and the dependence of learning time on accuracy were experimentally analyzed. The prediction accuracy is nearly equal to recently published results of batch retraining approaches while the computational efficiency is higher for the introduced approach.

Influence of type and level of noise on the performance of an adaptive novelty detector

This paper investigates the influence of the signal to noise ratio (SNR) and the type of a noise on the performance of two adaptive novelty detection methods. The evaluated methods are Learning Entropy (LE) and Error and Learning Based Novelty Detection (ELBND). The methods are compared in empirical way in classification framework. A classification based only on the error of the adaptive model was used as a reference. The research in this field is important, because a noise is present in every measured data and can drastically influence the result of tasks like the novelty detection. Moreover, various types of noise can influence the novelty detection in different ways, therefore the optimal method of adaptive novelty detection can be hard to choose. This assumption is supported by experimental results in this study.

Concept drift robust adaptive novelty detection for data streams

Abstract In this paper we study the performance of two original adaptive unsupervised novelty detection methods (NDMs) on data with concept drift. Newly, the concept drift is considered as a challenging data imbalance that should be ignored by the NDMs, and only system changes and outliers represent novelty. The field of application for such NDMs is broad. For example, the method can be used as a supportive method for real-time system fault detection, for onset detection of events in biomedical signals, in monitoring of nonlinearly controlled processes, for event driven automated trading, etc. The two newly studied methods are the error and learning based novelty detection (ELBND) and the learning entropy (LE) based detection. These methods use both the error and weight increments of a (supervised) learning model. Here, we study these methods with normalized least-mean squares (NLMS) adaptive filter, and while the NDMs were studied on various real life tasks, newly, we carry out the study on two types of data streams with concept drift to analyze the general ability for unsupervised novelty detection. The two data streams, one with system changes, second with outliers, represent different novelty scenarios to demonstrate the performance of the proposed NDMs with concept drifts in data. Both tested NDMs work as a feature extractor. Thus, a classification framework is used for the evaluation of the obtained features and NDM benchmarking, where two other NDMs, one based on the adaptive model plain error, second using the sample entropy (SE), are used as the reference for the comparison to the proposed methods. The results show that both newly studied NDMs are superior to the merely use of the plain error of adaptive model and also to the sample entropy based detection while they are robust against the concept drift occurrence.

Online data centering modifications for adaptive filtering with NLMS algorithm

This paper presents method with two modifications how to transform data in real-time for better performance of normalized least mean squares (NLMS) algorithm. The method centers input vector for adaptive filter online according to temporary or historical statistical attributes of the input vector. The method is derived for an adaptive filter with NLMS adaptation. The filter implementation is the linear neural unit. The stability condition for the given filter is also presented. The filter is tested on multiple simulated time series contaminated with white noise. The convergence of the suggested algorithms is also analyzed and time complexity is discussed.

V-shaped neurons in hidden layer of ANN universal approximator without flat domains

A three-layer perceptron ANN is designed to avoid difficulties during learning process. The resulting V-shaped Artificial Neural Network has universal approximation property and its learning is based on the minimization of least squares sum. The main advantage of this approach is in the absence of flat domains with a small norm of objective function gradiënt. Therefore, any optimization method which is based on local searching yields from this property of objective function. Due to multimodality of objective function in the case of ANN learning, Cuckoo Search heuristics with embedded Levy Flights was used both for V-shaped ANN learning and for the learning of MLP and RBF as referential ANNs. Time complexity and reliability of learning are demonstrated on simple example.