Charles R. Day

An investigation into the agreement between clinical, biomechanical and neurophysiological measures of spasticity:

Objective: To quantify agreement between three clinically usable methods of measuring spasticity. Methods: Patients with a first stroke who had no useful functional movement in the upper limb within six weeks from stroke onset were eligible to participate. Spasticity at the wrist joint was simultaneously measured using three methods, during an externally imposed passive stretch at two (uncontrolled) displacement velocities. The measures used were a common clinical measure (modified Ashworth Scale), a biomechanical measure (resistance to passive movement) and a neurophysiological measure (muscle activity). Results: One hundred patients (54 men and 46 women) with a median age of 74 years (range 43—91) participated. Median time since stroke was three weeks (range 1—6), the right side was affected in 52 patients and the left in 48 patients. Based on muscle activity measurement, 87 patients had spasticity. According to the modified Ashworth score 44 patients had spasticity. Sensitivity of modified Ashworth score, when compared with muscle activity recordings, was 0.5 and specificity was 0.92. Based on muscle activity patterns, patients could be classified into five subgroups. The biomechanical measures showed no consistent relationship with the other measures. Conclusion: The presentations of spasticity are variable and are not always consistent with existing definitions. Existing clinical scales that depend on the quantification of muscle tone may lack the sensitivity to quantify the abnormal muscle activation and stiffness associated with common definitions of spasticity. Neurophysiological measures may provide more clinically useful information for the management and assessment of spasticity.

Reservoir computing and extreme learning machines for non-linear time-series data analysis

Random projection architectures such as Echo state networks (ESNs) and Extreme Learning Machines (ELMs) use a network containing a randomly connected hidden layer and train only the output weights, overcoming the problems associated with the complex and computationally demanding training algorithms traditionally used to train neural networks, particularly recurrent neural networks. In this study an ESN is shown to contain an antagonistic trade-off between the amount of non-linear mapping and short-term memory it can exhibit when applied to time-series data which are highly non-linear. To overcome this trade-off a new architecture, Reservoir with Random Static Projections (R(2)SP) is investigated, that is shown to offer a significant improvement in performance. A similar approach using an ELM whose input is presented through a time delay (TD-ELM) is shown to further enhance performance where it significantly outperformed the ESN and R(2)SP as well other architectures when applied to a novel task which allows the short-term memory and non-linearity to be varied. The hard-limiting memory of the TD-ELM appears to be best suited for the data investigated in this study, although ESN-based approaches may offer improved performance when processing data which require a longer fading memory.

Defect Detection in Reinforced Concrete Using Random Neural Architectures

This article discusses how detecting defects within reinforced concrete is vital to the safety and durability of infrastructure. A non-invasive technique, ElectroMagnetic Anomaly Detection (EMAD) is used in this article to provide information into the electromagnetic properties of reinforcing steel for which data analysis is currently performed visually. The first use of two neural network approaches to automate the analysis of this data is investigated in this article. These approaches are called Echo State Networks (ESNs) and Extreme Learning Machines (ELMs) where fast and efficient training procedures allow networks to be trained and evaluated in less time than traditional neural network approaches. Data collected from real-world concrete structures are analyzed in this article using these two approaches as well as using a simple threshold measure and a standard recurrent neural network. Two ESN architectures provided the best performance for a mesh-reinforced concrete structure, while the ELM approach offers a large improvement in the performance of a single tendon-reinforced structure.

Artificial neural network analysis of hydrocarbon profiles for the ageing of Lucilia sericata for post mortem interval estimation

In analytical chemistry large datasets are collected using a variety of instruments for multiple tasks, where manual analysis can be time-consuming. Ideally, it is desirable to automate this process while obtaining an acceptable level of accuracy, two aims that artificial neural networks (ANNs) can fulfil. ANNs possess the ability to classify novel data based on their knowledge of the domain to which they have been exposed. ANNs can also analyse non-linear data, tolerate noise within data and are capable of reducing time taken to classify large amounts of novel data once trained, making them well-suited to the field of analytical chemistry where large datasets are present (such as that collected from gas chromatography-mass spectrometry (GC-MS)). In this study, the use of ANNs for the autonomous analysis of GC-MS profiles of Lucilia sericata larvae is investigated, where ANNs are required to estimate the age of the larvae to aid in the estimation of the post mortem interval (PMI). Two ANN analysis approaches are presented, where the ANN correctly classified the data with accuracy scores of 80.8% and 87.7% and Cohens Kappa coefficients of 0.78 and 0.86. Inspection of these results shows the ANN to confuse two consecutive days which are of the same life stage and as a result are very similar in their chemical profile, which can be expected. The grouping of these two days into one class further improved results where accuracy scores 89% and 97.5% were obtained for the two analysis approaches.

Pruning reservoirs with Random Static Projections

Reservoir Computing is a relatively new field of Recurrent Neural Networks in which only the output weights are re-calculated by the training process, removing the problems associated with traditional gradient descent algorithms. As the reservoir is recurrent, it can possess short term memory, but there is a trade-off between the amount of memory a reservoir can have and its nonlinear mapping capabilities. A new, custom architecture was recently proposed to overcome this by combining a reservoir with an extreme learning machine to deliver improved results. This paper extends this architecture further by introducing a ranking and pruning algorithm which removes neurons according to their significance. This provides further insight into the type of reservoir characteristics needed for a given task, and is supported by further reservoir measures of non-linearity and memory. These techniques are demonstrated on artificial and real world data.

Critical Mutation Rate Has an Exponential Dependence on Population Size in Haploid and Diploid Populations

Understanding the effect of population size on the key parameters of evolution is particularly important for populations nearing extinction. There are evolutionary pressures to evolve sequences that are both fit and robust. At high mutation rates, individuals with greater mutational robustness can outcompete those with higher fitness. This is survival-of-the-flattest, and has been observed in digital organisms, theoretically, in simulated RNA evolution, and in RNA viruses. We introduce an algorithmic method capable of determining the relationship between population size, the critical mutation rate at which individuals with greater robustness to mutation are favoured over individuals with greater fitness, and the error threshold. Verification for this method is provided against analytical models for the error threshold. We show that the critical mutation rate for increasing haploid population sizes can be approximated by an exponential function, with much lower mutation rates tolerated by small populations. This is in contrast to previous studies which identified that critical mutation rate was independent of population size. The algorithm is extended to diploid populations in a system modelled on the biological process of meiosis. The results confirm that the relationship remains exponential, but show that both the critical mutation rate and error threshold are lower for diploids, rather than higher as might have been expected. Analyzing the transition from critical mutation rate to error threshold provides an improved definition of critical mutation rate. Natural populations with their numbers in decline can be expected to lose genetic material in line with the exponential model, accelerating and potentially irreversibly advancing their decline, and this could potentially affect extinction, recovery and population management strategy. The effect of population size is particularly strong in small populations with 100 individuals or less; the exponential model has significant potential in aiding population management to prevent local (and global) extinction events.

Optimizing Echo State Networks for Static Pattern Recognition

Static pattern recognition requires a machine to classify an object on the basis of a combination of attributes and is typically performed using machine learning techniques such as support vector machines and multilayer perceptrons. Unusually, in this study, we applied a successful time-series processing neural network architecture, the echo state network (ESN), to a static pattern recognition task. The networks were presented with clamped input data patterns, but in this work, they were allowed to run until their output units delivered a stable set of output activations, in a similar fashion to previous work that focused on the behaviour of ESN reservoir units. Our aim was to see if the short-term memory developed by the reservoir and the clamped inputs could deliver improved overall classification accuracy. The study utilized a challenging, high dimensional, real-world plant species spectroradiometry classification dataset with the objective of accurately detecting one of the world’s top 100 invasive plant species. Surprisingly, the ESNs performed equally well with both unsettled and settled reservoirs. Delivering a classification accuracy of 96.60%, the clamped ESNs outperformed three widely used machine learning techniques, namely support vector machines, extreme learning machines and multilayer perceptrons. Contrary to past work, where inputs were clamped until reservoir stabilization, it was found that it was possible to obtain similar classification accuracy (96.49%) by clamping the input patterns for just two repeats. The chief contribution of this work is that a recurrent architecture can get good classification accuracy, even while the reservoir is still in an unstable state.

Using Convolutional Neural Network for edge detection in musculoskeletal ultrasound images.

Fast and accurate segmentation of musculoskeletal ultrasound images is an on-going challenge. Two principal factors make this task difficult: firstly, the presence of speckle noise arising from the interference that accompanies all coherent imaging approaches; secondly, the sometimes subtle interaction between musculoskeletal components that leads to non-uniformity of the image intensity. Our work presents an investigation of the potential of Convolutional Neural Networks (CNNs) to address both of these problems. CNNs are an effective tool that has previously been used in image processing of several biomedical imaging modalities. However, there is little published material addressing the processing of musculoskeletal ultrasound images. In our work we explore the effectiveness of CNNs when trained to act as a pre-segmentation pixel classifier that determines whether a pixel is an edge or non-edge pixel. Our CNNs are trained using two different ground truth interpretations. The first one uses an automatic Canny edge detector to provide the ground truth image; the second ground truth was obtained using the same image marked-up by an expert anatomist. In this initial study the CNNs have been trained using half of the prepared data from one image, using the other half for testing; validation was also carried out using three unseen ultrasound images. CNN performance was assessed using Mathews Correlation Coefficient, Sensitivity, Specificity and Accuracy. The results show that CNN performance when using expert ground truth image is better than using Canny ground truth image. Our technique is promising and has the potential to speed-up the image processing pipeline using appropriately trained CNNs.

Structural health monitoring of a footbridge using Echo State Networks and NARMAX

Abstract Echo State Networks (ESNs) and a Nonlinear Auto-Regressive Moving Average model with eXogenous inputs (NARMAX) have been applied to multi-sensor time-series data arising from a test footbridge which has been subjected to multiple potentially damaging interventions. The aim of the work was to automatically classify known potentially damaging events, while also allowing engineers to observe and localise any long term damage trends. The techniques reported here used data from ten temperature sensors as inputs and were tasked with predicting the output signal from eight tilt sensors embedded at various points over the bridge. Initially, interventions were identified by both ESNs and NARMAX. In addition, training ESNs using data up to the first event, and determining the ESNs’ subsequent predictions, allowed inferences to be made not only about when and where the interventions occurred, but also the level of damage caused, without requiring any prior data pre-processing or extrapolation. Finally, ESNs were successfully used as classifiers to characterise various different types of intervention that had taken place.

Artificial Neural Network Analysis of Volatile Organic Compounds for the Detection of Lung Cancer

Lung cancer is a widespread disease and it is well understood that systematic, non-invasive and early detection of this progressive and life-threatening disorder is of vital importance for patient outcomes. In this work we present a convergence of familiar and less familiar artificial neural network techniques to help address this task. Our preliminary results demonstrate that improved, automated, early diagnosis of lung cancer based on the classification of volatile organic compounds detected in the exhaled gases of patients seems possible. Under strictly controlled conditions, using Selected Ion Flow Tube Mass Spectrometry (SIFT-MS), the naturally occurring concentrations of a range of volatile organic compounds in the exhaled gases of 20 lung cancer patients and 20 healthy individuals provided the dataset that has been analysed. We investigated the performance of several artificial neural network architectures, each with complementary pattern recognition properties, from the domains of supervised, unsupervised and recurrent neural networks. The neural networks were trained on a subset of the data, with their performance evaluated using unseen test data and classification accuracies ranging from 56% to 74% were obtained. In addition, there is promise that the topological ordering properties of the unsupervised networks’ clusters will be able to provide further diagnostic insights, for example into patients who may have been heavy smokers but so far have not presented with any lung cancer. With the collection of data from a larger number of subjects across a long time period there is promise that an automated assistive tool in the diagnosis of lung cancer via breath analysis could soon be possible.