Gennaro Esposito

A machine learning methodology for the selection and classification of spontaneous spinal cord dorsum potentials allows disclosure of structured (non-random) changes in neuronal connectivity induced by nociceptive stimulation.

Previous studies aimed to disclose the functional organization of the neuronal networks involved in the generation of the spontaneous cord dorsum potentials (CDPs) generated in the lumbosacral spinal segments used predetermined templates to select specific classes of spontaneous CDPs. Since this procedure was time consuming and required continuous supervision, it was limited to the analysis of two specific types of CDPs (negative CDPs and negative positive CDPs), thus excluding potentials that may reflect activation of other neuronal networks of presumed functional relevance. We now present a novel procedure based in machine learning that allows the efficient and unbiased selection of a variety of spontaneous CDPs with different shapes and amplitudes. The reliability and performance of the present method is evaluated by analyzing the effects on the probabilities of generation of different classes of spontaneous CDPs induced by the intradermic injection of small amounts of capsaicin in the anesthetized cat, a procedure known to induce a state of central sensitization leading to allodynia and hyperalgesia. The results obtained with the selection method presently described allowed detection of spontaneous CDPs with specific shapes and amplitudes that are assumed to represent the activation of functionally coupled sets of dorsal horn neurones that acquire different, structured configurations in response to nociceptive stimuli. These changes are considered as responses tending to adequate transmission of sensory information to specific functional requirements as part of homeostatic adjustments.

A Randomized Algorithm for the Exact Solution of Transductive Support Vector Machines

Random sampling is an efficient method for dealing with constrained optimization problems. In computational geometry, this method has been successfully applied, through Clarkson’s algorithm (Clarkson 1996), to solve a general class of problems called violator spaces. In machine learning, Transductive Support Vector Machines (TSVM) is a learning method used when only a small fraction of labeled data is available, which implies solving a nonconvex optimization problem. Several approximation methods have been proposed to solve it, but they usually find suboptimal solutions. However, a global optimal solution may be obtained by using exact techniques, but at the cost of suffering an exponential time complexity with respect to the number of instances. In this article, an interpretation of TSVM in terms of violator space is given. A randomized method is presented that extends the use of exact methods, thus reducing the time complexity exponentially w.r.t. the number of support vectors of the optimal solution instead of exponentially w.r.t. the number of instances.

Markovian Analysis of the Sequential Behavior of the Spontaneous Spinal Cord Dorsum Potentials Induced by Acute Nociceptive Stimulation in the Anesthetized Cat

In a previous study we developed a Machine Learning procedure for the automatic identification and classification of spontaneous cord dorsum potentials (CDPs). This study further supported the proposal that in the anesthetized cat, the spontaneous CDPs recorded from different lumbar spinal segments are generated by a distributed network of dorsal horn neurons with structured (non-random) patterns of functional connectivity and that these configurations can be changed to other non-random and stable configurations after the noceptive stimulation produced by the intradermic injection of capsaicin in the anesthetized cat. Here we present a study showing that the sequence of identified forms of the spontaneous CDPs follows a Markov chain of at least order one. That is, the system has memory in the sense that the spontaneous activation of dorsal horn neuronal ensembles producing the CDPs is not independent of the most recent activity. We used this markovian property to build a procedure to identify portions of signals as belonging to a specific functional state of connectivity among the neuronal networks involved in the generation of the CDPs. We have tested this procedure during acute nociceptive stimulation produced by the intradermic injection of capsaicin in intact as well as spinalized preparations. Altogether, our results indicate that CDP sequences cannot be generated by a renewal stochastic process. Moreover, it is possible to describe some functional features of activity in the cord dorsum by modeling the CDP sequences as generated by a Markov order one stochastic process. Finally, these Markov models make possible to determine the functional state which produced a CDP sequence. The proposed identification procedures appear to be useful for the analysis of the sequential behavior of the ongoing CDPs recorded from different spinal segments in response to a variety of experimental procedures including the changes produced by acute nociceptive stimulation. They are envisaged as a useful tool to examine alterations of the patterns of functional connectivity between dorsal horn neurons under normal and different pathological conditions, an issue of potential clinical concern.

Approximate policy iteration using regularised Bellman residuals minimisation

In this paper we present an approximate policy iteration (API) method called API‐BRMϵ using a very effective implementation of incremental support vector regression (SVR) to approximate the value function able to generalise in continuous (or large) space reinforcement learning (RL) problems. RL is a methodology able to solve complex and uncertain decision problems usually modelled as Markov decision problems. API-BRMϵ is formalised as a non-parametric regularisation problem based on an outcome of the Bellman residual minimisation (BRM) which is able to minimise the variance of the problem. API-BRMϵ is incremental and can be applied to RL using the on-line agent interaction framework. Based on non-parametric SVR API-BRMϵ is able to find the global solution of the problem with convergence guarantees to the optimal solution. A value function should be defined to find the optimal policy specifying the total reward that an agent might expect in its current state taking one action. Therefore, the agent will use the value function to choose the action to take. Some experimental evidence and performance for well-known RL benchmarks are presented.

Intersegmental Synchronization of Spontaneous Cord Dorsum Potentials as a Clinical Parameter to Evaluate Changes in Neuronal Connectivity Produced by Peripheral Nerve and Spinal Cord Damage

We describe here an automatic selection method to retrieve spontaneous cord dorsum potentials from the spinal cord in the anesthetized cat. Previous studies have indicated that some of these potentials appear synchronized in several spinal segments and are generated by the activation of specific sets of dorsal horn neurons. Since their synchronization is affected in a characteristic manner by acute peripheral nerve and spinal lesions, as well as during capsaicin-induced skin inflammation, they can be used to describe the patterns of functional interconnectivity between specific sets of dorsal horn neurons, which makes them of potential clinical interest.

Bellman residuals minimization using online support vector machines

In this paper we present and theoretically study an Approximate Policy Iteration (API) method called API − BRM𝜖 using a very effective implementation of incremental Support Vector Regression (SVR) to approximate the value function able to generalize Reinforcement Learning (RL) problems with continuous (or large) state space. API − BRM𝜖 is presented as a non-parametric regularization method based on an outcome of the Bellman Residual Minimization (BRM) able to minimize the variance of the problem. The proposed method can be cast as incremental and may be applied to the on-line agent interaction framework of RL. Being also based on SVR which are based on convex optimization, is able to find the global solution of the problem. API − BRM𝜖 using SVR can be seen as a regularization problem using 𝜖−insensitive loss. Compared to standard squared loss also used in regularization, this allows to naturally build a sparse solution for the approximation function. We extensively analyze the statistical properties of API − BRM𝜖 founding a bound which controls the performance loss of the algorithm under some assumptions on the kernel and assuming that the collected samples are not-i.i.d. following a β−mixing process. Some experimental evidence and performance for well known RL benchmarks are also presented.