A New Neuromorphic Computing Approach for Epileptic Seizure Prediction
AA New Neuromorphic Computing Approach for Epileptic Seizure Prediction
Fengshi Tian , Jie Yang , Shiqi Zhao , Mohamad Sawan , Fellow, IEEE CenBRAIN Lab., School of Engineering, Westlake University, Hangzhou, Zhejiang, China 310024 School of Microeletronics, Fudan University, Shanghai, China 200433 Email: [email protected]
Abstract β Several high specificity and sensitivity seizure prediction methods with convolutional neural networks (CNNs) are reported. However, CNNs are computationally expensive and power hungry. These inconveniences make CNN-based methods hard to be implemented on wearable devices. Motivated by the energy-efficient spiking neural networks (SNNs), a neuromorphic computing approach for seizure prediction is proposed in this work. This approach uses a designed gaussian random discrete encoder to generate spike sequences from the EEG samples and make predictions in a spiking convolutional neural network (Spiking-CNN) which combines the advantages of CNNs and SNNs. The experimental results show that the sensitivity, specificity and AUC can remain 95.1%, 99.2% and 0.912 respectively while the computation complexity is reduced by 98.58% compared to CNN, indicating that the proposed Spiking-CNN is hardware friendly and of high precision.
Keywords β Epileptic seizure prediction, EEG, Neuromorphic computing, Spiking convolutional neural network. I. I NTRODUCTION
Epilepsy is one of the most common neurological diseases worldwide. Up to 35% of around 60 million epileptic patients are not receiving effective medical treatment because of being drug-refractory [1], [2], [3]. Those patients may suffer from severe co-morbidities, injuries and anxiety due to sudden seizure onset [4]. Therefore, it is significant to propose a highly accurate and energy-efficient approach to predict seizure onsets. Electroencephalography (EEG) is an electrophysiological technique for the recording of the electrical activity of brains. Representing brain activity of epileptic patient, EEG is commonly used for epileptic seizure prediction [5]. Typical EEG recordings of an epileptic patient can be defined as four states: Interictal (between seizures), Preictal (before seizure), Ictal (seizure) and Post-ictal (after seizure). The prediction task is aimed to identify preictal state from the other three states [6]. In recent years, deep learning algorithms have been used for EEG signals analysis, where the most representative algorithm is convolutional neural network (CNN). Truong et al. used Short Time Fourier Transform (STFT) and CNN with 2D convolution to process the EEG signals [7]. Eberlein et al. processed raw EEG signals in time domain with a deep CNN to predict seizure onset [8]. Xu et al. used 1D convolution for seizure prediction and got AUC and sensitivity of 0.985 and 98.2% respectively [9]. However, all these methods are computationally expensive and memory consuming, which makes it hard to deploy these methods on at-edge hardware systems where computation and memory resources are limited. Some researchers have made efforts to explore hardware friendly approaches. Truong et al. used integer CNN and binary weights CNN for seizure detection, where energy consumption per task is reduced by over 90%
Fig. 1.
Flow diagram of the proposed approach. [10]. Zhao et al. designed a 1D Binary-CNN for seizure prediction which managed to reduce the parameter memory by 7.2 times and keep sensitivity at 0.94 [11]. However,
Truongβs work uses 4-bit integer weight and Zhao βs work uses fully precise value in the first convolution layer and the first full-connected layer, which wastes both energy and computation resources. Biologically inspired, Spiking NN (SNN), the most representative form of neuromorphic computing, proved energy efficient and powerful [12-14]. It is expected to be the next generation of Artificial NN (ANN) [15]. Guo et al. proposed a supervised SNN with a selected spike encoder to detect interictal spikes in epilepsy based on EEG and achieved an accuracy of 92.67% [16]. Therefore, SNN can be a possible candidate for energy efficient seizure prediction. However, SNNs always use complex training methods such as unsupervised spike-timing-dependent plasticity (STDP) [17], Tempotron [18] and SpikeProp [19]. These methods are still hard to build networks as deep as CNN [20], which results in limitations to the performance and application of SNN. Spiking CNNs (Spiking-CNNs) combine the advantages of both CNN and SNN to solve above described drawbacks [20]. Weights are trained in CNN fashion and subsequently mapped to the SNN topology correspondingly [21]. A spike encoder is also used to encode the input data into spike sequences, which is specifically designed for different kinds of data and tasks. In this way, Spiking-CNN manages to keep the performance and reduce the computation complexity. In this paper, we describe the implementation of Spiking-CNN and spike encoder used to make seizure predictions accurately and energy-efficiently. The proposed approach achieves sensitivity of 95.1% and specificity of 99.2% on raw EEG data while the computation complexity is reduced by 98.58% compared to the original CNN. The remaining sections of this paper are organized as following. Section II introduces the proposed method. Methods and materials are described in Section III. Results are shown in Section IV. Section V concludes this paper. II.
THEORY OF THE
APPROACH The proposed approach consists of spike encoder, network models and weight mapping (Fig. 1). To predict epileptic seizures based on raw EEG, a CNN is first designed and trained to get weights, and then transformed
Fig. 2. Comparison between origin EEG signals and the encoded spike sequences based on 10 time-step (TS) accumulation.
Fig. 3. Comparison among the encoded spike sequences based on different time-step accumulation. into Spiking-CNN. To process EEG on Spiking-CNN, a spike encoder is needed to transform raw EEG to spike sequences. When weights are mapped correspondingly, the Spiking-CNN operates inferring and make the prediction. A. Spike-Encoder
To convert the continuous EEG signal into discrete time-dependent spike sequences, a temporal Gaussian random sparse encoding algorithm is proposed. The encoder converts the 2D input vector into a 3D time-dependent vector. The sample of each time step is generated by the proposed Gaussian random discretization algorithm. After passing through the encoder, the features of the input vector are dispersed into each time step. For the proposed spike encoder, the time-step , V th-up and V th-down are determined parameters and the original data is the input EEG sample. To generate spike sequence in time domain, a random matrix is generated in every step of time, which is of the same size as the input sample and consists of gaussian random values (mean value is set to be the mean value of V th-up and V th-down ). Every element of the original data is then compared with the generated random value. If the random one is greater, the corresponding spike value will be set ; otherwise, the spike value will be set 1. Given an input vector in the shape of [Channel, Height, Width], the output shall be [Time Step, Channel, Height, Width] consisting of . The key benefits of encoding are: (a) to sparse the features of raw EEG in time domain so that it can be processed by Spiking-CNN and (b) to help reduce the length of processed values from 32-bit to 1-bit but still keep the (a) (b) (c) (d) Fig. 4.
Neuron Model: (a) CNN Neuron, (b) SNN IF Neuron, (c) inputs and outputs of an IF Neuron, (d) Procedure of integrating, fire and reset of IF neurons. main features of samples so that the goal of energy-efficiency can be achieved. changes, but later returns to V rest after firing the spike. Fig. 2 shows the comparison between the origin and spike of interictal and preictal samples given the time step of 10. As shown, the spike sequences succeed to keep most of the features of the samples. Taking an interictal sample for example, Fig. 3 shows that the larger the time step is, the more data features the encoded sample retains, which also consumes more computing resources. B. Neuron Model Selection
For Spiking-CNN, biology-inspired neuron models are proposed [29]. The spiking neurons can receive multiple inputs and be activated multiple times in the format of spike sequences, while the common deep learning neuron is only used once in one single computation. The function of time
V(t) and V rest are used to express the membrane and the resting potentials respectively. After the spike arrives, V(t) changes, but later returns to V rest after firing the spike. To propose a hardware friendly method, integrate and fire (IF) neuron model [29] is used in this study. The CNN neuron is shown in Fig. 4(a) while the IF neuron is shown in Fig. 4(b). For SNN IF neurons, the membrane potential is expressed with V i (t) and is updated at each time step as expressed in eq. (1) π π ( π‘ )= π π ( π‘β
1) +
πΌπππ’π‘ ( π‘ ) β πΏππππππ (1) For IF neurons, βΞ΄ j (t-t j ) describes the accumulation of input spike sequences (Fig. 4(c)) and the leakages are set to be ΞV as show in eq. (2) π π ( π‘ )= π π ( π‘β
1) + βπ€ ππ βπΏ π ( π‘βπ‘ π ) β βπ (2) As shown in Fig. 4(d), when V i (t) reaches the threshold V th , a spike will be fired at the output. Then the membrane potential V i (t) is reset to the resting potential V rest . C. Network Design
The proposed CNN model is first designed, then transferred to Spiking-CNN. For the EEG signal samples, they have thousands of elements in the time axis but only about twenty in the channel axis. Each channel records different regions of the brain, so mixing up the channel and (a) (b)
Fig. 5.
Proposed Neural Network : (a) CNN structure ( β Conv β is convolution, β MP β is max-pooling, and β C β stands for channel) and (b) Spiking-CNN structure; R, P and I stand for βResultβ , βPreictalβ and βInteric tal β respectively. Table 1. Comparison between CNN and Spiking-CNN. dimension may decrease the prediction performance. Therefore, the proposed network adopts single-dimension kernel for convolution and max-pooling operations. The single-dimension convolutional kernels extract features while the max-pooling kernels keep the most significant information. The structure of the CNN is shown in Fig. 5(a), which is adapted to achieve good performance. The proposed CNN consists of 5 convolution & max-pooling layers (Conv & MP) and 2 full-connected layers (FC Layer) plus an input and an output layer. There is a rectified linear unit (RELU) activation layer connected between each convolution layer and max-pooling layer. The corresponding structure of the Spiking-CNN is shown in Figs. 5(b). Except for the encoding layer and the spike counter (Count Layer), the transferred Spiking-CNN has the same structure as the CNN which demonstrates that Spiking-CNN uses the same feature extracting method as CNN. Spiking-CNN does not need RELU because the activation of IF neuron behaves the same as RELU [30]. The optimization lies in that all the data transferred between layers is single bit value ( or ), which transfers the former multiplication operations in CNN into adding operations. Therefore, much less computation resource is required for Spiking-CNN than CNN which makes it hardware friendly. Furthermore, the Spiking-CNN deals with data in the time domain which can be processed either sequentially or parallelly depending on the architecture Fig. 6. Typical EEG signals in time domain. of the computing unit. Comparison between CNN and Spiking-CNN is shown in Table 1. D. Procedure & Weight Mapping
The proposed Spiking-CNN has the same structure consisting of Conv-Layer and Max-Pooling as CNN and process data in time domain (Fig. 5). To process raw EEG data, the input samples need to be encoded into spike sequences as mentioned in Section II. Set based on the values of the recorded EEG data, V th-up and V th-down are hyperparameters of the spike encoder and are adjusted along with the chosen time step to achieve the best performance. At every time step, the corresponding spike data is processed in the network. Every time this procedure is completed, either or both or neither of P and I neuron is activated. The spike counter in Fig. 5(b) records the number of times of activation of the two output neurons through all the time steps and greater one tells the result of prediction. As for the weight mapping, we use SINABS [22] in this work. Due to the same structure of two networks, the trained CNN weights (32-bit floating numbers) of every layer can be restored and mapped on Spiking-CNN directly. To make it more convenient to implement the network on hardware, the resting potential of all the IF neurons are set zero. The threshold of each layer β s IF neurons is adjusted along with the V th-up and V th-down of the spike encoder to improve the performance. III. DATA
PROCESSING A. EEG Dataset
To evaluate the performance of proposed neural network, we used the CHB-MIT EEG dataset, which contains scalp EEG data of 23 measurements from 22 patients [23]. All measurements are recorded at 256 Hz sampling rate using different signal acquisition settings of electrodes among patients. 15 measurements are recorded in the same fixed 23-electrode implementation, while there are some changes in electrode configuration for the remaining measurements [24]. Fig. 6 shows the different EEG zones in time domain. When dealing with seizure prediction tasks, a short time between the end of Preictal Interval and Seizure (Ictal Interval) is defined as Seizure Prediction Horizon (SPH). Preictal Interval Length (PIL) and SPH are two empirical parameters which determines the chosen intervals for the experiments. 30 min PIL and 5min SPH are chosen in this work. Then two categorized samples are extracted from Interictal Intervals and Preictal Intervals as shown in Fig. 6
Fig. 7. Performance of accuracy. TS stands for time step.
Fig. 8. ROC curve and AUC. Table 2. ACC, TPR and FPR of different count thresholds. with a fixed 20-second time window. For one input sample, the width is standing for the selected data and the height is the number of recording channels, which is for CHB-MIT dataset. However, the raw EEG data contains much more Interictal Intervals than Preictal Intervals causing imbalanced training sample problem which may lead the trained model to perform bad [25], [26]. To overcome this limit, preictal samples are extracted with 5-second overlapping but interictal samples are done without overlapping [9]. B. Training and Inferring
The EEG dataset is separated into 2 parts randomly with the ratio of 4:1--training set and testing set. The training set is used to train the proposed CNN. The trained weights of every layer which will be mapped on the corresponding Spiking-CNN. The testing set is used to evaluate the performance of the proposed Spiking-CNN. When making the prediction, the EEG data first goes through the spike encoder and transforms to spike sequences. Then the Spiking-CNN mapped with trained weights receives the spike sequences and give the output of prediction. By comparing the output and the label, the performance of the proposed method can be evaluated. IV. R ESULTS
In this experiment, only lead seizures occurring at least 4h after previous seizures are considered [27], so there are 7 subjects in CHB-MIT dataset suitable for experiments. In this work, the performance is the mean value of every metric
Table 3. Comparison with other works on CHB-MIT dataset. of the 7 subjects.
The comparison of the maximum accuracy among the proposed original CNN and Spiking-CNN with different time steps is shown in Fig. 7. The time step of 10 is sufficient for the encoder to extract features from the original data for it has achieved an accuracy of over 90%. The computation complexity of CNN is shown in eq. (3) and the computation complexity of the proposed Spiking-CNN is object of eq. (4). T CNN = β M H M W (K H K W + K H + K W - 1)C in C out (1.1)float-bits (3) T SCNN = β M H M W (K H + K W - 1)C in C out / 10 time-step (4) where T, M, K, C, H, W stand for
Time Complexity, Feature Map, Kernel, Channel, Height and
Width . With time step chosen to be 10, the time complexity is reduced by 98.58% (including the reduction caused by the value transforming from float value to single bit value) compared with the original CNN with just a 2% loss of accuracy (Fig. 7). Accuracy (ACC), sensitivity (TPR/Sen), false prediction rate (FPR), receiver operating characteristic (ROC) and area under curve (AUC) are also evaluated in this study. With different count thresholds, the metrics are shown in Table 2 and Fig. 8. As shown, the mean sensitivity, FPR and AUC reach 95.1%, 0.0827 and 91.2% respectively. Table 3 shows comparison of this work with other works on metrics of FPR, sensitivity, AUC as well as the estimated number of ADD operation, number of MUL operation, memory and number of weights. The comparison of overall performance with other works is based on a figure-of-merit (FOM) whose definition is given in eq. (5)
FOM = (SEN + AUC - FPR) / [ 2(10MUL + ADD + Mem)] (5) V.
CONCLUSION In this study, we propose a neuromorphic approach of Spiking-CNN and all the relevant modules for energy-efficient epileptic seizure prediction. By designing a temporal Gaussian random sparse encoding algorithm, we propose a spike encoder to transform raw EEG to spike sequences which can be processed in Spiking-CNN. To keep high accuracy and reduce computation resource, we design and train the CNN first and then map the weights on the Spiking-CNN using SINABS [22]. As demonstrated by the results, the proposed approach not only achieves the sensitivity and AUC of 95.1% and 0.912 respectively, but also manages to reduce computation complexity by 98.58% with just a 2% loss of accuracy. A
CKNOWLEDGMENT
The authors would like to acknowledge the financial support and tools received from Westlake University and Zhejiang Key R&D Program No. 2021C03002 to support this project.
EFERENCES [1]
P. Kwan, A. Arzimanoglou, A. T. Berg, M. J. Brodie, W. Allen Hauser, G. Mathern, et al., "Definition of drug resistant epilepsy: consensus proposal by the ad hoc task force of the ilae commission on therapeutic strategies",
Epilepsia , vol. 51, no. 6, pp. 1069-1077, 2010. [2]
T. Granata, N. Marchi, E. Carlton, C. Ghosh, J. Gonzalez-Martinez, A. V. Alexopoulos, et al., "Management of the patient with medically refractory epilepsy",
Expert review of neurotherapeutics , vol. 9, no. 12, pp. 1791-1802, 2009. [3]
E. Bou Assi, D. K. Nguyen, S. Rihana and M. Sawan, "Refractory epilepsy: Localization detection and prediction", , pp. 512-515, 2017. [4]
R. J. Racine, "Modification of seizure activity by electrical stimulation: Ii. motor seizure",
Electroencephalography and clinical neurophysiology , vol. 32, no. 3, pp. 281-294, 1972. [5]
P. Mirowski, D. Madhavan, Y. LeCun and R. Kuzniecky, "Classification of patterns of eeg synchronization for seizure prediction",
Clinical neurophysiology , vol. 120, no. 11, pp. 1927-1940, 2009. [6]
R. Hussein, M. O. Ahmed, R. Ward, Z. J. Wang, L. Kuhlmann and Y. Guo, "Human intracranial eeg quantitative analysis and automatic feature learning for epileptic seizure prediction", arXiv preprint arXiv:1904.03603 , 2019. [7]
N. D. Truong, A. D. Nguyen, L. Kuhlmann, M. R. Bonyadi, J. Yang, S. Ippolito, et al., "Convolutional neural networks for seizure prediction using intracranial and scalp electroencephalogram",
Neural Networks , vol. 105, pp. 104-111, 2018. [8]
M. Eberlein, R. Hildebrand, R. Tetzlaff, N. Hoffmann, L. Kuhlmann, B. Brinkmann, et al., "Convolutional neural networks for epileptic seizure prediction", , pp. 2577-2582, 2018. [9]
Y. Xu, J. Yang, S. Zhao, H. Wu and M. Sawan, "An End-to-End Deep Learning Approach for Epileptic Seizure Prediction," , Genova, Italy, 2020, pp. 266-270. [10]
N. D. Truong, A. D. Nguyen, L. Kuhlmann, M. R. Bonyadi, J. Yang, S. Ippolito, et al., "Integer convolutional neural network for seizure detection",
IEEE Journal on Emerging and Selected Topics in Circuits and Systems , vol. 8, no. 4, pp. 849-857, 2018. [11]
S. Zhao, J. Yang, Y. Xu and M. Sawan, "Binary Single-Dimensional Convolutional Neural Network for Seizure Prediction," , Sevilla, 2020, pp. 1-5. [12]
K. Roy, A. Jaiswal, P. Panda, Towards spike-based machine intelligence with neuromorphic computing.
Nature β
617 (2019). [13]
G. Srinivasan, A. Sengupta, K. Roy. Magnetic Tunnel Junction Based Long-Term Short-Term Stochastic Synapse for a Spiking Neural Network with On-Chip STDP Learning.
Sci Rep N. K. Kasabov, NeuCube: A spiking neural network architecture for mapping, learning and understanding of spatio-temporal brain data,
Neural Networks , Volume 52, 2014, Pages 62-76, ISSN 0893-6080. [15]
S. Ghosh-dastidar, H. Adeli, Spiking Neural Networks,
International Journal of Neural Systems
L. Guo, Z. Wang, M. Cabrerizo and M. Adjouadi, A cross-correlated delay shift supervised learning method for spiking neurons with application to interictal spike detection in epilepsy,
Int. J. Neural Syst.
D. Pu and M. Cook (2015) Unsupervised learning of digit recognition using spike-timing-dependent plasticity.
Front. Comput. Neurosci . 9:99. doi: 10.3389/fncom.2015.00099 [18]
L. R. Iyer and Y. Chua, "Classifying Neuromorphic Datasets with Tempotron and Spike Timing Dependent Plasticity," , Glasgow, United Kingdom, 2020, pp. 1-8, doi: 10.1109/IJCNN48605.2020.9207474. [19]
S. B. Shrestha and Q. Song, "Robustness to Training Disturbances in SpikeProp Learning," in
IEEE Transactions on Neural Networks and Learning Systems , vol. 29, no. 7, pp. 3126-3139, July 2018, doi: 10.1109/TNNLS.2017.2713125. [20]
A. Tavanaei, M. Ghodrati, S.R. Kheradpisheh, T. Masquelier, A. Maida, Deep learning in spiking neural networks,
Neural Networks , Volume 111, 2019, Pages 47-63, ISSN 0893-6080. D [21] B Rueckauer, I-A Lungu, Y
Hu, M Pfeiffer and S-C Liu (2017) Conversion of Continuous-Valued Deep Networks to Efficient Event-Driven Networks for Image Classification.
Front. Neurosci . 11:682. doi: 10.3389/fnins.2017.00682 [22] https://gitlab.com/aiCTX/sinabs. [23]
A. L. Goldberger, L.A. Amaral, L. Glass, J.M. Hausdorff, P.C. Ivanov, R.G. Mark, et al., "Physiobank physiotoolkit and physionet: components of a new research resource for complex physiologic signals",
Circulation , vol. 101, no. 23, pp. e215-e220, 2000. [24]
A. H. Shoeb, "Application of machine learning to epileptic seizure onset detection and treatment",
Ph.D. dissertation Massachusetts Institute of Technology , 2009. [25]
R. Barandela, R. M. Valdovinos, J. S. SΓ‘nchez and F. J. Ferri, "The imbalanced training sample problem: Under or over sampling?",
Joint IAPR international workshops on statistical techniques in pattern recognition (SPR) and structural and syntactic pattern recognition (SSPR) , pp. 806-814, 2004. [26]
N. Japkowicz and S. Stephen, "The class imbalance problem: A systematic study",
Intelligent data analysis , vol. 6, no. 5, pp. 429-449, 2002. [27]
R. Hussein, M. O. Ahmed, R. Ward, Z. J. Wang, L. Kuhlmann and Y. Guo, "Human intracranial eeg quantitative analysis and automatic feature learning for epileptic seizure prediction", arXiv preprint arXiv:1904.03603 , 2019. [28]
Y. Zhang, Y. Guo, P. Yang, W. Chen and B. Lo, "Epilepsy Seizure Prediction on EEG Using Common Spatial Pattern and Convolutional Neural Network," in
IEEE Journal of Biomedical and Health Informatics , vol. 24, no. 2, pp. 465-474, Feb. 2020. [29]
A. N. Burkitt, A Review of the Integrate-and-fire Neuron Model: I. Homogeneous Synaptic Input.
Biol Cybern β
19 (2006). [30]
Y. Cao, Y. Chen, & D. Khosla, Spiking Deep Convolutional Neural Networks for Energy-Efficient Object Recognition.
Int J Comput Vis β
66 (2015).66 (2015).