New Approach for Designing cVEP BCI Stimuli Based on Superposition of Edge Responses
NNew Approach for Designing cVEP BCI StimuliBased on Superposition of Edge Responses
Muhammad Nabi Yasinzai and Yusuf Ziya Ider Department of Electrical and Electronics Engineering, Bilkent University, AnkaraTurkeyE-mail: [email protected], [email protected] February 2020
Abstract.
The purpose of this study is to develop a new methodology for designingstimulus sequences for cVEP BCI based on experimental studies regarding the behaviorand the properties of the actual EEG responses of the visual system to coded visualstimuli, such that training time is reduced and possible number of targets is increased.EEG from 8 occipital sites is recorded with 2000 samples/sec per channel, in response tovisual stimuli presented on a computer monitor with 60Hz refresh rate. Onset and offsetEEG responses to long visual stimulus pulses are obtained through 160-trial signalaveraging. These edge responses are used to predict the EEG responses to arbitrarystimulus sequences using the superposition principle. A BCI speller which utilizes thetarget templates generated by this principle is also implemented and tested. It is foundthat, certain short stimulus patterns can be accurately predicted by the superpositionprinciple. BCI sequences that are constructed by combinations of these optimalpatterns yield higher accuracy (95.9%) and ITR (57.2 bpm) compared to when thesuperposition principle is applied to conventional m-sequences and randomly generatedsequences. Training time for the BCI application involves only the acquisition of theedge responses and is less than 4 minutes, and a huge number of sequences is possible.This is the first study in which cVEP BCI sequences are designed based on constraintsobtained by observing the actual brain responses to several stimulus patterns.
Keywords : Brain Computer Interface, Coded Visually Evoked Response, EEG, StimulusDesign, BCI Speller
1. Introduction
BCI based on visual evoked potentials (VEPs), which were first proposed in 1984 asa new communication channel [1] [2], are the most common type of EEG based BCIs.In 1992, code modulated VEP (cVEP) was introduced and 64-bit long binary stimuluspatterns (known as m-sequences) are employed for the task of target classification [3].A base m-sequence is assigned to one target (base target), and the sequences for theremaining targets are circularly shifted versions of the base sequence. A signal-averaged a r X i v : . [ q - b i o . N C ] A p r EEG response template is recorded for the base target, and the templates for the restof the targets are generated by circular shifting of the recorded EEG template. InBCI application EEG is recorded while the subject gazes at a certain target. Therecorded EEG is correlated with the templates of all targets, and the target that yieldsmaximum correlation is accepted. In m-sequence based cVEP, only a limited number ofshifts is possible based on the length of the stimulus pattern. For instance, a 63-bit longm-sequence is widely used for cVEP based BCI spellers (Visual Keyboard). In thesespellers, the m-sequence is circularly shifted by 2-bits for each target and therefore, theycan support a maximum of 32 targets. To increase the number of targets, it is necessaryto either increase the length of the sequence or decrease the number of bits for shifting,which will decrease the speed or accuracy of the BCI speller, respectively. Anotherviable solution is to combine multiple targets into a single group and use a specificm-sequence for each group [4]. However, this method is limited since there is a smallnumber of acceptable m-sequences for a specific sequence length [4] [2] and training isneed for each of the m-sequences.Training time is also an important factor to consider when assessing BCIs. M-sequence has the advantage of less training time because different templates are obtainedby shifting. Even though shifted m-sequences may themselves be mutually orthogonal,their EEG responses may not be as orthogonal [5]. To circumvent this problem, onemay consider different codes for each target which are not shifted versions of each other.This choice however increases the training time. Therefore, it is necessary to be ableto obtain templates for different targets from a small set of training data. One notablestudy in these lines is the work done by Nagel et. al. [2], who have investigated the useof arbitrary code sequences for cVEP BCI. A moving average (MA) model is proposedin his study to predict brain responses to arbitrary stimuli patterns. Some training dataare used to estimate the coefficients of the MA model and using this model all targettemplates are predicted. The accuracy and Information Transfer Rate (ITR) values ofthe BCI system which uses these templates are reported to be high.Nagel et. al. work is based to a large extent on the linearity of the system.However, it is known from modeling studies [6] that the visual system is nonlinear. The13 parameter Robinsons model has 4 sigmoid shaped nonlinear blocks in addition to gainand linear filter blocks [6]. Therefore the use of linear models have to be handled withcaution. The fact that the linear approach employed by Nagel et. al. has neverthelesssome significant success calls for investigating the nature of the visual system withrespect to the linear vs nonlinear perspective. In particular, it is important to investigateunder what circumstances the system obeys the superposition and shift-invarianceproperties. If such properties are indeed satisfied, at least to some degree, then onemay predict templates for many stimuli patterns using superposition of responses tosimple inputs, which has the advantage of shorter training time.BCI research is mostly application-specific, where the aim is to come up with a fasterand more reliable means of communication interface. Majority of the BCI systems aredesigned without considering the actual nature of the brain responses. Even some ofthe new studies which have focused on modeling the brain responses use random stimulipatterns along with some generic models to approximate EEG responses to them [2] [7].Therefore, the concept of designing a BCI system based on the nature and propertiesof the brain responses to visual stimuli is still untouched.In this paper, we investigate the possibility of using the superposition principle forpredicting the brain responses to different stimuli patterns. Initially, the EEG responsesfor simple stimulus patterns are acquired, which are then superimposed to generate theEEG responses to more complex stimuli patterns. Since we have found that the brainresponds to the edges of the code sequence, as shown in the results section, we havepaid attention to obtaining the edge responses correctly using averaging. In fact, we callthis phase as the training phase. We then performed several studies, using more generalstimulus sequences, to find the similarity between the generated (simulated) EEG signalsand the measured (observed) EEG signals. It was observed that the generated andobserved EEG had fairly high correlation, provided that some of the constraints on thestructure of the stimuli sequences are met. Later in this paper, a BCI system based onthe superposition property is proposed, implemented, and tested.
2. Materials and Methods
Seven participants (7 male, mean age of 29.7 years, and standard deviation of 5.5 years)participated in the experiment after providing a written consent form approved by theethical committee of Bilkent University. They had normal or corrected-to-normal vision,had no previous history of epilepsy, and they were informed about the objective of thisstudy.
The visual speller screen is designed in Matlab environment using Psychtoolbox [8] andis presented on a 25-inch LED (Dell Alienware AW2518HF) at 60 Hz refresh rate witha resolution of 1920 x 1080 pixels. Subjects were seated in front of the monitor screenat a distance of about 80 cm. The visual speller consists of 36 targets, arranged as a 6 x6 matrix on the screen (Fig. 1). Each target has a rectangular shape of 180 x 90 pixels(5.18cm x 2.6cm) with the respective letter/number written at its center. The targetsinclude letters from A to Z, numbers from 1 to 9 and a - symbol. The targets are toggledto black or white when the corresponding stimulus code is 0 and 1, respectively. Eachbit of the stimulus sequence is presented for 16.667ms (display time of a single frame).Ubuntu 18.04 with low latency kernel was used in the stimulus computer for havingaccurate stimulus timings. Close attention was given to the Psychtoolboxs missed-frames counter to make sure that no frames were dropped during an experiment. APIN photodiode circuit, explained in detail in our previous study [9], was used formeasurement of the actual refresh rate of the monitor, and it was found to be 59.94 Hz.Figure 1: Visual Speller Screen for the BCI experiments.For the purpose of synchronization, a marker pulse is transmitted from the stimuluscomputer to the EEG amplifier after the update of each monitor frame to keep trackof the flashing time for each bit. We also measured the exact delay between the frameupdate and the marker pulse. A negligible delay of 280 µs was found [10]. Brain Products, V-Amp-16 EEG amplifier is used for recording EEG, with the standard10-20 EEG cap which has 32 electrode locations (Brain Products, Gilching, Germany).V-Amp-16 contains a total of 16 EEG amplifier channels, however, in our experimentsEEG is recorded from 8 channels of the amplifier with electrodes placed at positions O1,Oz, O2, Pz, P3, P4, P7 and P8. The reference electrode is placed on FCz position, andthe ground electrode is located on the forehead above nasion. Active/wet electrodesare used and the electrode impedances are measured using ImpBox (Brain Products,Gilching, Germany) and are kept below 10kΩ. The sampling rate of the EEG amplifier isset to 2000 sps per channel. BCI2000 [11] with FieldTrip [12] are used for recording EEGresponses along with the synchronization markers, to another computer (the ”recordingcomputer”) using MATLAB session in real time.
A 4-40 Hz bandpass filter is applied to all of the EEG signals along with a 50 Hz notchfilter to remove any of the remaining 50 Hz interference. EEG responses of the stimulisequences are averaged over the trials with the help of the synchronization markers.Furthermore, the signals are then spatially averaged using the coefficients determinedby canonical correlation analysis (CCA) [13]. CCA helps to increase the signal to noiseratio and reduces the 8-channel data into a single signal.Figure 2: Wide pulse stimulus with randomized duration of the high and low regions.
The experiments are divided into three types. In the first type, EEG responses towide stimuli pulses are acquired. In the second type of experiments, EEG responses to9 different stimulus sequences with simple (short) patterns are obtained. In the finalexperiment type, various long BCI sequences are studied, and a BCI speller is realizedand tested. The stimulus pattern for the first experiment type is given in Fig. 2.Each time this stimulus pattern is repeated, the lengths of the high and low regionsare randomly chosen between 500 ms and 750 ms to remove any artifacts that mayarise due to the otherwise periodic nature of the EEG. The stimulus pattern in Fig.2 is assigned to target A and repeated 160 times for averaging. The other targets areassigned randomly generated sequences and the subject is asked to gaze at target A.For the second type of experiments, the stimulus sequences shown in Fig. 3 areused to carry out the following studies; Pulse Width (PW) study, Pulse Separation (PS)study, and Pulse Repetition (PR) study. In the PW study, stimulus sequences PW15and PW69 are used, which cover pulse widths of 1-5 and 6-9 bits, respectively. In Fig.3, the widths of each part of the stimulus sequences are written below in units of bits,and they add up to 120 bits. The PS study is divided into 2 parts. In the first part, theseparation between two 1-bit pulses is changed between 1 to 5 bits (sequence PS15W1)and between 6 to 9 bits (sequence PS69W1). In the second part, the separation betweentwo 2-bit pulses is changed between 1 to 5 bits (sequence PS15W2) and between 6 to 9bits (sequence PS69W2). Finally, in the PR study, different repetitions of 1-bit, 2-bit,and 3-bit wide pulses are studied. In this study, the distance between the pulses isequal to the width of the pulses (pulses are periodically repeated with 50% duty cycle).The stimulus sequences PR36W1, PR35W2 and PR24W3 cover 3-6 repetitions of 1-bitwide pulses, 3-5 repetitions of 2-bit wide pulses, and 2-4 repetitions of 3-bit wide pulses,respectively. It should be noted that PR36W1 and PR35W2 sequences do not includethe case of 2 repetitions of 1-bit and 2-bit pulses because they are already covered inthe PS study.The solid lines in Fig. 3 are the patterns under consideration, whereas the dottedlines are the separations between these patterns. Separations of 17-26 bits (283 ms - 433ms) are introduced, so that the EEG responses to the individual patterns do not overlap.For each of the 9 stimulus sequences, EEG is recorded for 30 trials for signal averaging.These 9 stimuli sequences are assigned to target A, and the subject is asked to gaze attarget A while the remaining targets are assigned randomly generated sequences.Figure 3: 9 different stimuli sequences designed to investigate the nature of EEGresponses to simple visual stimulus patterns.In the last type of experiments, we studied 5 different 120-bit long sequences whichare potentially to be used in BCI experiments (Fig. 4). These sequences are named asRandomly Generated (RG) Sequence, Pulse Position Modulated (PPM) Sequence, m-sequence, 7-in-15 Change Random (7-in-15CR) Sequence and Superposition OptimizedPulse (SOP) Sequence. RG Sequence is obtained by assigning 1 or 0 to every nextbit position with 50% probability. The PPM sequence contains 1-bit pulses which areseparated randomly by a distance of 1 to 4 bits. The 120-bit m-sequence is actuallya truncated version of a 127-bit m-sequence, discussed in our previous paper [5]. The7-in-15CR sequence contains 7 changes for every 15 bits of the sequence. This codewas proposed by Nagel et. al. [2] and was reported to have a better performancethan randomly generated codes. Finally, the SOP sequence is proposed by us in theresult section. To generate such a code we first selected 16 different small sequencepatterns. They include 1-bit pulse followed by 5-10 zero bits, 2-bit pulse followed by5-10 zero bits, 3 and 4 repetitions of 2-bit pulses followed by 5 zero bits and finally, 3and 4 repetitions of 3-bit pulses followed by 5 zero bits. Hence, there are 16 differentpatterns to choose from, and an SOP sequence is generated by randomly picking upthese patterns, concatenating them, and truncating the final pattern to 120 bits.
The training phase of our BCI speller involves obtaining the average EEG responses towide pulses shown in Fig. 2. These responses are then used to predict the templateEEG responses for all targets using the superposition based procedure, as explained inthe results section. During the test phase, the subjects are asked to spell 35 targets insequence from A-Z and 1-9, respectively. The EEG responses are recorded for 2 trialsFigure 4: Different Types of 120-bit stimulus sequencesand are averaged. During testing, the acquired EEG is correlated with the generatedEEGs (templates) of all targets, and the target which has the maximum correlation isselected.
IT R = 60 T × ( log N + P log P + (1 − P ) log − PN −
3. Results
Fig. 5 shows that the 160-repetition averaged EEG responses of each of the 7 subjectsstart 50 ms after either the positive edge (onset) or the negative edge (offset) of thestimulus and wane within 350 ms. It can be observed that each subject has his owndistinct onset and offset responses, but they do not deviate significantly from theaveraged responses of all 7 subjects. Fig. 5 also shows the edge responses acquired2 weeks later, indicating that onset and offset responses are repeatable. The onsetresponses acquired from the two acquisitions have correlations of 0.90, 0.82, 0.93,0.86, 0.81, 0.61, and 0.96 for subjects 1-7, respectively. For the offset responses, thecorresponding correlations are 0.79, 0.80, 0.44, 0.82, 0.04, 0.67 and 0.87. The relativelylower correlation values for the offset responses are due to lower signal to noise ratiobecause of the low amplitudes of the offset responses. The average RMS value of theFigure 5: Onset and Offset responses are on the Left and Right, respectively. Dotted-blue lines are the edge responses obtained 2 week after the first set of responses shownin red.onset responses for acquisitions 1 and 2 are 0.303 and 0.289, respectively, whereas thecorresponding average RMS values of offset responses are 0.149 and 0.160.
If the system is linear and if the system responds to the edges only, then it should bepossible to predict the EEG response to a general stimulus pattern by superposition,that is, by shifting the onset and offset response to the positions of the correspondingedges of the stimulus sequence and adding them. In the following, we investigate to whatextent the superposition principle is valid. Correlation between the predicted and theactual responses are then used to evaluate the performance of the superposition-basedprocedure first for the different simple pulse patterns provided in Fig. 4, and then alsofor long BCI sequences.
In this PW study, theEEG responses for different pulse widths are acquired using the stimulus patterns PW15Figure 6: Subject 1, Acquired (Red-Solid) and Generated (Dotted-Blue) EEG responsesfor 1-9 bit wide pulses.and PW69. The observed and generated (predicted) EEG responses for different pulsewidths are given in the Fig. 6 for Subject 1, and it is observed that as the pulse width isincreased from 1 to 9 bits the correlation falls from 0.63 to 0.32. The average correlations,over all 7 subjects, between the generated and the observed EEGs for different stimuluspulse widths are illustrated in Fig. 10a. For PWs of 1 and 2 bits the correlations arearound 0.7 and are lower for the other PWs. Repeated measures ANOVA test showsthat there is significant difference among the correlations obtained for different PWs(p < < In this PS study, theeffect of time separation between adjacent pulses is studied. The pulses of interestare either 1-bit wide or 2-bit wide pulses. First, the separation distance between twoadjacent 1-bit pulses is changed from 1 to 9 bits and the observed signals for subject1 are given in Fig. 7. It is found that as the separation is increased from 1 bit to 9bits the correlation between the measured and predicted (generated) EEG responsesincreases from 0.36 (for 1-bit separation) to a maximum of 0.72 (for 6-bit separation )and then gradually falls to 0.47 (for 9-bit separation). The average correlations, overall 7 subjects, between the generated and the observed EEGs for different separationsof 1-bit wide pulses are illustrated in Fig. 10b. On the average, correlation is around0.4 for separations of 1-3 bits and for higher separations the correlations are around 0.6.Therefore, in general for 1-bit wide pulses if the separation between neighboring pulses0Figure 7: Subject 1, Acquired (Red-Solid) and Generated (Dotted-Blue) EEG responsesfor stimulus patterns of 1-9 bit separation between 1-bit wide pulses.is 4 bits or more, then the EEG can be predicted with high accuracy, which is logicalbecause then the overlap between the responses of the individual pulses becomes less.Repeated measures ANOVA test shows that there is a significant difference among thecorrelations obtained for different pulse separations (p = 0.013). Furthermore, pairwisecomparisons using paired t-test shows that the correlations for 4-9 bit separations arestatistically not different, whereas correlations for 1-3 bit separations are significantlydifferent from column 6. Therefore, the choice of 4 to 9 bit separations between 1-bitwide pulses seems reasonable.Similarly for 2-bit wide pulses, the same pattern is observed. The generated andobserved EEG plots for different pulse separations for Subject 1 are provided in Fig. 8.The average correlations, over all 7 subjects, between the generated and the observedEEGs for different separations of 2-bit wide pulses are illustrated in Fig. 10c. Theaverage correlation increases from 0.32 to 0.65 as separation is increased from 1 to 3bits. From 3-bit up to 9-bit separations small variation in the correlations are observed.Hence, it can be inferred that if the separation between the 2-bit wide pulses is greaterthan or equal to 3 bits, the responses of the individual pulses will have less overlap andthe generated sequence response will have a high correlation with the observed EEGresponse. Repeated measures ANOVA test shows that the null hypothesis of havingequal means for different separations is to be rejected (p = 0.001). Furthermore, pairwisecomparisons using paired t-test shows that the mean correlations for 3-9 bit separationsare similar to each other and the mean correlations for 1-bit and 2-bit separations aresignificantly different from 3, 4 and 9 bit separations. Therefore in general choice of3-bit to 9-bit separation between 2-bit pulses seems reasonable.
In this PR study, thegenerated and measured EEG are compared for different number of repetitions of 1-bit,2-bit, and 3-bit wide pulses. The stimulus sequences used for this study are PR36W1,PR35W2 and PR24W3 for 1, 2 and 3-bit pulses, respectively, as shown in Fig. 3. The1Figure 8: Subject 1, Acquired (Red-Solid) and Generated (Dotted-Blue) EEG responsesfor stimulus patterns of 1-9 bit separation between 2-bit wide pulses.Figure 9: Subject 1, Acquired (Red-Solid) and Generated (Dotted-Blue) EEG for pulserepetitions of 1-3 bit wide pulses.results of this study for subject 1 is shown in Fig. 9. For 1-bit pulse, the correlation for 3repetitions is 0.34, which then drops to 0.15 as the number of repetitions is increased to6. This is because of the destructive interaction of the individual edge responses, whichcan also be observed in Fig. 9 by the low amplitude of the acquired EEG. Therefore,it appears that repetitions of 1-bit pulses are not to be preferred. For 2 repetitions of2-bit pulses the correlation is less than 0.5 (as seen From Fig. 8). However for 3, 4and 5 repetitions, the correlations are between 0.67 and 0.77. For 3-bit wide pulses, thecorrelation for 2, 3 and 4 repetitions is between 0.56 and 0.66. Therefore, 3-5 and 2-4repetitions of 2-bit and 3-bit wide pulses give better accuracy results, respectively.The average correlations over all 7 subjects and their confidence intervals fordifferent repetitions of 1-bit, 2-bit and 3-bit wide pulses are illustrated in Fig. 10d, 10eand 10f, respectively. Overall, the results for all subjects are in parallel to the results2Figure 10: Average Correlation values with 95% confidence interval. (a) For 1-9 PulseWidths, (b) For 1-9 Separations between 1-bit wide pulses, (c) For 1-9 Separationsbetween 2-bit wide pulses, (d) For 2-6 Repetitions of 1-bit wide pulse, (e) For 2-5Repetitions of 2-bit wide pulse, (f) For 2-5 Repetitions of 3-bit wide pulse,we have explained for subject 1. Repeated measures ANOVA indicates that significantdifference exists between different repetitions of 1-bit pulses (p = 0.03) but pairwisepaired t-tests do not show any significant difference between different repetitions (at5% significance level) and also the average correlation values for all repetitions are verylow (less than 0.411). Therefore, repetition of 1-bit pulses is not preferred. On theother hand, the correlation for 2-bit wide pulses shows much higher values and thisphenomenon in observed in almost all of the subjects. Repeated measures ANOVAindicates that correlations for different repetition are significantly different (p < In this study we tested,for subject 1, the 5 different stimulus sequences which are provided in Fig. 4 of sectionII.E. All of these sequences are 120 bits long, they are repeated for 60 trials to get a3Figure 11: Generated and Observed EEG responses for a) RG Sequence, b) PPMSequence, c) M-Sequences, d) 7-in-15CR Sequences, and e) SOP.good averaged signal, and the results are shown in Fig. 11. The correlation betweenthe generated and actual EEG responses for the RG sequence, PPM sequence, m-sequence, 7-in-15CR sequence, and the SOP sequence are 0.41, 0.23, 0.46, 0.55, and0.79, respectively. The EEG response to the PPM sequence is difficult to predict becausethe separations between 1-bit pulses are very small. The correlation values for the m-sequence and the RG sequence are also low, probably because of the same reason. The7-in-15CR sequence performed better than the RG, PPM and m-sequences. Finally,our proposed SOP sequence performed the best, and a high correlation was recordedbetween the generated and observed EEGs.In the following, a BCI application is proposed, which uses target code sequenceswhich are designed by taking the above results into consideration. In fact, the ”atomicpulse waveforms” which are used to design the proposed SOP stimulus sequences, whichare explained in section II.E (Stimulus Sequences) are decided upon as a result of theaforementioned observations. Acquiring BCI results for all of the 5 sequence typesshown in Fig. 4 would be very demanding for the subjects, and therefore we decidedto compare the least performing PPM sequence, the moderately performing 7-in-15CRsequence, and the best performing SOP sequence.
Table 1 provides the accuracy and ITR values of the BCI application using the threestimulus types for each subject. For all of the subjects, the PPM sequences performedthe worst with an overall accuracy and ITR of 6.94% and 1.7 bits/min, respectively. Theperformance of the 7-in-15CR sequences is comparatively better than the PPM sequence4Table 1: BCI Application Results
PPM 7-in15CR SOPSubNo Acc(% ) ITR(bits/min) Acc(% ) ITR(bits/min) Acc(% ) ITR(bits/min)01 Avg 6.94 1.7 27.75 10.53 95.9 57.19 with an average accuracy and ITR of 27.75% and 10.53 bits/min, respectively. Finally, asexpected, the SOP sequences performed the best with an accuracy of 95.9% and ITR of57.19%, respectively. It is worth repeating that the PPM sequences performed the worstbecause in these sequence only 1-bit wide pulses are used and the separation betweenthe pulses are randomly chosen between 1 and 4 bits. As we have shown in sectionPrediction Performance for different Pulse Separations, the pulse separation should begreater than 3 bits for better prediction accuracy. Also in the 7-in-15CR sequences thereare many instances when pulse separations are short, and this may explain why thesesequences also do not perform well. In general we may conclude, although from thecomparison results of just 3 sequence types, that sequences which do not obey the rulesthat we have identified in the previous sections, perform poorly.
4. Discussion
The human brain is a highly nonlinear and complex system, and yet, most of thecVEP based applications are designed without considering the characteristics of thebrain responses. In m-sequence based cVEP BCI applications, the assumption ofshift-invariance seems to hold. Similarly, the BCI application based on an MA model[13] also performed quite well. Therefore, it can be inferred from these studies thatmodeling or just making some assumptions on the characteristics of the system may beuseful in designing BCI applications. In our study, we have gone one step further tounderstand the system better and have investigated how well the brain complies withthe superposition property. This is the first study to investigate the nature of brainresponses to different visual stimulus patterns to verify that the brain responses followsuperposition under certain constraints on the stimulus patterns. The observations arethen used for designing optimum sequences for BCI.The constraints which we are suggesting for BCI stimulus sequences decrease the5number of possible targets. However, it is still a huge set, and a large number oftargets can be introduced. Furthermore, we have performed experiments on only a fewsimple stimulus patterns, and the patterns that performed well among them are used indesigning the long stimulus sequences for our BCI application. Further studies can becarried out in order to identify additional predictable simple patterns, and they can beadded to the set of acceptable simple patterns to increase the number of targets.The positive and negative edges of the stimulus sequence are the main factorsinfluencing the EEG response. These positive and negative responses have an initialdelay of 50 ms and diminish 350 ms after the edge. These responses appear to besimilar to the step response of a low order, such as 2nd or 3rd order, linear system.Hence, further studies can be carried out to obtain a model of the system by estimatingthe parameters of such a system from the edge responses.The edge responses of the seven subjects in our study have a common pattern, itis worth investigating if a universal edge response can be used for predicting responsesto BCI target sequences. Using a universal edge response would have the advantage ofeliminating the training stage for acquiring the edge responses of an individual subject.However, at this stage, until further studies are undertaken, we suggest that a trainingsession should be carried out for each individual to acquire his/her edge responses.The results that we have obtained not only serve for better design of BCIexperiments but also shine light on the workings of the visual system regarding itslinearity and shift-invariance properties. We hope that our results will give insight toresearchers who deal with the fundamental aspects of the visual system in addition toinvestigators who undertake application-oriented research such as BCIs.
5. Conclusion
In this study, responses of the visual system to several code patterns are first studiedin order to come up with rules (constraints) for constructing beneficial sequences forBCI applications. It is first found that the onset and offset responses of the brain tovisual stimulus pulses are delayed by about 50 ms and wane within 350 ms. Theseedge responses are then used to predict EEG responses to any stimulus sequences byusing superposition. It is found that 1-bit and 2-bit wide pulses with 4-9 bit and 3-9 bit separations respectively, and 2-bit and 3-bit pulses with 3-4 and 2-4 repetitionsrespectively, can be predicted with good accuracy. These simple patterns were randomlycombined in 120-bit stimulus sequences to be used in a BCI speller application. It isconfirmed that with such target stimulus sequences, the BCI application will give betterclassification results with 95.9% accuracy. Furthermore, the BCI application proposedin our study has short training time because the training is carried out only to acquirethe edge responses, and the proposed methodology allows for a large number of possibletargets. The results of our study indicate that although the visual system is knownto be nonlinear; nevertheless, based on some simple constraints on the structure ofthe stimulus sequences, a linear operation like superposition can be used in a BCI6application.
References [1] Sutter E E 1984 The visual evoked response as a communication channel
Proceedings of the IEEESymposium on Biosensors pp 95–100[2] Nagel S and Sp¨uler M 2018
PloS one e0206107[3] Sutter E E 1992 Journal of Microcomputer Applications IEEE Transactions on NeuralSystems and Rehabilitation Engineering Biomedical Physics & Engineering Express Physical Review E bioRxiv Spatial vision Biomedical Physics & Engineering Express Hybrid and model based approaches for new BCI spellers
Ph.D. thesis BilkentUniversity[11] Schalk G, McFarland D J, Hinterberger T, Birbaumer N and Wolpaw J R 2004
IEEE Transactionson biomedical engineering Computational intelligence andneuroscience
IEEE Transactions on Neural Systems andRehabilitation Engineering UPPLEMENTARY MATERIAL
S1. Illustration of the idea of using superposition for predicting cVEPresponses
The process of predicting EEG response to a stimulus pattern employs shifting of theedge responses to the corresponding location of these edges in the pattern and thenadding these shifted edge responses. This superposition-based procedure is illustratedin Figure S1.Figure S1: The blue waveform is the stimulus pattern, and it consists of 2 pulses only.The green and red signals are the onset and offset responses that are shifted accordingto the location of the corresponding edges in the stimulus sequence. These shifted edgeresponses are added to get the predicted EEG (orange signal).
S2. Prediction Performance for different short Pulse Widths
In this PW study, the EEG responses for different pulse widths are acquired using thestimulus patterns PW15 and PW69. The correlation values for each of the 7 subjectsfor is given in Table S1. For all subjects except subject 5, the maximum correlationis obtained either for 1-bit or 2-bit pulse widths. In general, it can be noted that asthe pulse width is increased from 1-bit the correlation between the generated EEG and8observed EEG decreases and reaches a minimum correlation of 0.35 for 5-bit wide pulseand then increase slowly for 5-9 bits of pulse width.Table S7 provides the results of the Repeated Measure ANOVA test for this study.The small p-value (p < S3. Prediction Performance for different separations between 1-bit pulses
In the first part of this PS study, EEG responses for 1-9 bit separations between two1-bit wide pulses are studied. The correlation for all of the 7 subjects are provided inTable S2. The average correlation over all subjects is around 0.3 for 1-3 bit separationsbetween the pulses, whereas the correlation is around 0.6 for 4-5 bit separation betweenthe pulses.Table S8 provides the results of the Repeated Measure ANOVA test. This testshows that the null hypothesis of having equal means for the columns of Table S2 is tobe rejected (p = 0.013).Table S14 gives the pairwise comparisons results between the correlations obtainedfor 1-9 bit separations between 1-bit wide pulses using paired t-test. This test clearlyshows that the correlations obtained for 4-9 bit separations are similar. Table S20provides the actual p-values for the pairwise paired t-tests.
S4. Prediction Performance for different separations between 2-bit pulses
In the second part of this PS study, EEG responses for 1-9 bit separations between two2-bit wide pulses are studied. The correlations for all of the 7 subjects are provided inTable S3. In general the average correlation over all subjects is less than 0.51 for pulseseparations of 1-2 bits, and the correlation is above 0.54 for 3-9 bits of separations.Table S9 provides the results of the repeated measures ANOVA test and it showsthat the null hypothesis of having equal means for the columns of Table S3 is to berejected (p < S5. Prediction Performance For 2-6 Repetitions of 1-bit wide pulse
Table S4 provides the correlation values between the generated and acquired EEGresponses for 2-6 repetitions of 1-bit wide pulse. In general the correlation is low for allrepetitions of 1-bit wide pulse (below 0.41).Table S10 gives the results for the repeated measures ANOVA test. This test showsthat there is a significant difference between the different repetitions of 1-bit wide pulse(p = 0.03).The results of the pairwise paired t-tests are given in Table S16. These tests do notshow any significant difference between the correlations obtained for 1-6 repetitions of1-bit wide pulse. The detailed p-values for the pairwise paired t-tests are given in TableS22.
S6. Prediction Performance For 2-5 Repetitions of 2-bit wide pulse
Table S5 provides the correlation values between the generated and acquired EEGresponses for 2-5 repetitions of 2-bit wide pulse. The average correlation, over allsubjects, is around 0.5 for 1 and 2 repetitions and it is around 0.6 for 3 and 4 repetitionsof 2-bit wide pulse.Table S11 provides the results of the repeated measures ANOVA test. This testindicates that correlations obtained for 2-5 repetitions of 2 bit wide pulse are significantlydifferent (p < S7. Prediction Performance For 2-4 Repetitions of 3-bit wide pulse
Table S6 provides the correlation values between the generated and acquired EEGresponses for 2-4 repetitions of 3-bit wide pulse. The average correlation, over allsubjects, is above 0.58 for 2-4 repetitions of the 3-bit wide pulse.Table S12 provides the results for the repeated measures ANOVA test. Thistest indicates that correlations for different 2-4 repetitions of 3-bit wide pulse are notsignificantly different (p = 0.158).The results for the pairwise paired t-tests are given in Table S18. The resultsindicate that correlations obtained for 2-4 repetitions of 3-bit wide pulses are statisticallysimilar. The actual p-values for the pairwise paired t-tests are provided in Table S240Table S1: Correlation values between the generated and recorded EEG responses of 1-9bit wide pulses for each of the 7 subjects (PW study).
Subject No 1 2 3 4 5 6 7 8 91 (Avg ± std) ± ± ± ± ± ± ± ± ± Subject No 1 2 3 4 5 6 7 8 91 (Avg ± std) ± ± ± ± ± ± ± ± ± Subject No 1 2 3 4 5 6 7 8 91 (Avg ± std) ± ± ± ± ± ± ± ± ± Subject No 2 3 4 5 61 ± std) 0.41 ± ± ± ± ± Subject No 2 3 4 51 ± std) 0.51 ± ± ± ± Subject No 2 3 41 ± std) 0.58 ± ± ± SumSq DF MeanSq F pValue(Intercept): Measurements
Error (Measurements)
SumSq DF MeanSq F pValue(Intercept):Measurements
Error(Measurements)
SumSq DF MeanSq F pValue(Intercept):Measurements
Error(Measurements)
SumSq DF MeanSq F pValue(Intercept):Measurements
Error(Measurements)
SumSq DF MeanSq F pValue(Intercept):Measurements
Error(Measurements)
SumSq DF MeanSq F pValue(Intercept):Measurements
Error(Measurements)
PW 1 2 3 4 5 6 7 8 91 PS1 1 2 3 4 5 6 7 8 91 PS2 1 2 3 4 5 6 7 8 91 PR1 2 3 4 5 62 PR2 2 3 4 52 PR3 2 3 42 PW 1 2 3 4 5 6 7 8 91 PS1 1 2 3 4 5 6 7 8 91 PS2 1 2 3 4 5 6 7 8 91 PR1 2 3 4 5 62 PR2 2 3 4 52 PR3 2 3 424