Review of the State-of-the-art on Bio-signal-based Brain-controlled Vehicles
11 Review of the State-of-the-art onBio-signal-based Brain-controlled Vehicles
Amin Hekmatmanesh, Pedro H. J. Nardelli,
Senior Member, IEEE,
Heikki Handroos
Abstract —Brain-controlled vehicle (BCV) is an already established technology usually designed for disabled patients. This reviewfocuses on the most relevant topics on the brain controlling vehicles, especially considering terrestrial BCV (e.g., mobile car, carsimulators, real car, graphical and gaming cars) and aerial BCV, also named BCAV (e.g., real quadcopter, drone, fixed wings, graphicalhelicopter and aircraft) controlled using bio-signals such as electroencephalogram (EEG), electrooculogram and electromyogram. Forinstance, EEG-based algorithms detect patterns from motor imaginary cortex area of the brain for intention detection, patterns likeevent related desynchronization\event related synchronization, state visually evoked potentials, P300, and generated local evokedpotential patterns. We have identified that the reported best performing approaches employ machine learning and artificial intelligenceoptimization methods, namely support vector machine, neural network, linear discriminant analysis, k-nearest neighbor, k-means, waterdrop optimization and chaotic tug of war optimization optimization. We considered the following metrics to analyze the efficiency of thedifferent methods: type and combination of bio-signals, time response, and accuracy values with the statistical analysis. The presentwork provides an extensive literature review of the key findings of previous ten years, indicating the future perspectives in the field.
Index Terms —Bio-signal Patterns, Controlling, Machine Learning, Artificial Intelligence Simulator, Vehicle, Aerial vehicle (cid:70)
NTRODUCTION
The recent research in neuroscience supported by the devel-opment of high-precision sensors and artificial intelligencemethods has significantly increased our knowledge abouthow the brain works. In particular, human body movementsactivate the neurons in the sensorimotor cortex area. Theactivated neurons generate action potentials for differentactions in which has different patterns with specific prop-erties. Several studies have had performed for exploringthe patterns in the Electroencephalogram (EEG) signals.Thereafter, automatic methods of identifying and predictingthe patterns specifically onset of a voluntary movementhave been started [1].The Brain Computer Interface (BCI) science use thepatterns for controlling applications such as bionic hand,bionic leg, mobile robots, vehicles and typing. Among BCI’svast variety of applications, this review focuses on theBrain-Controlled Vehicle (BCV) and Brain-Controlled AerialVehicle (BCAV) applications mainly designed for normalpeople and specifically brain stroke disabled patients. Oneof the most important bio-signals is the EEG and the firststep is knowing the EEG rhythms and changes after tasksand stimulation. The important patterns for diagnosing theintention of the drivers are Event Related Potentials (ERPs),State Visually Evoked Potentials (SSVEP), Desynchroniza-tion\Event Related Synchronization (ERD\ERS), ReadinessPotentials (RP) and Local Evoked Potentials (LEP). Figure 1illustrates the BCV applications.EEG is a real-time signal that the current resolutionis not good enough for the BVC and BCAV applications. ‚ The authors are with School of Energy Systems, LUT University, Finland.This paper is partly supported by Academy of Finland via: (a) ee-IoTproject n.319009, (b) FIREMAN consortium CHIST-ERA/n.326270, and(c) EnergyNet Research Fellowship n.321265/n.328869.
Therefore, hybrid methods have been developed to coverthe deficits of previous methods. For example, using EEGwith other bio-signals such as Electromyogram (EMG),Electrooculogram (EOG) and functional Near-Infrared Spec-troscopy (fNIRS) to gain more information from human forcontrolling applications. In addition to bio-signals, externalsensors are employed for recording and analysing the envi-ronment information for better analysing the EEG and thesituation.In specific, BVC aims at tasks related to car navigation,namely keep the lane, passing the cars, following cars, turn-ing, Obstacle Avoidance Control (OAC), braking in differentsituations, specifically the Emergency Brakes Control (EBC).The same commands are computed for the BCAV with twomore directions of moving upward (take off) and downward(landing).By a real intention of movements, specific patterns ap-pear in the EEG about 0.5s to 2s earlier and then the inten-tion turn to action [2]. The concept of the reviewed studiesare developing novel algorithms for finding the onset ofImaginary Movement (IM) patterns such as ERD\ERS andReadiness Potentials (RP).The aim of the present review is preparing a comprehen-sive review on the BVC and BCAV studies in the previous10 years. Also, we expect that the present contributionwould be helpful for understanding the recent history ofthe field, how the ideas and studies have been developedand improved. Then, new ideas for the future develop-ments, which are based on the recent technologies, could bebetter contextualized. The papers we have covered in thisstudy are summarized in Table 1 to Table 4, presented inAppendix, to provide a systematic comparison between thedifferent contributions.The rest of this paper is divided as follows: Section 2we provide the background knowledge, mainly based on a r X i v : . [ ee ss . SP ] J un brain rhythms with intention identification approaches anddata acquisition model, both applied in BCV and BCAV, aswell as limitations and open questions. Section 3 discuss thealgorithms for automatically predicting the drivers inten-tion based on patterns from bio-signals. Section 4 presentsapplications for training and testing the models in real-timemode. Section 5 concludes this review by posing the alreadysolved questions and current limitations, while providingour future vision about the topic. ACKGROUND ON
BVC
Brain is a multi-functional system that different neuronsgenerate different rhythms with specific features. The de-tectable rhythms changes based on the the type of actions,stimulation and task experiments. Changes of the rhythmsis also a key clue for early diagnosing of a disease andserious situations. By focusing on the sensorimotor cortexarea rhythms it is possible to predict the subject’s intentionof movement. Some of the studied patterns for intentiondetection (thinking) are ERD [3], ERP [4], ERS [3], andSSVEP [5] in which they are defined as follows.
ERD is a cognitive pattern, in which appears after intendingto move and ERS is the second pattern appear immediatelyafter the ERD if the intention turns to action. The location ofrecording the pattern is the sensorimotor cortex area of thebrain [6].
The SSVEP is a response pattern, in which appears when avisual stimulation applied on a human. By applying a visualstimulation in a specific range the same evoked potentialpatterns named as SSVEP will appear in the visual cortex.The advantages of the SSVEP is the high Signal to NoiseRate (SNR) in compare to other patterns. [5].
ERPs are the measured electrophysiological response byEEG to a specific stimulation. The P300 is a known brainresponse to a cognitive event after 300 ms. Some of theother patterns are N100, N200 and p100 etc. The P300 isthe aim pattern in the controlling applications [3]. As anexample, the P300 pattern has been used for typing (predict-ing, decision making) applications for disabled patients, byconcentrating on the letters. In BCV applications, the P300is employed for destination selection.
Some studies searching for new ERPs for better controllingsystem. Therefore, new tasks are designed and applied tostimulate neurons other than sensorimotor cortex area suchas auditory tasks, then the obtained patterns are employedfor further computations and controlling applications [7].
RP is a pattern generated before real movement about 1.5sto 1s. The RP pattern is a useful pattern for repetitivevoluntary movements such as walking. In the processing,RP is divided into early and late RPs. The early appearsabout 1.5s before voluntary movement at the central area ofthe cortex and the late RP appears about 500ms before thevoluntary movement at the primary motor cortex area [8],[9].
In order to control a BCI application using the bio-signals,amplifiers for measuring the human body changes dur-ing the experiments is required. The well-known devicesare EEG, EMG, EOG amplifies (are suitable for real-timeprocessing) and fNIRS and functional Magnetic ResonanceImaging (fMRI) devices that the details are presented asfollows:
In order to measure noninvasive signals from heart, brainactivities and muscles ECG, EEG and EMG amplifiers aremanufactured, respectively. The usual electrodes for acquir-ing EEG, EMG and ECG signals are the Ag\AgCl thatknown as nonpolarized electrodes. The other popular elec-trode is disposable electrode, named as gel-based or Bio-Potential (BP) electrode which is one time use. In theory,the BP electrode senses ion flow on the tissue surface, andthen converts it to electron current. For the EMG measure-ment using the BP electrodes, the ion distribution generatesthrough applying the nervous stimuli and muscles contrac-tion. The employed electrodes categorized as nonpolarizedand polarized. The nonpolarized electrodes (Ag\Agcl) passthe current across the electrolyte interface. Therefore, lessnoise records in compare to the polarized electrodes in caseof movement noise. Also, the nonpolarized electrodes areeasy for manufacturing and it has very low half-cell poten-tial named as dc offset. Therefore, the Ag\Agcl is popularfor the EEG recording in compare to other electrodes. Thepolarized electrodes do not let the current moves freelyacross the interface between the electrode and the electrolytein which acts similar to capacitors. fNIRS is a noninvaseivean imaging system for measur-ing the hemoglobin (Hb) concentration changes in neuro-vascular of the brain. The Hb concentration changes ismeasured by optical intensity measurements (characteristicabsorption spectra) through the near-infrared light. Thestudies employed fNIRS are usually hybrid methods withEEG signals for real-time controlling of the BVCA applica-tions. The fNIRS has been used for the primary motor cortexarea for imaginary tasks to find the precise executed areasand using them for identification procedures [10].
The fMIR is an accurate noninvaseive imaging system fordemonstrating the localized power in a brain map withhigh resolution. The mechanism is working based on thehemodynamic changes of the brain which is associated with
Fig. 1: Different BCV applications: A. is a vehicle simulator, B. is a vehicle with different external sensors and camera, C. is a real vehicle, D. is avideo game vehicle, E. is a mobile vehicle controlled by EEG neuronal activity [11]. In the present review, the fMRI isemployed for controlling BCAV applications. The fMRI isusually employed as a hybrid method with EEG to obtainhigh results in real-time systems.
Hybrid methods are a combination of different signals toincrease the efficiency of the results. In some methods, acombination of different bio-signals with non-bio-signal areemployed for identifying the driver’s intention and navigat-ing accurately such as combination of the EEG with EMG,Global Positioning System (GPS), camera, fNIRS, googleglasses and motion sensors named as external sensors (ac-celeration ,velocity, wind speed, etc.) [12], [13].
In order to control a vehicle, either BCV or BCAV, usingbrain signals following main steps are required:1) pre-processing,2) feature extraction,3) optimization4) feature selection,5) classifiers,6) statistical analysis,7) real-time experiment.Figure 2 provides a more detailed description of thosesteps, and possible options to be considered that will bedescribed next.The improvements of the mathematical identificationmethods in the challenges of controlling vehicles throughbio-signals and the road map of improvements during theyears 2010 to 2020 are considered. The initial BCV EEG-based questions and limitations were how to find the sourceof the neuron’s activation, frequency range of neurons activ-ities, the specific patterns related to the applied stimulation and develop algorithms for finding the patterns automati-cally. Since now, many of them are solved.The unsolved problems are mathematical algorithmsfor noise rejection and identifying specified patterns auto-matically with high precision. More specifically, the mostchallenging section is developing effective algorithms forfeature extraction and classification for automatic patternidentification. Further questions raised of which neurons areconnected in a specific task and how neurons communicateafter the stimulation, the topic, named as neuron’s connec-tivity.The next ongoing challenges are the mathematical ap-proaches for predicting patterns, designing real-time algo-rithms and speeding up processing of the time-consumingmethods such as wavelet-based methods. The key problemsin the BCV applications based on the EEG are (i) the nonlin-earity of the brain, in which generate patterns with differentvarieties for individual participants; (ii) the denoising the af-fected EEG signals by white noise (which is highly nonlinearand includes whole frequencies and it is similar to EEG); (iii)hardware limitations (distance and speed) communicationfor portable and wireless devices (irrespective of Bluetoothand Wi-Fi) in the real-time applications.In the following, we explain the methods applied toidentify the intentions of drivers based on brain signals.
RIVER ’ S I NTENTION I DENTIFICATION
To detect and predict the driver’s intention for controllinga BCV and BCAV, the steps presented in the previous sub-section needs to be followed. We will provide a brief reviewof each of them throughout this section (supported by thecompilation presented from Table 1 to Table 4 at Appendix.
The preprocessing is an important step to remove unwantedsignals named as noise from data and prepare it for fur-ther processing. The preprocessing consist of filtering and
Fig. 2: The applied methods (features and classifiers) in an algorithm to identify the drivers intention for BCV and BVAC applications. A sampleof controlling a mobile vehicle and drone application for training and testing in illustrated. segmentation. Different studies based on their aims variousfilters are employed. For example, EEG is filtered to obtainthe Alpha waves, [14] used a filter bank to extract mainfrequency waves such as Delta band (1-4 Hz), Theta band (4-8 Hz), Alpha band (8-14 Hz), Beta band (14-30 Hz), and lowGamma band (30-60 Hz) [15], and filtered EEG for frequencyrange of 8-16 Hz to extract the ERD\ERS patterns [6], [16].In the next step, the denoised signals in the aim frequencyranges are employed for feature extraction.
Feature extraction algorithms are one of the most importantsteps of the identification algorithms. A good feature isa feature that shows high distinction for a specific partof a signal against other part of the signal. Features arecategorized as linear and nonlinear. Some of employedfeatures for BCV and BCAV are average, median, power,amplitude, variance, PSD, FFT, autoregressive, long termcorrelation, cross correlation, spectral amplitude, frequencyfiltered signal such as alpha wave, Common Spatial Pattern(CSP), Independent Component Analysis (ICA), FastICA,wavelet, Detrended Fluctuation Analysis (DFA), chaoticalgorithms such as largest lyapunov exponent, HbO andHbR (hemoglobin concentration changes) for fNIRS etc. Insome algorithms, the initial values of the features requiredoptimization.
WDO is an evolutionary developed algorithm which isbased on the water river behavior to find it’s way. The WDO’s aim is searching for optimum values in functionsusing the water behavior in the river. The idea of thealgorithm is constructed based on two water characteristicsduring moving, 1- velocity and 2- number of conveyed soilswith water. The advantage of this approach is high speedconverging [17]. This algorithm is also useful in optimiza-tion classification approaches.
CTWO is a recent developed optimization algorithm withthe concept of competition in rope pulling. The CTWO selecttwo teams as solution candidates for applying in pullingforces (interaction between teams), the amount of forcesis relative to the quality of solutions. In the algorithm, ithas five steps of 1- initialization 2- weight assignment 3-competition 4- new generation 5- termination. The advan-tage of the CTWO is higher speed in comparison with thestochastic searches [18], [19]. This algorithm is also effectivefor optimizing classifiers.In some algorithms, the extracted features are needed tobe selected by feature selection approaches.
In some cases of data recording, noise affect the qualityof data highly. Therefore, the extracted features combinedwith noise, which are needed to be selected. The employedfeature selection algorithms for BCV and BCAV are Prin-cipal component Analysis (PCA) and Linear DiscriminantAnalysis (LDA). The selected features are then fed to theclassifiers for categorising.
In this section, the developed methods as classifiers areconsidered. The utilized classifiers were used in three modesof offline, semi-autonomous and real-time modes. Offlinemode is useful for training a model for the real-time mode,but in some researches only the efficiency of the methods inoffline mode is presented. Real-time mode is the aim of BCIstudies to find out how much the developed algorithms arereliable. The semi-autonomous applications is usually com-bination of real-time mode and automatic methods basedon external sensors which are not under control of users.In another words, only some limited commands are undercontrol of user and the rest are automatic.Classifiers are divided into three types of supervised,semi-supervised and unsupervised. In the presented re-views studies, the employed algorithms for the BCV andBCAV applications are supervised and unsupervised. In thesupervised algorithms, the labels of segmented signals fordifferent sates are determined, but in unsupervised classi-fiers the labels are enigmatic. In the following steps, theemployed classifiers for controlling a BCV and BCAV arepresented and the efficiency of individuals are presented inTable 1 to Table 4.
K-NN is a supervised multi classifier, which is based onvoting. The voting computations is based on the Euclideandistance of the features around (neighbours) a feature. Thenumber of neighbors is the K parameter which is veryimportant. The results are highly related to the numberof selected K neighbors and the right value is definedusually by test and test. For example, by selecting K=7, thealgorithm select the seven closest data to the new comingdata. If at least four of the neighbours belongs to Class 1then the algorithm vote the new data to be classified as Class1, otherwise categorized in class 2.
LDA is a linear classifier, which is also used for featureselection by reducing the feature space dimension. Theconcept of the algorithm is maximizing between-group scat-tering over within-group scattering. In another words, thealgorithm searches for the projections that reduce the interclass variance whilst increase the distance between classes.The decision maker is a hyperplane, which is applied on themean of the two classes distance [16], [20], [21].The fundamental concept of the RLDA is the same asLDA that the regularization mode enable ability of employ-ing more correlated features at the same time with lesserror. The concept of the regularizing the LDA algorithmis regularizing the scattering of the inter class features tohave a non-singular matrix. The LDA is then applied on theregularized matrix. The RLDA is also fast processing andgenerate efficient results.
NN is a nonlinear supervised classifiers which is suitable formulti classifications. The structure of NN with back propa-gation is based on the assigned weights for the neurons thatare nonlinear functions such as Sigmoidal and Gaussian. The structure is so flexible to be changed according to thefeatures distribution and results.
RBFNN is a supervised classifier, which is based on the RBFactivation function. In the algorithm, the RBFNN containsinput-, hidden- and output- layers with the activation func-tions (neurons). The neurons trained with the labeled fea-tures in a training procedure. For classifying a new comingdata, the neurons compute the difference between the newcoming data and trained weights. The new data classified asthe same as most similar neuron which is belongs to.The RBF is flexible to be used as a kernel of classifierssuch as SVM (defined in the next part). A generalizedversion of the RBF named as Generalized RBF (GRBF) isintroduced and employed as a kernel in SVM. The GRBF isparametrized by three free parameters of width and centerin Gaussian function. The results are more stable and higheraccuracies obtained, when it is used as a kernel [22], [23].In order to increase the efficiency, speed and optimize thenetwork size of the RBFNN a method named as generalizedgrowing and pruning (GGAP) is developed. The GGAPalgorithm, link the aim desired accuracy of the RBFNNwith the importance measurements of the closest addednew neuron. The importance measurement computed usingaverage content of the specific neuron [24].
SVM is a supervised binary classifier. In the procedure,the the input data is transferred into higher dimension bykernels such as RBF, GRBF, polynomial etc and then a lineardecision maker applied. In cases of multi classification atechnique of one versus all is employed. For example, in acase of three classes, first class 1 is classified against featuresof class 2 and 3 as one class, and then class 2 is classifiedagainst class 3.Different versions of SVM is also employed in the clas-sification studies such as SMSVM, the base on the same asSVM but an optimization algorithm is set to customize thecost function and regularization algorithm. The SMSVM ismore efficient algorithm with the GRBF kernel [6], [16].
Thresholds play an important role in the threshold-basedclassifiers. Employing only thresholds as classifiers has riskof high error rates in the results, specifically for the nonlin-ear features. In order to consider the risks of the threshold-based classifiers algorithms such as Vector Phase Analysis(VPA) are employed. VPA is a threshold-based classifier,which is employed for hemoglobin concentration changes(HbO and HbR) in the fNIRS features.
QN is an analytic method for solving equations efficiently.The QN developing idea was constructing models for pre-dicting waiting time in queues. In a BCV study, the QNpredictive model is employed for controlling steering in avehicle [25]. The QN for BCV is constructed by preview, pre-dict and control modules. The input of the preview module is the desired path to determine the desired vehicle position.The predictive module input is the road information andthe vehicle states, which are obtained by external sensorsto determine the predictive position. The control moduleinput is the subtraction of the preview from the predictivemodule to compute the error for the steering commandcomputations.
CNN is a supervised classifier, which is based on deeplearning and widely employed in image processing. TheCNN algorithm is divided into three parts of convolution-and pooling- layers, and feed forward NN. A constructedCNN has ability of having several convolution and poolinglayers that the number of layers needs to be adjusted. Theconvolution layer is employed for producing features fromthe input data. The pooling layer is then employed fordimension reduction of the convolution layer and the NNis for classifying.
EC classifier is a combination of several learning algo-rithms in a classifier. It is then known as a generalizednew approach to increase the efficiency of any classifiers incomparison with individual classifiers such as boosting andbagging methods. Boosting is a popular method in EC forreducing bias to obtain strong dependency in the data. Inthe study [26], a multi-class boosting method is employed,named as AdaBoost [27].
MPC is a method for controlling a process meanwhilesatisfying equation criterion. The remarkable advantagesof the MPC are flexibility and open formula for linear,nonlinear and multi variable equations without changingthe MPC controlling algorithm. In a BCV application, theMPC employed for predicting the driver’s intention basedon a model [28].
The LR is a variable dependent binary linear supervisedclassifier, which is based on a logistic function (S-shapedfunction) in a statistical model. The LR model’s aim ismodeling the probability of the features related to indi-vidual classes such as imagination of right and left handmovements. In another words, the LR mission is finding alinear decision boundary between classes using the param-eters which are assigned to the features. The computationsare based on relation between dependent binary variablesof the classes and maximum likelihood estimation. Theweights are then adjusted and applied on the features forclassification.
K-means is an unsupervised clustering algorithm. In thealgorithm, K number of clusters are defined and then themathematical approaches find the K centers which are cen-ter of clusters. Each data is then clustered based on the dis-tance to the centers. The clusters are formed by minimizingthe within-cluster variance.
GMM is an unsupervised clustering method for categoris-ing features into C number of classes. In the algorithm,it assumes that the features of each class has a Gaussiandistribution and the feature space that consist of severalclasses follows a rule of mixing finite Gaussian distributionsand each Gaussian has specific center and width. There-fore, GMM is a probabilistic model in which features aredistributed based on a mixture of a number of Gaussianfunctions with unknown parameters that are computed byMaximum Likelihood Estimation.
The HMM concept comes form the interpretation of theworld problems that comes from visible and invisiblesources. The HMM is an unsupervised clustering, which isthe extension of the Markov Model (MM), that the principalsare based on the Markov Chain (MC). In another definition,HMM is a statistical model, which is based on observablepatterns that are relative to unobservable interior factors. Inthe procedure of the HMM, the observable patterns and un-observable internal factors are named as patterns and states,respectively. The algorithm has two random processes forthe layers, named as hidden and visible processes for thehidden states and observable patterns, respectively. The hid-den states compute a MC and the probability distribution ofthe patterns relative to the states. The features will be thencategorized Based on the probability computations [29].
In order to measure the efficiency of the classifiers for com-parison, the statistical algorithms such as accuracy, sensitiv-ity and specificity are employed that the computations arebased on on four parameters as follows: True Positive (TP),True Negative (TN), False Positive (FP) and False Negative(FN). Sensitivity is the TP rate and Sensitivity is the TNrate in classifications. TP is the correct features which arecategorized correctly, TN is the the correct features whichare categorized incorrectly, FP is the false features (incorrect)which are categorised correctly and FN is the false featureswhich are categorized incorrectly [22], [23], [30].
In different studies disparate applications are employed.Even if the rule of controlling are approximately the same,but each application employed tricks to obtain better accu-racy. The real-time BCV and BCAV employed applicationsin the reviewed papers are presented as follows: vehiclesimulator, graphical game, real car in real world, mobilerobot, quadcopter, drone, helicopter and aircraft. In thefollowing section, we will describe in details studies aboutcontrolling BCV and BCAV.
TUDIES ON
BCV
AND
BCAV
In order to control a vehicle different methods using thebio-signals have been employed. Published studies on theBCV and BCAV topics are driver’s intention detection forcontrolling vehicles for navigation, changing lane, steeringcontrol, [31], [32], EBC [26], [33], and OAC [13], [15]. Some studies reported the accuracy results based on individualsubjects that we compute the average values and reportthem in Table 1 to Table 4.
In a key series study, Haufe et al. [34] implemented an EBCsystem for BCV applications using EEG and EMG signalsin a graphical racing car task. In the algorithm, the ERPpatterns relative to the emergency brakes were employed fortrain the algorithm and then tested on a simulated vehicle.In the algorithm, the computed features were the area underthe curve and then classified using the RLDA classifier.The evaluations for braking in real-time mode were basedon the detection accuracy and response time (reaction)parameters. Then, in their next identification study, Kimet al. [35] implement an algorithm for an assistant brake(soft and sharp braking) for different situations specificallyEBC based on the driver’s intention. The experiment wasperformed using a simulated vehicle which is combinedwith the EEG and EMG devices. The braking scenario forthe simulated vehicle is designed in a graphical traffic street.The idea in the algorithm was extracting features from threepatterns as follows: combination of RPs (time interval from300 ms before stimulation to 600 ms after stimulation),MI (ERD\ERS- obtained by filtering EEG data between 5and 35 Hz) and ERP (obtained by Hilbert transformation)patterns. Results showed that the detection results increasedin comparison with the previous study. Also, the authorsnoticed that neurons generate specific ERP patterns relativeto emergency cases. Therefore, Haufe et al. [36] identifiedthe ERP patterns related to the emergency brakes in a task,and then improved the robustness of the previous methodsin a real-time experiment [34], [35]. The aim of the study wasemploying the designed algorithm in [35] in a real-worldexperiment based on different tasks. In Haufe et al. study,the experimental tasks were auditory and vehicle-followingfor predicting the emergency brakes based on ERPs in anon-traffic area. The algorithm and trained classifier is thenemployed for a real-world (real-time) experiment. However,the accuracy and robustness values did not report.In a different study by Gohring et al. [12], designedan experiment for controlling a semi-autonomous vehicle.In the experiment, a set of 16 external sensors with acamera is combined for navigating the vehicle on a roadsemi-automatically, except braking and steering commands.For controlling the vehicle, steering and braking, EEG sig-nal is recorded in two different scenarios for OAC anddriving. The commands for controlling the steering andbraking states are computed using the ERD\ERS patterns.The utilized camera and external sensors were significantlyhelpful for decreasing the Evoke Potential (EP) detectionerror rates. The algorithm is then applied on a real vehiclethat improved the results, but the complete mathematicalmethods did not present.In a continuous study of different groups for controllinga vehicle, different tasks were designed and EEG patternswere employed. Bi et al. [14] designed a head-up displaytask based on the SSVEP pattern for controlling a vehiclesimulator. In the experiment, the first step was identifyingthe alpha waves using the LDA classifier to turn the vehicle on and off. Next, a vehicle navigation (turn right, left andmove forward) based on SVM classifier is employed for theOAC algorithm for an offline mode. The trained algorithmis then tested in a real-time mode. Results for the OAC andturn the vehicle on and off were significant, but the resultsfor the three directions navigation showed high variantaccuracies. Limitations for this study were small numberof participants, the response time did not consider and therecommended speed for this algorithm is 30-40 Km. As acontinuous study, Bi et al [37] employed the P300 patternfor selecting the driver’s intention destination for the samesimulated vehicle in [14]. Results showed higher accuracywith double number of participants in comparison withthe previous study. In continuous, Fan et al. [38] combinedtheir previous methods and application using the SSVEPpattern and alpha EEG waves for controlling a vehiclesimulator with the following commands: starting, stopping,staying in lane, OAC and curve control. In the algorithm,the PSD features were computed from the patterns, andthen categorized using the LDA algorithm. Then, Bi et al.[25] proposed a model for controlling the BCV steering onthe same application in [14] and [37]. In the algorithm, amodel designed, named as queuing network for consideringthe performance of driver’s intention identification (usingthe SSVEP pattern) in advance. The identified commandsemployed for predicting the driver’s intention for goingforward, turning left and right. The designed algorithmwas based on velocity, acceleration, road information andvehicle position in a road. The performance of the modelreported based on the accuracy that is increased in comparewith their previous above-mentioned studies. Regarding theresults, the response time for the model reported 500 ms andaccuracy results were not stable because of high variationbetween subjects. In the next study, Bi et al. [13] developedan identification method for emergency brakes. In the ex-periment, a set of external sensors in an embedded systemwas employed to analyze the environment condition. Theimplemented algorithm in comparison with the previousmethods in [14], [25], [37] for emergency brakes was morecomplicated and the obtained accuracy and time responseresults increased significantly.Some algorithms are effective and need adjusting byoptimization algorithms. Teng et al. [15] implemented an al-gorithm for EBC in an OAC simulated task. In the algorithm,a model consists of feature extraction and classificationdesigned utilizing a combination of two approaches: CSPfeatures and RLDA classifier. The novelty of the algorithm isusing an optimization method for feature selection, namedas the sequential forward floating search. The algorithmalso tested in pseudo-online mode and results showed sig-nificant ascending accuracy rate and descending responsetime. Then, Bi et al. [13] improved Teng et al. method byincreasing safety of the BCV through external sensors foranalyzing the environment information and detecting obsta-cles. Results showed increasing response time increased, butaccuracies remained as the [15] study. Regarding one recentsuccessful study, Bi et al. [13] employed a combination of ex-ternal sensors and road information for autonomous drivingto increase driver’s safety. Bi et al. [13] study has potentialof increasing driver’s safety significantly. As a future world-wide road map, to expand the Bi et al. method, the road model needs to be connected to a database such as googlemap to receive the road information with high speed con-nection, a millisecond delay will cause a disaster. Therefore,utilizing external sensors has ability of updating the roaddatabase for improving the model accuracy and updatingthe road information. Regarding the study considerations,the next telecommunication generations has potential ofsolving the distance and speed constraints impressively.A continues series of studies has been implementedto evaluate the driver’s emotions when they are nervousand relax. Zhang et al. [39] designed a real-time algorithmbased on error-related potentials to control a simulated andreal vehicle. In the experiment, authors employed the PSDfeatures based on the filtered error-related ERP signals. TheLDA classifier is then applied for controlling speed, lanechange and dynamic of a vehicle. Results were not impres-sive in compare to the last studies. Next, Yang et al. [40]designed an algorithm for predicting the driver’s behavior(aggressive and unaggressive behavior) using methods oflateral- (changing lane) and longitude control (speed ac-celeration) as a driver assistant model. In the algorithm,the computed features were amplitude, long-transformedpower, PSD from different frequency bands. The method’snovelty was designing two recognition layer in the algo-rithm that consists of two supervised (SVM, KNN) andone non-supervised learning (K-means) classifiers. Then,Zhuang et al. [26] implemented an EEG-based algorithmwith online visual feedback for controlling a simulatedvehicle for a BCV application. Zhuang et al. employed thecombination of wavelet and Canonical correlation analysis(CCA) for filtering data, and then two methods, named asthe ensemble model, CSP with the SVM and CNN classifierswere employed to identify the MI patterns. The task wascontrolling a vehicle in three states of right- and left- steeringand acceleration for the OAC task. Although, the wavelet isknown as a time-consuming algorithm and causes delay, au-thors did not mention the delay in the real-time experiment.In a recent continuous study, optimization algorithmswere also employed for adjusting chaotic features withvarious classifiers for controlling a mobile vehicle. Hekmat-manesh et al. [16], [41] implemented a method for control-ling a mobile vehicle for moving forward, braking states andthe same method has been applied on a prosthetic hand.In the procedure, several classifiers with the Filter BankCSP (FBCSP) and Discrimination Sensitive Learning VectorQuantization (DSLVQ) training algorithm were employedfor considering the effectiveness of the DSLVQ coefficientsand finding the best classifier for nonlinear features. Resultsshowed that the SMSVM classifier using the generalizedRBF (GRBF) kernel obtained the best results in comparisonwith the traditional SVM, KNN and NN classifiers. Then,the best classifier employed in their further studies. TheGRBF kernel were used in their previous bio-signal process-ing studies [22], [23]. The limitation of [16] was employingthe CSP, which is useful for only two classes and the ex-tended CSP approaches for multi classes increases the errorrates significantly. In another study, Hekmatmanesh et al.[6], [32], [42] employed nonlinear features for detecting theERD\ERS patterns. In [6], [32], a remote vehicle controlledusing the XBEE Bluetooth connection chipset. In the algo-rithm, customized mother wavelets were designed based on the ERD\ERS patterns of individual subjects, and thenimported to the wavelet packet algorithm automatically.Then, the wavelet packet integrated with the detrendedfluctuation analysis (DFA) method to compute the long-termcorrelation. The features were classified using the best se-lected classifier in [16]. The results for controlling the vehiclefor braking and moving forward improved in comparisonwith predefined mother wavelet methods. Wavelet is atime-consuming method which is the main limitation. Thecomputed delay in a real-time system, obtained between oneto two seconds. On the other hand, the wireless hardwarelimitation was distance connectivity, which was about 12meters. In the next step, Hekmatmanesh et al. [19], [30]attended to improve nonlinear features in chaotic theoryfor controlling a BVC based on the ERD\ERS patterns.In the algorithm, the Largest Lyapunov Exponent (LLE)was computed, and then the initial values were optimizedusing the WDO [43] and CTWO [19] optimization intelligentmethods. Results improved in comparison with the normalLLE only in offline mode.In a recent key series studies, for increasing the SNRand accuracy recognition rate several training algorithmsare combined in two layers. Lu et al. [44] designed analgorithm based on longitude control system for controllingspeed of a simulated vehicle. In the algorithm, CSP isemployed for augmenting the EEG signal SNR, and thenPSD features were extracted (from the SSVEP patterns) andclassified using the SVM classifier with RBF kernel. Accu-racy results showed high accuracy variation for individualsubjects. Later, Lu et al. [31] extended the longitude controlby combining the lateral control for the simulated vehiclethrough the EEG. In another word, the idea is extendingtwo classes to four classes with the same identificationclassifier. The driver’s intention task was changing lane,selecting path and following cars. Accuracy results showedhigh variation for subjects. Next, Lu et al. [28] developed amethod named as MPC to increase the performance. Thealgorithm was designed based on penalty values, whichare computed using the cost function for safety criterion.The MPC controlling model was introduced by the sameauthors in [45]. The algorithm was combination of twovirtual scene scenarios, controlling road-keeping test andOAC. The reported achieved results showed high variationsfor the algorithm performance.Combining supervised and unsupervised classifiers forobtaining a better results is also a valuable idea. Zhao etal. [24], attended to design models for driver’s intentionin case of braking. Therefore, a combination of the Gaus-sian Mixture-Hidden Markov Model (GHMM) and GGAPwith RBFNN (GGAPRBFNN) is employed for increasingaccuracy and decreasing time response. The algorithm usedfor identifying slight- and normal- braking states and thentested in a real vehicle. The obtained results compared withtheir previous work [46] that was significant, but the timeresponse did not consider.In order to have a complete review, Stawiki et al. [47]designed a method for the BCV systems based on the SSVEPpatterns. In the experiment, it is attended to control a mobilevehicle with a graphical user interface and camera for livefeedback system. The novelty of the algorithm was utilizinga computational approach to remove noise and increase the amplitude of the SSVEP patterns before feature extraction.The achieved results for the large number of participantswere significant. Hernandez et al. [33], designed an al-gorithm for identify vehicle brake in different conditionsbased on the EEG signals. In the experiment, a vehiclesimulator with OAC and emergency braking scenario isprepared. Then, time domain features with the SVM andCNN classifiers were categorized, the time domain featureswere the preprocessed EEG signals. The obtained responsetime for braking in emergency cases (high speed) were notsignificant. The obtained results did not show significantaccuracy changes in comparison with the other studies withthe same aim. In another recent study, Nguyen et al. [48]developed a method for identifying driver’s intention forEBC in a vehicle. In the experiment, a simulated vehicleis utilized. The algorithm consists of the EEG band powerand auto-regressive features with a NN classifier. Resultsobtained showed significant accuracy and response timeimprovement.The second interesting vehicle is the aerial vehicles.Many attempts have been performed to control BCAV usingbio-signals. In the next part, the employed methods forcontrolling a BCAV is considered.
In the present review, the second type of vehicles namedas aerial vehicles such as drones, quadcopters, helicoptersand airplanes. BCAV applications are the other interestingtopics for investigation, Figure 3. Recently, a review of aerialvehicles in 2018 [49] is published, in which focused onstudies before 2015 and studies between 2013 to 2015 aremostly conferences. In review [49], the focus is on catego-rizing types of aerial vehicles and controlling methods, inwhich employed in our considerations. Here, we present acomplementary methodological review based on bio-signalprocessing on the effective studies from 2010 to 2020.Recently, drone technology is a commercialized appli-cation and many industries and organizations have beenemployed to increase their productivity and efficiency. Thecombination of the unmanned aerial vehicles and BCI is anewborn idea, which is the goal of researches to gain thebenefits of it. Advantages of using drones are significant, forexample low cost producing, low cost transferring, low costmaintenance, ready to fly quickly, very productive for theconditions that pilots cannot fly, clean energy, difficult ap-plications such as spacecraft applications [50], etc. There aremany categories of the aerial vehicles for different missionssuch as health care and military. In the present survey, thefocus is reviewing on bio-signal processing techniques forcontrolling BCAV applications in non-military applications.For considering the other types of the aerial unmannedvehicles and structures please refer to [50].Recent, generation of hybrid methods have been em-ployed for controlling aerial vehicles such as EEG, (f)MRIand (f)NIRS measurements. The fNIRS and the fMRI meth-ods has limitation of real-time mode usage, but they havehigh resolution. On the other hand, the EEG has abilityof utilizing in the real-time mode, but it does not havethe same resolution as in the fNIRS and fMRI. Therefore,some studies combine the advantages of both techniques at the same time and introduce hybrid methods such as EEGwith the fNIRS [51], [52], EEG with fMRI and EEG withthe eye tracker [53] etc. In the EEG-based aerial controllingalgorithms, the following patterns are employed for featureextraction: ERD\ERS, ERPs, SSVEP, eye movements andblinking. The computed features for the above-mentionedpatterns are cross correlation, LR, mean, peaks and PSDwhich are classified with different classifiers such as SVM[51], [54] and LDA [52].The difference between BCV and BCAV is four navi-gation commands such as take off (up for drons), landing(down for drons) rotates for drones (differ from turning).The preliminary commands for navigation of a fixed wingsand helicopters was controlling four main directions aftermanually take off. In a continuous studies, Royer et al. [55]attended to control a graphical helicopter in four main di-rections using the ERD\ERS patterns. In the algorithm, theextracted features were cross correlation and difference ofthe autoregressive spectral amplitude between right and lefthemisphere. Then, features were categorized using a linearclassifier. Two weak points of the algorithm were delay of2.1 sec reaction time and low precision. In another study,Akce et al. [56], used the ERD\ERS patterns for controllinga fixed-wing aerial vehicle. In the experiment, a controllingalgorithm based on selecting trajectory of a path through abinary classifier was designed. The obtained results werenot impressive. Doud et al. [57] improved Royer et al.[55] study results, using control of a virtual helicopter forsix directions based on time-frequency analysis and PSDfeatures. Finally, the helicopter accuracy results improvedby Lafleur et al. [58]. In Lafleur et al. study, the helicopterwas controlled in six directions based on the MI patters. Theevaluations were based on a modification of informationtransfer rate approach.In a continuous hybrid studies, Kim et al. [53] attendedto control a quadcopter in eight directions based on ninepoints eye gazing training using eye tracker and EEG sig-nals. In the algorithm, the camera data is first employedto extract eye’s pupil features and then power of the EEGdata and EOG paradigms are computed. The feature arethen classified using the SVM classifier with a linear ker-nel. As a different application, Shi et al. [59] controlled ahex-copter using the ERD\ERS patterns, cross correlationfeatures and LR classification. In the application, a livecamera is employed for the OAC to obtain better results.Later in an impressive study, Coenen et al. [7] used differenttechnique of response to mental task patterns for controllinga drone in two directions. The signal was recorded in anauditory imagination and spatial navigation mental task.The generated different patterns were key of improving theresults. Next, kosmyna et al. [60], [61] attended to controla quadcopter for three directions using a hybrid EEG andEMG bio-signals. In the task left hand, right hand and foottapping were employed to generate the patterns for control-ling of turn right, left and move down. In the algorithm, theMI patterns with the facial patterns were extracted from theEEG and EMG signals, respectively. The features were thenidentified using the KNN algorithm and adaptive recurrentNN classifiers.Some studies employing the same BCV experimentaltasks for producing different patterns. kryger et al. [62] Fig. 3: Some employed applications for BCAV, a. virtual helicopter, b. fixed wings c. controlling based on following a selected path, d. drone. controlled an aircraft simulator for six direction using EEG.In the experiment, only one subject is participated. Un-fortunately, the authors did not mention the algorithmsand mathematical methods in the study. In another study,Wang et al. [63] proposed a method based on the fourdifferent flickering LEDs for controlling a quadcopter in fourdirections. The authors employed the Head head-mounteddevice (HMD) in the virtual task. In order to identify theSSVEP patterns the canonical correlation analysis (CCA)with a threshold classifier is employed. The reported resultsbased on only threshold classifier is interesting.Recently, hybrid method of EEG and fNIRS become asuccessful approach for controlling a BCAV application.Recording the EEG and fNIRS signals in real-time modeis new proposed method to increase the precision of theresults. The first series of hybrid by EEG, fNIRS studiesperformed by Lin et al. [64] for controlling a quadcopterfor six directions. In the algorithm, EEG and EMG signalswere recorded in a facial gesture task. The features werecomputed based on the EEG signals and then numbers offeatures were reduced using the PCA algorithm. Resultswere not reported as a significant achievement. Anotherhybrid study is performed by Zhang et al. [65] to improveKhan et al. [66] idea. For controlling the quadcopter in sixdirections, a Google glass with the EEG signals was utilized.In the experiment, a task based on head posture MI isdesigned, then the ERD\ERS and SSVEP were extractedfor feature computations. The spectral features are thencomputed and then selected by PCA. The features werethen classified using the SVM classifier with the RBF kernel.The identification algorithm was also the combination of14 external sensors for the navigation. The results were notreported as significant achievements.The second series of hybrid studies (EEG with fNIRS)which are combined with BCV techniques is performed as follows: Khan et al. [66] utilized the EEG and NIRS datafor controlling a quadcopter. In the task, data was recordedbased on the MI for two directions and a combinationof the ERD\ERS and SSVEP patterns was employed forfeature extraction. In the algorithm, the PSD features wereextracted from the SSVEP patterns and the oxygenated anddeoxygenated hemoglobin features were extracted from theNIRS data. The obtained results based on the hybrid datarecording were significant. Next, Khan et al. [67] used theEEG, EOG and FNIRS data for extracting features for fourdirections and tested in a real-time experiment. In the algo-rithm, IM of left hand, left- and right- eye movements wereemployed for navigating the quadcopter with live videofeedback. Also, an OAC algorithm was then developedusing the SSVEP patterns. The method is applied on threesubjects and then improved in [52]. The developments ofKhan. et al. [52] were generating new patterns based on dif-ferent stimulation such as Mental arithmetic, mental count-ing, word formation, and mental rotation stimulation, andincrease the number of features peak, skewness, mean andpower from the EEG, mean, peak, slope, peak, minimumand skewness from the fNIRS. The results were increasedin compare to their previous method in [66]. Next, Khanet al. [52] improved the EEG-fNIRS method by exceedingthe number of decoded commands to eight commands(clockwise and unclockwise rotations added) for controllinga quadcopter. In the experiment, two LDAs were employedfor classifying. One LDA were used forclassifiying the fNIRSfeatures and the other LDA used for the EEG features.Results showed significant success for classifying the eightquadcopter states. Later the same team in another [51], em-ployed only the fNIRS signals to control the quadcopter forone state of moving forward. In the algorithm, mean, slope,peak, changes of HbO and HbR, HbT and COE were em-ployed as features and then categorized by SVM, threshold circle and vector phase analysis classifiers. The weak pointis the limited algorithm for two classes identification with2.3s delay. Therefore, Zafar et al. [54] performed anotherstudy for controlling the drone for three states of up, downand move forward. Therefore, mental arithmetic and mentalcounting tasks were employed with the same classifiersand features in [51]. An impressive average accuracy resultwere achieved but the delay was remained 2.3s. Recently,Kavichai et al. [68] improved the time delay in [52] usingShared Control Strategy (SCS) method. The SCS methodemployed environment information using external sensors.Therefore, Kavichai et al. combined the fNIRS and EEGfeatures with three following measurements: eye movement,distance (measurement sensor), and Global Positioning Sys-tem (GPS). Finally, through the fNIRS four commands andthrough the EEG four other commands were computed. Theaim of the approach was OAC and reducing time responsedelay.However, the challenge points, for BCV and BCAVnamed as finding fixed pattern (continuous varying EEGpatterns of drivers causes variation in accuracy), responsetime, class identification precision, and robustness are stillexisting problems and causes faults in the system. INAL D ISCUSSIONS
In the presented BCI studies, bio-signal patterns have beenemployed for controlling BCV and BCAV applications. Oneof the main parameters in the BCV- and BCAV-based appli-cations is reducing the delay of response time for processingand sending commands. Therefore, solving the delay ofprocessing in real-time systems with high accuracy is animportant limitation that needs more study. The secondcritical issue is the limitation of distance for communicationsystems in the BCV and BCAV applications.New technologies supported by beyond 5G systems(e.g., [69]) have great potential for higher quality of commu-nication with virtually no delay as in real-time processing.The high speed communications enable the applications toload high amount of data in the cloud/edge servers to storeand use them within strict time constraints. Also, it is moreapplicable to use the road information through internet fordifferent applications. By developing the BCV and BCAVmethods further, the issue of reliability of the application’ssecurity [70] will become a crucial point for future research.All in all, detecting driver’s intention for emergencybraking is a challenging point in the real world while stress,fatigue, mental workload, different emotions and environ-mental noise exists and varies individually. The secondchallenging in the vehicle emergency braking projects basedon the bio-signals is the response time. The question ishow much time is needed to prevent a crush in differentspeeds that needs more considerations. Despite the above-mentioned limitations, identifying the emergency brake sit-uation based on the EEG has high risks. Emergency casessuch as obstacle avoidance has limitations: 1- identifying anobstacle is different from predicting an obstacle, which isa difference between real dangerous situation or somethingwith potential of danger, 2- environment has high negativeinfluence on the results that increase the risk rate. R EFERENCES [1] H. Von Helmholtz,
Handbuch der physiologischen Optik . Voss, 1867,vol. 9.[2] D. J. Crammond and J. F. Kalaska, “Prior information in motorand premotor cortex: activity during the delay period and effecton pre-movement activity,”
Journal of neurophysiology , vol. 84, no. 2,pp. 986–1005, 2000.[3] C. Neuper and W. Klimesch,
Event-related dynamics of brain oscilla-tions . elsevier, 2006.[4] E. M. Sokhadze, M. F. Casanova, E. L. Casanova, E. Lamina, D. P.Kelly, and I. Khachidze, “Event-related potentials (erp) in cog-nitive neuroscience research and applications,”
NeuroRegulation ,vol. 4, no. 1, pp. 14–14, 2017.[5] A. H. Kemp, M. A. Gray, P. Eide, R. Silberstein, and P. J. Nathan,“Steady-state visually evoked potential topography during pro-cessing of emotional valence in healthy subjects,”
NeuroImage ,vol. 17, no. 4, pp. 1684–1692, 2002.[6] A. Hekmatmanesh, H. Wu, A. Motie-Nasrabadi, M. Li, andH. Handroos, “Combination of discrete wavelet packet transformwith detrended fluctuation analysis using customized motherwavelet with the aim of an imagery-motor control interface foran exoskeleton,”
Multimedia Tools and Applications , pp. 1–20, 2019.[7] J. Coenen, “Uav bci comparison of manual and pedal controlsystems for 2d flight performance of users with simultaneous bcicontrol,” 2015.[8] H. H. Kornhuber and L. Deecke, “Hirnpotentialänderungen beiwillkürbewegungen und passiven bewegungen des menschen:Bereitschaftspotential und reafferente potentiale,”
Pflüger’s Archivfür die gesamte Physiologie des Menschen und der Tiere , vol. 284, no. 1,pp. 1–17, 1965.[9] C. H. Brunia, “Cnv and spn: Indices of anticipatory behavior,” in
The Bereitschaftspotential . Springer, 2003, pp. 207–227.[10] N. Naseer and K.-S. Hong, “fnirs-based brain-computer interfaces:a review,”
Frontiers in human neuroscience , vol. 9, p. 3, 2015.[11] R. Abreu, A. Leal, and P. Figueiredo, “Eeg-informed fmri: a reviewof data analysis methods,”
Frontiers in human neuroscience , vol. 12,p. 29, 2018.[12] D. Göhring, D. Latotzky, M. Wang, and R. Rojas, “Semi-autonomous car control using brain computer interfaces,” in
In-telligent Autonomous Systems 12 . Springer, 2013, pp. 393–408.[13] L. Bi, H. Wang, T. Teng, and C. Guan, “A novel method ofemergency situation detection for a brain-controlled vehicle bycombining eeg signals with surrounding information,”
IEEE Trans-actions on Neural Systems and Rehabilitation Engineering , vol. 26,no. 10, pp. 1926–1934, 2018.[14] L. Bi, X.-a. Fan, K. Jie, T. Teng, H. Ding, and Y. Liu, “Usinga head-up display-based steady-state visually evoked potentialbrain–computer interface to control a simulated vehicle,”
IEEETransactions on Intelligent Transportation Systems , vol. 15, no. 3, pp.959–966, 2013.[15] T. Teng, L. Bi, and Y. Liu, “Eeg-based detection of driver emer-gency braking intention for brain-controlled vehicles,”
IEEE Trans-actions on Intelligent Transportation Systems , vol. 19, no. 6, pp. 1766–1773, 2017.[16] A. Hekmatmanesh, H. Wu, F. Jamaloo, M. Li, and H. Handroos, “Acombination of csp-based method with soft margin svm classifierand generalized rbf kernel for imagery-based brain computerinterface applications,”
Multimedia Tools and Applications , pp. 1–29,2020.[17] H. Shah-Hosseini, “The intelligent water drops algorithm: anature-inspired swarm-based optimization algorithm,”
Interna-tional Journal of Bio-inspired computation , vol. 1, no. 1-2, pp. 71–79,2009.[18] A. Kaveh, “Optimum design of castellated beams using the tugof war algorithm,” in
Applications of Metaheuristic OptimizationAlgorithms in Civil Engineering . Springer, 2017, pp. 9–30.[19] A. Hekmatmanesh, R. M. Asl, H. Wu, and H. Handroos, “Eegcontrol of a bionic hand with imagination based on chaoticapproximation of largest lyapunov exponent: A single trial bciapplication study,”
IEEE Access , vol. 7, pp. 105 041–105 053, 2019.[20] K. Fukunaga,
Introduction to statistical pattern recognition . Elsevier,2013.[21] A. Shakeel, M. S. Navid, M. N. Anwar, S. Mazhar, M. Jochumsen,and I. K. Niazi, “A review of techniques for detection of movementintention using movement-related cortical potentials,”
Computa-tional and mathematical methods in medicine , vol. 2015, 2015. [22] S. M. R. Noori, A. Hekmatmanesh, M. Mikaeili, andK. Sadeghniiat-Haghighi, “K-complex identification in sleep eegusing melm-grbf classifier,” in . IEEE, 2014, pp. 119–123.[23] A. Hekmatmanesh, S. M. R. Noori, and M. Mikaili, “Sleep spindledetection using modified extreme learning machine generalizedradial basis function method,” in . IEEE, 2014, pp. 1898–1902.[24] X. Zhao, S. Wang, J. Ma, Q. Yu, Q. Gao, and M. Yu, “Identificationof driver’s braking intention based on a hybrid model of ghmmand ggap-rbfnn,” Neural Computing and Applications , vol. 31, no. 1,pp. 161–174, 2019.[25] L. Bi, Y. Lu, X. Fan, J. Lian, and Y. Liu, “Queuing network model-ing of driver eeg signals-based steering control,”
IEEE Transactionson Neural Systems and Rehabilitation Engineering , vol. 25, no. 8, pp.1117–1124, 2016.[26] J. Zhuang, K. Geng, and G. Yin, “Ensemble learning based brain-computer interface system for ground vehicle control,”
IEEE Trans-actions on Systems, Man, and Cybernetics: Systems , 2019.[27] Y. Freund and R. E. Schapire, “A desicion-theoretic generalizationof on-line learning and an application to boosting,” in
Europeanconference on computational learning theory . Springer, 1995, pp. 23–37.[28] Y. Lu, L. Bi, and H. Li, “Model predictive-based shared controlfor brain-controlled driving,”
IEEE Transactions on Intelligent Trans-portation Systems , 2019.[29] B.-J. Yoon, “Hidden markov models and their applications inbiological sequence analysis,”
Current genomics , vol. 10, no. 6, pp.402–415, 2009.[30] A. Hekmatmanesh, M. Mikaeili, K. Sadeghniiat-Haghighi, H. Wu,H. Handroos, R. Martinek, and H. Nazeran, “Sleep spindle de-tection and prediction using a mixture of time series and chaoticfeatures,”
Advances in Electrical and Electronic Engineering , vol. 15,no. 3, pp. 435–447, 2017.[31] Y. Lu and L. Bi, “Combined lateral and longitudinal control ofeeg signals-based brain-controlled vehicles,”
IEEE Transactions onNeural Systems and Rehabilitation Engineering , vol. 27, no. 9, pp.1732–1742, 2019.[32] A. Hekmatmanesh, H. Wu, M. Li, A. M. Nasrabadi, and H. Han-droos, “Optimized mother wavelet in a combination of waveletpacket with detrended fluctuation analysis for controlling a re-mote vehicle with imagery movement: A brain computer interfacestudy,” in
New Trends in Medical and Service Robotics . Springer,2019, pp. 186–195.[33] L. G. Hernández, O. M. Mozos, J. M. Ferrández, and J. M. Antelis,“Eeg-based detection of braking intention under different cardriving conditions,”
Frontiers in neuroinformatics , vol. 12, p. 29,2018.[34] S. Haufe, M. S. Treder, M. F. Gugler, M. Sagebaum, G. Curio,and B. Blankertz, “Eeg potentials predict upcoming emergencybrakings during simulated driving,”
Journal of neural engineering ,vol. 8, no. 5, p. 056001, 2011.[35] I.-H. Kim, J.-W. Kim, S. Haufe, and S.-W. Lee, “Detection ofbraking intention in diverse situations during simulated drivingbased on eeg feature combination,”
Journal of neural engineering ,vol. 12, no. 1, p. 016001, 2014.[36] S. Haufe, J.-W. Kim, I.-H. Kim, A. Sonnleitner, M. Schrauf, G. Cu-rio, and B. Blankertz, “Electrophysiology-based detection of emer-gency braking intention in real-world driving,”
Journal of neuralengineering , vol. 11, no. 5, p. 056011, 2014.[37] L. Bi, X.-A. Fan, N. Luo, K. Jie, Y. Li, and Y. Liu, “A head-up display-based p300 brain–computer interface for destinationselection,”
IEEE Transactions on Intelligent Transportation Systems ,vol. 14, no. 4, pp. 1996–2001, 2013.[38] X.-a. Fan, L. Bi, T. Teng, H. Ding, and Y. Liu, “A brain–computerinterface-based vehicle destination selection system using p300and ssvep signals,”
IEEE Transactions on Intelligent TransportationSystems , vol. 16, no. 1, pp. 274–283, 2014.[39] H. Zhang, R. Chavarriaga, Z. Khaliliardali, L. Gheorghe, I. Iturrate,and J. d R Millán, “Eeg-based decoding of error-related brainactivity in a real-world driving task,”
Journal of neural engineering ,vol. 12, no. 6, p. 066028, 2015.[40] L. Yang, R. Ma, H. M. Zhang, W. Guan, and S. Jiang, “Driving be-havior recognition using eeg data from a simulated car-followingexperiment,”
Accident Analysis & Prevention , vol. 116, pp. 30–40,2018. [41] A. Hekmatmanesh, F. Jamaloo, H. Wu, H. Handroos, andA. Kilpeläinen, “Common spatial pattern combined with kernellinear discriminate and generalized radial basis function for motorimagery-based brain computer interface applications,” in
AIPConference Proceedings , vol. 1956, no. 1. AIP Publishing LLC, 2018,p. 020003.[42] A. Hekmatmanesh, “Investigation of eeg signal processing forrehabilitation robot control,” 2019.[43] A. Hekmatmanesh, R. M. Asl, H. Handroos, and H. Wu, “Opti-mizing largest lyapunov exponent utilizing an intelligent waterdrop algorithm: A brain computer interface study,” in . IEEE, 2019, pp. 1–5.[44] Y. Lu and L. Bi, “Eeg signals-based longitudinal control systemfor a brain-controlled vehicle,”
IEEE Transactions on Neural Systemsand Rehabilitation Engineering , vol. 27, no. 2, pp. 323–332, 2018.[45] L. Bi, M. Wang, Y. Lu, and F. A. Genetu, “A shared controllerfor brain-controlled assistive vehicles,” in . IEEE, 2016,pp. 125–129.[46] S. Wang, Q. Yu, and X. Zhao, “Study on driver’s turning in-tention recognition hybrid model of ghmm and ggap-rbf neuralnetwork,”
Advances in Mechanical Engineering , vol. 10, no. 3, p.1687814018764985, 2018.[47] P. Stawicki, F. Gembler, and I. Volosyak, “Driving a semiau-tonomous mobile robotic car controlled by an ssvep-based bci,”
Computational intelligence and neuroscience , vol. 2016, 2016.[48] T.-H. Nguyen and W.-Y. Chung, “Detection of driver brakingintention using eeg signals during simulated driving,”
Sensors ,vol. 19, no. 13, p. 2863, 2019.[49] A. Nourmohammadi, M. Jafari, and T. O. Zander, “A survey onunmanned aerial vehicle remote control using brain–computerinterface,”
IEEE Transactions on Human-Machine Systems , vol. 48,no. 4, pp. 337–348, 2018.[50] B. Z. Allison, S. Dunne, R. Leeb, J. D. R. Millán, and A. Nijholt,
Towards practical brain-computer interfaces: bridging the gap fromresearch to real-world applications . Springer Science & BusinessMedia, 2012.[51] A. Zafar, U. Ghafoor, M. J. Khan, and K.-S. Hong, “Drone controlusing functional near-infrared spectroscopy,” in . IEEE,2018, pp. 384–387.[52] M. J. Khan and K.-S. Hong, “Hybrid eeg–fnirs-based eight-command decoding for bci: application to quadcopter control,”
Frontiers in neurorobotics , vol. 11, p. 6, 2017.[53] B. H. Kim, M. Kim, and S. Jo, “Quadcopter flight control usinga low-cost hybrid interface with eeg-based classification and eyetracking,”
Computers in biology and medicine , vol. 51, pp. 82–92,2014.[54] A. Zafar, M. J. Khan, J. Park, and K.-S. Hong, “Initial-dip basedquadcopter control: Application to fnirs-bci,”
IFAC-PapersOnLine ,vol. 51, no. 15, pp. 945–950, 2018.[55] A. S. Royer, A. J. Doud, M. L. Rose, and B. He, “Eeg control of avirtual helicopter in 3-dimensional space using intelligent controlstrategies,”
IEEE Transactions on neural systems and rehabilitationengineering , vol. 18, no. 6, pp. 581–589, 2010.[56] A. Akce, M. Johnson, and T. Bretl, “Remote teleoperation ofan unmanned aircraft with a brain-machine interface: Theoryand preliminary results,” in . IEEE, 2010, pp. 5322–5327.[57] A. J. Doud, J. P. Lucas, M. T. Pisansky, and B. He, “Continuousthree-dimensional control of a virtual helicopter using a motorimagery based brain-computer interface,”
PloS one , vol. 6, no. 10,2011.[58] K. LaFleur, K. Cassady, A. Doud, K. Shades, E. Rogin, and B. He,“Quadcopter control in three-dimensional space using a nonin-vasive motor imagery-based brain–computer interface,”
Journal ofneural engineering , vol. 10, no. 4, p. 046003, 2013.[59] T. Shi, H. Wang, and C. Zhang, “Brain computer interface systembased on indoor semi-autonomous navigation and motor imageryfor unmanned aerial vehicle control,”
Expert Systems with Applica-tions , vol. 42, no. 9, pp. 4196–4206, 2015.[60] N. Kosmyna, F. Tarpin-Bernard, and B. Rivet, “Towards braincomputer interfaces for recreational activities: Piloting a drone,”in
IFIP Conference on Human-Computer Interaction . Springer, 2015,pp. 506–522. [61] ——, “Adding human learning in brain–computer interfaces(bcis) towards a practical control modality,” ACM Transactions onComputer-Human Interaction (TOCHI) , vol. 22, no. 3, pp. 1–37, 2015.[62] M. Kryger, B. Wester, E. A. Pohlmeyer, M. Rich, B. John, J. Beaty,M. McLoughlin, M. Boninger, and E. C. Tyler-Kabara, “Flightsimulation using a brain-computer interface: A pilot, pilot study,”
Experimental neurology , vol. 287, pp. 473–478, 2017.[63] M. Wang, R. Li, R. Zhang, G. Li, and D. Zhang, “A wearable ssvep-based bci system for quadcopter control using head-mounteddevice,”
Ieee Access , vol. 6, pp. 26 789–26 798, 2018.[64] J.-S. Lin and Z.-Y. Jiang, “Implementing remote presence usingquadcopter control by a non-invasive bci device,”
Computer Scienceand Information Technology , vol. 3, no. 4, pp. 122–126, 2015.[65] M. M. Jackson, “Quadcopter navigation using google glass andbrain-computer interface.”[66] M. J. Khan, K.-S. Hong, N. Naseer, and M. R. Bhutta, “Hybrideeg-nirs based bci for quadcopter control,” in . IEEE, 2015, pp. 1177–1182.[67] M. J. Khan, A. Zafar, and K.-S. Hong, “Hybrid eeg-nirs basedactive command generation for quadcopter movement control,”in . IEEE,2016, pp. 200–205.[68] E. Kavichai, R. Huang, and S.-W. Woo, “Shared control strategiesfor bci-based quadcopter control,” in . IEEE, 2019, pp.155–158.[69] R. C. Moioli, P. H. Nardelli, M. T. Barros, W. Saad, A. Hekmat-manesh, P. Gória, A. S. de Sena, M. Dzaferagic, H. Siljak, W. vanLeekwijck et al. , “Neurosciences and 6g: Lessons from and needsof communicative brains,” arXiv preprint arXiv:2004.01834 , 2020.[70] A. Singandhupe, H. M. La, and D. Feil-Seifer, “Reliable securityalgorithm for drones using individual characteristics from an eegsignal,”
IEEE Access , vol. 6, pp. 22 976–22 986, 2018.[71] H. Wang, L. Bi, and T. Teng, “Eeg-based emergency brakingintention prediction for brain-controlled driving considering oneelectrode falling-off,” in . IEEE,2017, pp. 2494–2497. A PPENDIX