Han-Jeong Hwang

Development of an SSVEP-based BCI spelling system adopting a QWERTY-style LED keyboard.

In this study, we introduce a new mental spelling system based on steady-state visual evoked potential (SSVEP), adopting a QWERTY style layout keyboard with 30 LEDs flickering with different frequencies. The proposed electroencephalography (EEG)-based mental spelling system allows the users to spell one target character per each target selection, without the need for multiple step selections adopted by conventional SSVEP-based mental spelling systems. Through preliminary offline experiments and online experiments, we confirmed that human SSVEPs elicited by visual flickering stimuli with a frequency resolution of 0.1 Hz could be classified with classification accuracy high enough to be used for a practical brain-computer interface (BCI) system. During the preliminary offline experiments performed with five participants, we optimized various factors influencing the performance of the mental spelling system, such as distances between adjacent keys, light source arrangements, stimulating frequencies, recording electrodes, and visual angles. Additional online experiments were conducted with six participants to verify the feasibility of the optimized mental spelling system. The results of the online experiments were an average typing speed of 9.39 letters per minute (LPM) with an average success rate of 87.58%, corresponding to an average information transfer rate of 40.72 bits per minute, demonstrating the high performance of the developed mental spelling system. Indeed, the average typing speed of 9.39 LPM attained in this study was one of the best LPM results among those reported in previous BCI literatures.

Classification of selective attention to auditory stimuli: Toward vision-free brain-computer interfacing

Brain-computer interface (BCI) is a developing, novel mode of communication for individuals with severe motor impairments or those who have no other options for communication aside from their brain signals. However, the majority of current BCI systems are based on visual stimuli or visual feedback, which may not be applicable for severe locked-in patients that have lost their eyesight or the ability to control their eye movements. In the present study, we investigated the feasibility of using auditory steady-state responses (ASSRs), elicited by selective attention to a specific sound source, as an electroencephalography (EEG)-based BCI paradigm. In our experiment, two pure tone burst trains with different beat frequencies (37 and 43 Hz) were generated simultaneously from two speakers located at different positions (left and right). Six participants were instructed to close their eyes and concentrate their attention on either auditory stimulus according to the instructions provided randomly through the speakers during the inter-stimulus interval. EEG signals were recorded at multiple electrodes mounted over the temporal, occipital, and parietal cortices. We then extracted feature vectors by combining spectral power densities evaluated at the two beat frequencies. Our experimental results showed high classification accuracies (64.67%, 30 commands/min, information transfer rate (ITR) = 1.89 bits/min; 74.00%, 12 commands/min, ITR = 2.08 bits/min; 82.00%, 6 commands/min, ITR = 1.92 bits/min; 84.33%, 3 commands/min, ITR = 1.12 bits/min; without any artifact rejection, inter-trial interval = 6s), enough to be used for a binary decision. Based on the suggested paradigm, we implemented a first online ASSR-based BCI system that demonstrated the possibility of materializing a totally vision-free BCI system.

Evaluation of various mental task combinations for near-infrared spectroscopy-based brain-computer interfaces

Abstract. A number of recent studies have demonstrated that near-infrared spectroscopy (NIRS) is a promising neuroimaging modality for brain-computer interfaces (BCIs). So far, most NIRS-based BCI studies have focused on enhancing the accuracy of the classification of different mental tasks. In the present study, we evaluated the performances of a variety of mental task combinations in order to determine the mental task pairs that are best suited for customized NIRS-based BCIs. To this end, we recorded event-related hemodynamic responses while seven participants performed eight different mental tasks. Classification accuracies were then estimated for all possible pairs of the eight mental tasks (C82=28). Based on this analysis, mental task combinations with relatively high classification accuracies frequently included the following three mental tasks: “mental multiplication,” “mental rotation,” and “right-hand motor imagery.” Specifically, mental task combinations consisting of two of these three mental tasks showed the highest mean classification accuracies. It is expected that our results will be a useful reference to reduce the time needed for preliminary tests when discovering individual-specific mental task combinations.

A new dual-frequency stimulation method to increase the number of visual stimuli for multi-class SSVEP-based brain-computer interface (BCI).

In the present study, we introduce a new dual-frequency stimulation method that can produce more visual stimuli with limited number of stimulation frequencies for use in multiclass steady-state visual evoked potential (SSVEP)-based brain-computer interface (BCI) systems. Methods for increasing the number of visual stimuli are necessary, particularly for the implementation of multi-class SSVEP-based BCI, as available stimulation frequencies are generally limited when visual stimuli are presented through a computer monitor. The new stimulation was based on a conventional black-white checkerboard pattern; however, unlike the conventional approach, ten visual stimuli eliciting distinct SSVEP responses at different frequencies could be generated by combining four different stimulation frequencies. Through the offline experiments conducted with eleven participants, we confirmed that all ten visual stimuli could evoke distinct and discriminable single SSVEP peaks, of which the signal-to-noise ratios were high enough to be used for practical SSVEP-based BCI systems. In order to demonstrate the possibility of the practical use of the proposed method, a mental keypad system was implemented and online experiments were conducted with additional ten participants. We achieved an average information transfer rate of 33.26 bits/min and an average accuracy of 87.23%, and all ten participants succeeded in calling their mobile phones using our online BCI system.

Evaluation of feature extraction methods for EEG-based brain–computer interfaces in terms of robustness to slight changes in electrode locations

To date, most EEG-based brain–computer interface (BCI) studies have focused only on enhancing BCI performance in such areas as classification accuracy and information transfer rate. In practice, however, test–retest reliability of the developed BCI systems must also be considered for use in long-term, daily life applications. One factor that can affect the reliability of BCI systems is the slight displacement of EEG electrode locations that often occurs due to the removal and reattachment of recording electrodes. The aim of this study was to evaluate and compare various feature extraction methods for motor-imagery-based BCI in terms of robustness to slight changes in electrode locations. To this end, EEG signals were recorded from three reference electrodes (Fz, C3, and C4) and from six additional electrodes located close to the reference electrodes with a 1-cm inter-electrode distance. Eight healthy participants underwent 180 trials of left- and right-hand motor imagery tasks. The performance of four different feature extraction methods [power spectral density (PSD), phase locking value (PLV), a combination of PSD and PLV, and cross-correlation (CC)] were evaluated using five-fold cross-validation and linear discriminant analysis, in terms of robustness to electrode location changes as well as regarding absolute classification accuracy. The quantitative evaluation results demonstrated that the use of either PSD- or CC-based features led to higher classification accuracy than the use of PLV-based features, while PSD-based features showed much higher sensitivity to changes in EEG electrode location than CC- or PLV-based features. Our results suggest that CC can be used as a promising feature extraction method in motor-imagery-based BCI studies, since it provides high classification accuracy along with being little affected by slight changes in the EEG electrode locations.

Improving the Robustness of Myoelectric Pattern Recognition for Upper Limb Prostheses by Covariate Shift Adaptation

Fundamental changes over time of surface EMG signal characteristics are a challenge for myocontrol algorithms controlling prosthetic devices. These changes are generally caused by electrode shifts after donning and doffing, sweating, additional weight or varying arm positions, which results in a change of the signal distribution - a scenario often referred to as covariate shift. A substantial decrease in classification accuracy due to these factors hinders the possibility to directly translate EMG signals into accurate myoelectric control patterns outside laboratory conditions. To overcome this limitation, we propose the use of supervised adaptation methods. The approach is based on adapting a trained classifier using a small calibration set only, which incorporates the relevant aspects of the nonstationarities, but requires only less than 1 min of data recording. The method was tested first through an offline analysis on signals acquired across 5 days from seven able-bodied individuals and four amputees. Moreover, we also conducted a three day online experiment on eight able-bodied individuals and one amputee, assessing user performance and user-ratings of the controllability. Across different testing days, both offline and online performance improved significantly when shrinking the training model parameters by a given estimator towards the calibration set parameters. In the offline data analysis, the classification accuracy remained above 92% over five days with the proposed approach, whereas it decreased to 75% without adaptation. Similarly, in the online study, with the proposed approach the performance increased by 25% compared to a test without adaptation. These results indicate that the proposed methodology can contribute to improve robustness of myoelectric pattern recognition methods in daily life applications.

Concurrent Adaptation of Human and Machine Improves Simultaneous and Proportional Myoelectric Control

Myoelectric control of a prosthetic hand with more than one degree of freedom (DoF) is challenging, and clinically available techniques require a sequential actuation of the DoFs. Simultaneous and proportional control of multiple DoFs is possible with regression-based approaches allowing for fluent and natural movements. Conventionally, the regressor is calibrated in an open-loop with training based on recorded data and the performance is evaluated subsequently. For individuals with amputation or congenital limb-deficiency who need to (re)learn how to generate suitable muscle contractions, this open-loop process may not be effective. We present a closed-loop real-time learning scheme in which both the user and the machine learn simultaneously to follow a common target. Experiments with ten able-bodied individuals show that this co-adaptive closed-loop learning strategy leads to significant performance improvements compared to a conventional open-loop training paradigm. Importantly, co-adaptive learning allowed two individuals with congenital deficiencies to perform simultaneous 2-D proportional control at levels comparable to the able-bodied individuals, despite having to a learn completely new and unfamiliar mapping from muscle activity to movement trajectories. To our knowledge, this is the first study which investigates man-machine co-adaptation for regression-based myoelectric control. The proposed training strategy has the potential to improve myographic prosthetic control in clinically relevant settings.

An EEG-based real-time cortical rhythmic activity monitoring system

In the present study, we introduce an electroencephalography (EEG)-based, real-time, cortical rhythmic activity monitoring system which can monitor spatiotemporal changes of cortical rhythmic activity on a subjects cortical surface, not on the subjects scalp surface, with a high temporal resolution. In the monitoring system, a frequency domain inverse operator is preliminarily constructed, considering the subjects anatomical information and sensor configurations, and then the spectral current power at each cortical vertex is calculated for the Fourier transforms of successive sections of continuous data, when a particular frequency band is given. A preliminary offline simulation study using four sets of artifact-free, eye-closed, resting EEG data acquired from two dementia patients and two normal subjects demonstrates that spatiotemporal changes of cortical rhythmic activity can be monitored at the cortical level with a maximal delay time of about 200 ms, when 18 channel EEG data are analyzed under a Pentium4 3.4 GHz environment. The first pilot system is applied to two human experiments-(1) cortical alpha rhythm changes induced by opening and closing eyes and (2) cortical mu rhythm changes originated from the arm movements-and demonstrated the feasibility of the developed system.

Near-infrared spectroscopy (NIRS)-based eyes-closed brain-computer interface (BCI) using prefrontal cortex activation due to mental arithmetic

We propose a near-infrared spectroscopy (NIRS)-based brain-computer interface (BCI) that can be operated in eyes-closed (EC) state. To evaluate the feasibility of NIRS-based EC BCIs, we compared the performance of an eye-open (EO) BCI paradigm and an EC BCI paradigm with respect to hemodynamic response and classification accuracy. To this end, subjects performed either mental arithmetic or imagined vocalization of the English alphabet as a baseline task with very low cognitive loading. The performances of two linear classifiers were compared; resulting in an advantage of shrinkage linear discriminant analysis (LDA). The classification accuracy of EC paradigm (75.6 ± 7.3%) was observed to be lower than that of EO paradigm (77.0 ± 9.2%), which was statistically insignificant (p = 0.5698). Subjects reported they felt it more comfortable (p = 0.057) and easier (p < 0.05) to perform the EC BCI tasks. The different task difficulty may become a cause of the slightly lower classification accuracy of EC data. From the analysis results, we could confirm the feasibility of NIRS-based EC BCIs, which can be a BCI option that may ultimately be of use for patients who cannot keep their eyes open consistently.

Effect of higher frequency on the classification of steady-state visual evoked potentials

OBJECTIVE Most existing brain-computer interface (BCI) designs based on steady-state visual evoked potentials (SSVEPs) primarily use low frequency visual stimuli (e.g., <20 Hz) to elicit relatively high SSVEP amplitudes. While low frequency stimuli could evoke photosensitivity-based epileptic seizures, high frequency stimuli generally show less visual fatigue and no stimulus-related seizures. The fundamental objective of this study was to investigate the effect of stimulation frequency and duty-cycle on the usability of an SSVEP-based BCI system. APPROACH We developed an SSVEP-based BCI speller using multiple LEDs flickering with low frequencies (6-14.9 Hz) with a duty-cycle of 50%, or higher frequencies (26-34.7 Hz) with duty-cycles of 50%, 60%, and 70%. The four different experimental conditions were tested with 26 subjects in order to investigate the impact of stimulation frequency and duty-cycle on performance and visual fatigue, and evaluated with a questionnaire survey. Resting state alpha powers were utilized to interpret our results from the neurophysiological point of view. MAIN RESULTS The stimulation method employing higher frequencies not only showed less visual fatigue, but it also showed higher and more stable classification performance compared to that employing relatively lower frequencies. Different duty-cycles in the higher frequency stimulation conditions did not significantly affect visual fatigue, but a duty-cycle of 50% was a better choice with respect to performance. The performance of the higher frequency stimulation method was also less susceptible to resting state alpha powers, while that of the lower frequency stimulation method was negatively correlated with alpha powers. SIGNIFICANCE These results suggest that the use of higher frequency visual stimuli is more beneficial for performance improvement and stability as time passes when developing practical SSVEP-based BCI applications.