Tsvetomira Tsoneva

Emotional brain-computer interfaces

Research in Brain-Computer Interface (BCI) has significantly increased during the last few years. In addition to their initial role as assisting devices for the physically challenged, BCIs are now proposed for a wider range of applications. As in any HCI application, BCIs can also benefit from adapting their operation to the emotional state of the user. BCIs have the advantage of having access to brain activity which can provide significant insight into the users emotional state. This information can be utilized in two manners. 1) Knowledge of the influence of the emotional state on brain activity patterns can allow the BCI to adapt its recognition algorithms, so that the intention of the user is still correctly interpreted in spite of signal deviations induced by the subjects emotional state. 2) The ability to recognize emotions can be used in BCIs to provide the user with more natural ways of controlling the BCI through affective modulation. Thus, controlling a BCI by recollecting a pleasant memory can be possible and can potentially lead to higher information transfer rates. These two approaches of emotion utilization in BCI are elaborated in detail in this paper in the framework of non-invasive EEG based BCIs.

Automated summarization of narrative video on a semantic level

The movie industry produces thousands of feature films and TV series annually. Such massive data volumes would take consumers more than a lifetime to watch. Therefore, summarization of narrative media, which engages in providing concise and informative video summaries, has become a popular topic of research. However, most of the summarization solutions so far aim to represent just the overall atmosphere of the video at the expense of the story line. In this paper we describe a novel approach for automated creation of summaries for narrative videos. We propose an automated content analysis and summarization framework for creating moving-image summaries. We aim at preserving the story line to the level that users can watch the summary instead of the original content. Our solution is based on textual cues available in subtitles and movie scripts. We extract features like keywords, main characters names and presence, and combine them in an importance function to identify the moments most relevant for preserving the story line. We develop several summarization methods and evaluate the quality of the resulting summaries in terms of user understanding and user satisfaction through a user test.The efficiency of reputation system depends on the quality of feedbacks. However current reputation models in peer-to-peer systems can not process such strategic feedbacks as correlative and collusive ratings. Furthermore in them there exists unfairness to blameless peers. We propose a new reputation management mechanism to restrain false feedbacks. Our method uses two metrics to evaluate peers: feedback and service trust. After a transaction both service consumer and provider report the quality of this transaction. According to two ratings, service trust of server and feedback trust of consumer are separately updated. Furthermore the former is closely related to the latter. Besides reputation model we also propose a punishment mechanism to prevent malicious servers and liars from iteratively exerting bad behaviors in the system. However under punishment server is only restrained from providing services and it can continuously send out service requests; consumer is restrained from launching requests while it can provide services. Simulation shows our approach can effectively process aforesaid strategic feedbacks and mitigate unfairness.

Neural dynamics during repetitive visual stimulation

OBJECTIVE Steady-state visual evoked potentials (SSVEPs), the brain responses to repetitive visual stimulation (RVS), are widely utilized in neuroscience. Their high signal-to-noise ratio and ability to entrain oscillatory brain activity are beneficial for their applications in brain-computer interfaces, investigation of neural processes underlying brain rhythmic activity (steady-state topography) and probing the causal role of brain rhythms in cognition and emotion. This paper aims at analyzing the space and time EEG dynamics in response to RVS at the frequency of stimulation and ongoing rhythms in the delta, theta, alpha, beta, and gamma bands. APPROACH We used electroencephalography (EEG) to study the oscillatory brain dynamics during RVS at 10 frequencies in the gamma band (40-60 Hz). We collected an extensive EEG data set from 32 participants and analyzed the RVS evoked and induced responses in the time-frequency domain. MAIN RESULTS Stable SSVEP over parieto-occipital sites was observed at each of the fundamental frequencies and their harmonics and sub-harmonics. Both the strength and the spatial propagation of the SSVEP response seem sensitive to stimulus frequency. The SSVEP was more localized around the parieto-occipital sites for higher frequencies (>54 Hz) and spread to fronto-central locations for lower frequencies. We observed a strong negative correlation between stimulation frequency and relative power change at that frequency, the first harmonic and the sub-harmonic components over occipital sites. Interestingly, over parietal sites for sub-harmonics a positive correlation of relative power change and stimulation frequency was found. A number of distinct patterns in delta (1-4 Hz), theta (4-8 Hz), alpha (8-12 Hz) and beta (15-30 Hz) bands were also observed. The transient response, from 0 to about 300 ms after stimulation onset, was accompanied by increase in delta and theta power over fronto-central and occipital sites, which returned to baseline after approx. 500 ms. During the steady-state response, we observed alpha band desynchronization over occipital sites and after 500 ms also over frontal sites, while neighboring areas synchronized. The power in beta band over occipital sites increased during the stimulation period, possibly caused by increase in power at sub-harmonic frequencies of stimulation. Gamma power was also enhanced by the stimulation. SIGNIFICANCE These findings have direct implications on the use of RVS and SSVEPs for neural process investigation through steady-state topography, controlled entrainment of brain oscillations and BCIs. A deep understanding of SSVEP propagation in time and space and the link with ongoing brain rhythms is crucial for optimizing the typical SSVEP applications for studying, assisting, or augmenting human cognitive and sensorimotor function.

Eliciting steady state visual evoked potentials near the visual perception threshold

Steady state visual evoked potentials (SSVEP) are widely used in EEG research as they offer a relatively high signal to noise ratio allowing the investigation of visual processing at the cortical level. Presentation of a repetitive visual stimulus (flicker, RVS) at a frequency in the range from approximately 1 to 100 Hz, elicits an oscillatory response at the same frequency of the stimulus and/or harmonics which can be observed in the electroencephalogram (EEG), particulary at occipital sites. The frequency and the modulation depth of the RVS determine the strength of the corresponding SSVEP response. While a stronger response allows for a higher signal-to-noise ratio, it does also negatively influence comfort and safety. By relying on visual perception research which characterize the flicker perception thresholds for different frequencies and modulation depths, we focus in this paper on analyzing the SSVEP response near the perception threshold and we show that it is possible to design quasi-imperceptible RVS which elicits a sufficiently strong SSVEP response.

EEG-rhythm dynamics during a 2-back working memory task and performance

Working memory is an essential component of human cognition and determines to a large extent an individuals intellectual ability. In this paper, the human brain oscillatory response system associated with working memory performance is evaluated in an experimental and analysis setting involving 10 volunteers performing a visual 2-back task. Event-related dynamics in three bands: theta (3.5–7 Hz), alpha (7.5–12 Hz) and upper beta (17–29 Hz) at 32 locations distributed over the scalp are examined analyzing the event-related desynchronization (ERD)/synchronization (ERS) in these bands. Both global dynamics as well as trial- and subject-specific trends were considered. The overall across participants trend shows that the theta level synchronizes during working memory engagement, whereas beta and alpha desynchronizes. While common features seem to emerge, different subjects exhibit equally significant but opposite in direction correlation between reaction time and power dynamics.

Towards error-free interaction

Human-machine interaction (HMI) relies on pat- tern recognition algorithms that are not perfect. To improve the performance and usability of these systems we can utilize the neural mechanisms in the human brain dealing with error awareness. This study aims at designing a practical error detection algorithm using electroencephalogram signals that can be integrated in an HMI system. Thus, real-time operation, customization, and operation convenience are important. We address these requirements in an experimental framework simulating machine errors. Our results confirm the presence of brain potentials related to processing of machine errors. These are used to implement an error detection algorithm emphasizing the differences in error processing on a per subject basis. The proposed algorithm uses the individual best bipolar combination of electrode sites and requires short calibration. The single-trial error detection performance on six subjects, characterized by the area under the ROC curve ranges from 0.75 to 0.98.

Towards a Ground Truth for Affective Classification in Movies

Automatic affective movie classification can be a powerful technology that can facilitate searching, recommending and play listing of movies. In the process of developing a good classification system, the identification of suitable classes and the selection of proper training material is crucial. In this paper we describe the process of developing a reliable ground truth database that will be used for automated classification of emotions conveyed in movies. We identify a list of emotion labels that people commonly use to describe emotions in movies and employ them to annotate movie fragments. The most adequate fragments are then selected to form the final ground truth database which can later be used for affective classification in movies.

Closed-loop system to enhance slow-wave activity

OBJECTIVE Recent evidence reports cognitive, metabolic, and sleep restoration benefits resulting from the enhancement of sleep slow-waves using auditory stimulation. Our objective is to make this concept practical for consumer use by developing and validating an electroencephalogram (EEG) closed-loop system to deliver auditory stimulation during sleep to enhance slow-waves. APPROACH The system automatically detects slow-wave sleep with 74% sensitivity and 97% specificity and optimally delivers stimulation in the form of 50 ms-long tones separated by a constant one-second inter-tone interval at a volume that is dynamically modulated such that louder tones are delivered when sleep is deeper. The system was tested in a study involving 28 participants (18F, 10M; 36.9  ±  7.3 years old; median age: 40 years old) who used the system for ten nights (five nights in a sham condition and five in a stimulation condition). Four nights in each condition were recorded at-home and the fifth one in-lab. MAIN RESULTS The analysis in two age groups defined by the median age of participants in the study shows significant slow wave activity enhancement (+16.1%, p  <  0.01) for the younger group and absence of effect on the older group. However, the older group received only a fraction (57%) of the stimulation compared to the younger group. Changes in sleep architecture and EEG properties due to aging have influenced the amount of stimulation. The analysis of the stimulation timing suggests an entrainment-like phenomenon where slow-waves align to the stimulation periodicity. In addition, enhancement of spindle power in the stimulation condition was found. SIGNIFICANCE We show evidence of the viability of delivering auditory stimulation during sleep, at home, to enhance slow wave activity. The system ensures the stimulation delivery to be at the right time during sleep without causing disturbance.

Automatic characterization of sleep need dissipation using a single hidden layer neural network

In the two process sleep model, the rate of sleep need dissipation is proportional to slow wave activity (SWA; EEG power in the 0.5 to 4 Hz band). The dynamics of sleep need dissipation are characterized by two parameters (the initial sleep need So and the decay rate γ) that can be calculated from SWA values in NREM sleep. The goal in this paper is to use a neural network classifier to automatically detect NREM sleep and estimate Ŝo and γ using a single EEG signal that is captured during sleep at home. The data from twenty subjects (4 sleep nights per subject) was used in this research. The neural network architecture was optimized using as training and validation sets the EEG sleep data from a previous study. Given the nature of the model, only three stages were considered (NREM, REM, and WAKE). The classification accuracy characterized by the Kappa value achieved in this study dataset was 0.63 (substantial agreement with manual staging) and the specificity/sensitivity for NREM detection were 0.87 and 0.8 respectively. The higher specificity in NREM detection led to systematic So underestimation (i.e. So > Ŝo) and 7 overestimation (i.e. γ < γ). However the variability of the, Ŝo and γ across nights of the same subject is lower compared to the variability of S0 and γ This shows that using automatic staging to characterize sleep need dissipation leads to capturing the most specific and less variable EEG segments that contribute to SWA. This is suitable to characterize sleep need outside sleep lab settings (e.g. at home) that cannot be controlled to the same extent as sleep lab studies.

Effect of Light Priming and Encouraging Feedback on the Behavioral and Neural Responses in a General Knowledge Task

The increase of cognitive demands in society nowadays requires new ways to deal with problems, such as burnout and mental fatigue. Lately, more and more scientifically-based rigorous research in the area of brain-computer interfaces has been done in the quest for restoring and augmenting cognition. The current research work investigates light-based priming and positive reinforcement as possible mediators of cognitive enhancement.