Shohei Yano

Estimating the Intended Sound Direction of the User: Toward an Auditory Brain-Computer Interface Using Out-of-Head Sound Localization

The auditory Brain-Computer Interface (BCI) using electroencephalograms (EEG) is a subject of intensive study. As a cue, auditory BCIs can deal with many of the characteristics of stimuli such as tone, pitch, and voices. Spatial information on auditory stimuli also provides useful information for a BCI. However, in a portable system, virtual auditory stimuli have to be presented spatially through earphones or headphones, instead of loudspeakers. We investigated the possibility of an auditory BCI using the out-of-head sound localization technique, which enables us to present virtual auditory stimuli to users from any direction, through earphones. The feasibility of a BCI using this technique was evaluated in an EEG oddball experiment and offline analysis. A virtual auditory stimulus was presented to the subject from one of six directions. Using a support vector machine, we were able to classify whether the subject attended the direction of a presented stimulus from EEG signals. The mean accuracy across subjects was 70.0% in the single-trial classification. When we used trial-averaged EEG signals as inputs to the classifier, the mean accuracy across seven subjects reached 89.5% (for 10-trial averaging). Further analysis showed that the P300 event-related potential responses from 200 to 500 ms in central and posterior regions of the brain contributed to the classification. In comparison with the results obtained from a loudspeaker experiment, we confirmed that stimulus presentation by out-of-head sound localization achieved similar event-related potential responses and classification performances. These results suggest that out-of-head sound localization enables us to provide a high-performance and loudspeaker-less portable BCI system.

Comparability of activity monitors used in Asian and Western-country studies for assessing free-living sedentary behaviour

This study aims to compare the outputs of the waist-worn Active style Pro HJA-350IT (ASP; used in studies with Asian populations), the waist-worn ActiGragh™GT3X+ using the normal filter (GT3X+) and the thigh-worn activPAL3 (AP) in assessing adults’ sedentary behaviour (total sedentary time, number of breaks) under free-living conditions. Fifty healthy workers wore the three monitors simultaneously during their waking hours on two days, including a work day and a non-work day. Valid data were at least 10 hours of wearing time, and the differences between monitors on the sedentary outputs using the AP as criterion measurement were analyzed by ANOVA. The number of participants who had complete valid data for work day and non-work day was 47 and 44, respectively. Total sedentary time and breaks estimated by the AP were respectively 466.5 ± 146.8 min and 64.3 ± 24.9 times on the work day and 497.7 ± 138.3 min and 44.6 ± 15.4 times on the non-work day. In total sedentary time, the ASP estimated 29.7 min (95%CI = 7.9 to 51.5) significantly shorter than the AP on the work day but showed no significant difference against the AP on the non-work day. The GT3X+ estimated 80.1 min (54.6 to 105.6) and 52.3 (26.4 to 78.2) significantly longer than the AP on the work day and the non-work day, respectively. For the number of breaks from sedentary time, on both days, the ASP and the GT3X+ estimated significantly more than the AP: 14.1 to 15.8 times (6.3 to 22.5) for the ASP and 27.7 to 28.8 times (21.8 to 34.8) for the GT3X+. Compared to the AP as the criterion, the ASP can underestimate total sedentary time and the GT3X+ can overestimate it, and more so at the lower levels of sedentary time. For breaks from sedentary time, compared to the AP, both the GT3X+ the ASP can overestimate.

Validity and Reliability of Japanese-Language Self-reported Measures for Assessing Adults Domain-Specific Sedentary Time

Background Good quality measures of Japanese adults’ sedentary behaviors are needed to accurately assess correlates of specific sedentary behaviors. The present study assessed criterion validity of total sedentary behavior and test-retest reliability of six domain-specific sedentary behaviors. Methods We administered a questionnaire, based on previous studies, that measured domain-specific sedentary behaviors. To examine validity, agreement between self-reported time spent in sedentary behaviors from the questionnaire and objectively-measured sedentary time using accelerometers was compared among 392 adults (aged 40–64 years) in two Japanese cities. For reliability, a 2-week interval test-retest was administered to a convenience sample of 34 participants. Results The correlation between total self-reported and objectively measured sedentary time was significant (all P < 0.001) and fair-to-good for workdays (ρ = 0.57) and whole week (ρ = 0.49), but was low for non-workdays (ρ = 0.23). The difference between the two measures was significant for whole week (z = −2.25, P = 0.03) and non-workdays (z = −5.50, P < 0.001), but was not significant for workdays (z = −0.60, P = 0.55). There was a significant positive association between the difference in the two measures and the average of these two measures (workdays: r = 0.53; non-workdays: r = 0.45; and whole week: r = 0.54, all P < 0.001). There was fair-to-good test-retest reliability of total sedentary time for each domain (workdays: interclass correlation coefficient [ICC] = 0.77, non-workdays: ICC = 0.53, and whole week: ICC = 0.7; all P < 0.01). Conclusions The scale of domain-specific sedentary behaviors is reliable for estimating where and for what purpose Japanese adults spend their sedentary time, and total sedentary time is valid for workdays and the whole week.

Estimation of direction of attention using EEG and out-of-head sound localization

Brain-Machine Interfaces (BMIs) are being researched controlling external devices such as robots and computers by measuring the cranial nerve activity of the operator. The brain activities evoked by visual stimuli have been studied intensively. However, few studies have considered a BMI that uses the brain activities evoked by auditory stimuli. This study investigated whether a persons direction of attention can be estimated using an event-related potential (ERP) generated by selective attention to an auditory stimulus. An auditory stimulus and an out-of-head sound localization system that can create an audio image outside the head that is presented through an earphone were used instead of a loudspeaker system. This system was experimentally evaluated by presenting the subject auditory cues from one of six directions while the subject directed his attention in one direction. An EEG response similar to an ERP was observed. The direction of attention was estimated using support vector machine with an accuracy of 89.2[%] on average for the three subjects. This suggests that a BMI system based on the estimated direction of attention can be developed by using out-of-head sound localization.

Adaptive modeling of HRTFs based on reinforcement learning

Although recent studies on out-of-head sound localization technology have been aimed at applications in entertainment, this technology can also be used to provide an interface to connect a computer to the human brain. An effective out-of-head system requires an accurate head-related transfer function (HRTF). However, it is difficult to measure HRTF accurately. We propose a new method based on reinforcement learning to estimate HRTF accurately from measurement data and validate it through simulations. We used the actor-critic paradigm to learn the HRTF parameters and the autoregressive moving average (ARMA) model to reduce the number of such parameters. Our simulations suggest that an accurate HRTF can be estimated with this method. The proposed method is expected to be useful for not only entertainment applications but also brain-machine-interface (BMI) based on out-of-head sound localization technology.

Improving the Performance of an Auditory Brain-Computer Interface Using Virtual Sound Sources by Shortening Stimulus Onset Asynchrony

Recently, a brain-computer interface (BCI) using virtual sound sources has been proposed for estimating user intention via electroencephalogram (EEG) in an oddball task. However, its performance is still insufficient for practical use. In this study, we examine the impact that shortening the stimulus onset asynchrony (SOA) has on this auditory BCI. While very short SOA might improve its performance, sound perception and task performance become difficult, and event-related potentials (ERPs) may not be induced if the SOA is too short. Therefore, we carried out behavioral and EEG experiments to determine the optimal SOA. In the experiments, participants were instructed to direct attention to one of six virtual sounds (target direction). We used eight different SOA conditions: 200, 300, 400, 500, 600, 700, 800, and 1,100 ms. In the behavioral experiment, we recorded participant behavioral responses to target direction and evaluated recognition performance of the stimuli. In all SOA conditions, recognition accuracy was over 85%, indicating that participants could recognize the target stimuli correctly. Next, using a silent counting task in the EEG experiment, we found significant differences between target and non-target sound directions in all but the 200-ms SOA condition. When we calculated an identification accuracy using Fisher discriminant analysis (FDA), the SOA could be shortened by 400 ms without decreasing the identification accuracies. Thus, improvements in performance (evaluated by BCI utility) could be achieved. On average, higher BCI utilities were obtained in the 400 and 500-ms SOA conditions. Thus, auditory BCI performance can be optimized for both behavioral and neurophysiological responses by shortening the SOA.

Improving the Localization Accuracy of Virtual Sound Source through Reinforcement Learning

Localization of virtual sound source is a technology that allows the reproducing of three-dimensional sounds using stereo earphones, and applications of this technology in Brain-machine interface that use auditory stimuli are being investigated. In order to achieve virtual sounds using this technology, the Head-related Transfer Function (HRTF) of the user must be measured accurately. The HRTF can be measured accurately with the appropriate placement of the microphones and measurement environment, but procuring an ideal setup is usually difficult. To overcome this, we instead attempt to obtain an accurate HRTF using reinforcement learning. We performed simulations and verified that with the proposed method the HRTF accuracy improved on 24 horizontal directions. Also, in online learning experiments, the localization accuracy was improved for 3 subjects, suggesting the validity of our method.