Shunji Mitsuyoshi

Validity of the Mind Monitoring System as a Mental Health Indicator

We have been developing a method to evaluate the mental health condition of a person by the sound of his or her voice. Now, we have applied this technology as a system to create a smartphone app. Since using voice to measure ones mental health condition is a non-invasive method and as it could be used continually through the smartphone, one carries, unlike a routine checkup, it could be used for monitoring on a daily basis. The purpose of this study is to compare this vitality score and the widely used BDI (Beck Depression Inventory) and evaluate its validity. This experiment was conducted at COI (Center of Innovation) Program of the University of Tokyo with a total of 50 employees of multiple corporations as subjects between early December 2015 and early February 2016. The test subjects were each lent a smartphone with our app recording their voices automatically during calls, and in addition to it, we had them read and record a fixed phrase daily. BDI test was conducted at the beginning of the experiment period. The vitality score was calculated based on the voice data collected during the first two weeks of the experiment and considered it the vitality score of the time BDI was conducted. When these two indicators were compared, we found there was a negative correlation between BDI and the vitality score. Additionally, it was a useful method to identify a test subject with a high BDI score.

Voice disability index using pitch rate

We have developed pitch rate (pitch detection ratio) in the speech segment as a voice disability index for use in free form speech data from smartphones. Its performance was compared with jitter, shimmer, and harmonic to noise ratio (HNR) indices. Conventionally, the performance of these indices is evaluated using long vowel sounds, but in this study we used the read speech of “Rainbow passage”. As a result, although jitter, shimmer, HNR displayed good performance for long vowel sounds, for the read speech data, performance of shimmer and HNR was significantly low. However, although jitter performed comparatively well for read speech, pitch rate was found to be a better indicator for voice disability between patients and healthy individuals.

Study on Depression Evaluation Indicator in the Elderly using Sensibility Technology.

Depression is important issue with aging of global population. Previously we have proposed a method to evaluate the mental health status of a person by his or her voice and developed a smartphone-based system to monitor mental health from voice during a call. Although the system has excellent continuous monitoring capability, it has not enough specificity for screening. Therefore, in this study we propose an evaluation indicator to assess depression status in the elderly, based on multivariate analysis using the emotional components of the voice data collected in the aforementioned system and the BDI score. The voice emotion data on subjects was divided into two groups according to BDI score, one where doctor’s diagnosis was deemed necessary and the other not so. A significant difference between the two groups was observed in t-test when the mean of the evaluation indicator estimated using data of each group and applying logistic regression prediction equation was compared. Moreover, a performance corresponding to AUC of approximately 0 .75 was achieved in the ROC curve of the derived evaluation indicator. The results suggest that a new method to evaluate depression using voice has likely been developed.

Validity of a Voice-Based Evaluation Method for Effectiveness of Behavioural Therapy

In this study, we used General Health Questionnaire 30 (GHQ30) and voice to evaluate the stress reduction effect of a stress resilience program, and examined the validity of stress evaluation by voice. We divided the subjects who participated in the program into two groups by the number of training sessions. The results showed a stress-reduction effect only in the group with more training sessions (more than 13 sessions) for both GHQ30 and voice-based indexes. Moreover, both indexes showed a highly negative correlation between the pre-training value and the difference between the post-training and pre-training values. This implies that the effect of the training is more evident for subjects with higher stress levels. The voice-based evaluation showed trends similar to those displayed by GHQ30.

The Influence of the Voice Acquisition Method to the Mental Health State Estimation Based on Vocal Analysis

Mental health disorders have become a social problem, and countermeasures are thus required. Previously, the authors developed the MIMOSYS (Mind Monitoring System) algorithm to evaluate an individual’s mental health state using their voice. An individual’s mental health state is detected using aspects of emotions that are present in their voice; however, since emotions change subtly, influence to emotions will be concerned owing to voice acquisition methods such as a natural conversation with someone else or reading fixed phrases. The aim of this research was to evaluate the influence of the type of voice acquisition method on the estimation of emotion and mental health state using the vocal analysis in MIMOSYS. In the experiments, we collected emotions and MIMOSYS analysis results from voice recordings during calls and during readings of fixed phrases from the application for over two weeks. In addition, the Beck’s Depression Inventory (BDI) test was used to evaluate participants’ subjective depression levels at the beginning of the experiment. In the evaluation, we analyzed recordings of calls and readings of fixed phrases for the participants in the normal range of the BDI test. Results indicated that the expression of emotions was suppressed in the recordings of readings of fixed phrases when compared to the recordings of calls, and the analysis result by MIMOSYS tended to be lower. Consequently, when measuring an individual’s mental health state from their voice, it may be necessary to match the type of voice acquisition methods or correct the estimations according to acquisition method.

Feasibility Study of Evaluation of Therapeutic Effect for Sleep Apnea Syndrome Using Mental Healthiness Evaluated from Voice

Dealing with sleep apnea syndrome (SAS) is important because of its social burden, however current standard diagnosis requires a costly examination (polysomnography, PSG). Therefore, strong demand for easy screening methods for SAS exists. There is also a need for evaluation of therapeutic effect by continuous positive airway pressure (CPAP). Because CPAP requires adjustment of parameters (titration), or CPAP used inadequately will not improve a patient’s symptom. Considering quality of life, evaluation of therapeutic effect requires monitoring of mental and physical conditions in daytime. We already reported that mental healthiness evaluated from voice, called “vitality,” showed some correlation with severity of SAS. In this study we examined feasibility of vitality as an index of therapeutic effect by CPAP. We recorded voices from subjects when they were examined by PSG for first diagnosis and titration, before and after each examination. Then we evaluated vitality of the subjects at the recording. The subjects were categorized into two groups; subjects of a group started using CPAP after titration, and subjects of another group started using CPAP before titration. As results, direction of change in vitality in the former group varied by subjects at titration, while vitality in the latter group showed tendency of improvement. Within the latter group, the change in vitality tends to get larger as usage rate of CPAP before titration is higher. This result suggests that vitality has a potential for an easy method to evaluate therapeutic effect for SAS, and that diligence of CPAP usage is important for effective treatment.

Study on Indicators for Depression in the Elderly Using Voice and Attribute Information

As the age of the human population increases worldwide, depression in elderly patients has become a problem in medical care. In this study, we analyzed voice-emotion component data, attribute data, and Beck Depression Inventory (BDI) scores by multivariate analysis, particularly in the elderly, and proposed evaluation indicators for estimating the state of depression of elderly patients. We divided the data into two groups according to BDI scores: a state of depression and the absence of this state. The labels distinguishing the two groups were dependent variables, while the voice-emotion component and attribute information were set as independent variables, and we performed logistic regression analysis on the data. We obtained a prediction model with significantly sufficient fitness. In the receiver operating characteristic curve for the proposed depression evaluation indicator, a sorting performance with an area under the curve of approximately 0.93 was obtained.