Tinakon Wongpakaran

A comparison of Cohen’s Kappa and Gwet’s AC1 when calculating inter-rater reliability coefficients: a study conducted with personality disorder samples

BackgroundRater agreement is important in clinical research, and Cohen’s Kappa is a widely used method for assessing inter-rater reliability; however, there are well documented statistical problems associated with the measure. In order to assess its utility, we evaluated it against Gwet’s AC1 and compared the results.MethodsThis study was carried out across 67 patients (56% males) aged 18 to 67, with a mean SD of 44.13 ± 12.68 years. Nine raters (7 psychiatrists, a psychiatry resident and a social worker) participated as interviewers, either for the first or the second interviews, which were held 4 to 6 weeks apart. The interviews were held in order to establish a personality disorder (PD) diagnosis using DSM-IV criteria. Cohen’s Kappa and Gwet’s AC1 were used and the level of agreement between raters was assessed in terms of a simple categorical diagnosis (i.e., the presence or absence of a disorder). Data were also compared with a previous analysis in order to evaluate the effects of trait prevalence.ResultsGwet’s AC1 was shown to have higher inter-rater reliability coefficients for all the PD criteria, ranging from .752 to 1.000, whereas Cohen’s Kappa ranged from 0 to 1.00. Cohen’s Kappa values were high and close to the percentage of agreement when the prevalence was high, whereas Gwet’s AC1 values appeared not to change much with a change in prevalence, but remained close to the percentage of agreement. For example a Schizoid sample revealed a mean Cohen’s Kappa of .726 and a Gwet’s AC1of .853 , which fell within the different level of agreement according to criteria developed by Landis and Koch, and Altman and Fleiss.ConclusionsBased on the different formulae used to calculate the level of chance-corrected agreement, Gwet’s AC1 was shown to provide a more stable inter-rater reliability coefficient than Cohen’s Kappa. It was also found to be less affected by prevalence and marginal probability than that of Cohen’s Kappa, and therefore should be considered for use with inter-rater reliability analysis.

Reliability and Validity of the Multidimensional Scale of Perceived Social Support (MSPSS): Thai Version.

This study examines the Thai version of the Multidimensional Scale of Perceived Social Support (MSPSS) for its psychometric properties. Methods: In total 462 participants were recruited - 310 medical students from Chiang Mai University and 152 psychiatric patients, and they completed the Thai version of the MSPSS, the State Trait Anxiety Inventory (STAI), the Rosenberg Self-Esteem Scale (RSES) and the Thai Depression Inventory (TDI). Test-retest reliability was conducted over a four week period. Results: Factor analysis produced three-factor solutions for both patient (PG) and student groups (SG), and overall the model demonstrated adequate fit indices. The mean total score and the sub-scale score for the SG were statistically higher than those in the PG, except for ‘Significant Others’. The internal consistency of the scale was good, with a Cronbach’s alpha of 0.91 for the SG and 0.87 for the PG. After a four week retest for reliability exercise, the intra-class correlation coefficient (ICC) was found to be 0.84. The Thai-MSPSS was found to have a negative correlation with the STAI and the TDI, but was positively correlated with the RSES. Conclusion: The Thai MSPSS is a reliable and valid instrument to use.

Prevalence of major depressive disorders and suicide in long-term care facilities: a report from northern Thailand.

Background:  Depression is a common mental illness among elderly Thais, but little is known about the occurrence of major depressive disorders (MDD) in long‐term care (LTC) facilities. This study investigated the prevalence of MDD and suicide risk among residents in an LTC facility in northern Thailand.

Diagnosing delirium in elderly Thai patients: utilization of the CAM algorithm.

BackgroundDelirium is a common illness among elderly hospitalized patients. However, under-recognition of the condition by non-psychiatrically trained personnel is prevalent. This study investigated the performance of family physicians when detecting delirum in elderly hospitalized Thai patients using the Thai version of the Confusion Assessment Method (CAM) algorithm.MethodsA Thai version of the CAM algorithm was developed, and three experienced Thai family physicians were trained in its use. The diagnosis of delirium was also carried out by four fully qualified psychiatrists using DSM-IV TR criteria, which can be considered the gold standard. Sixty-six elderly patients were assessed with MMSE Thai 2002, in order to evaluate whether they had dementia upon admission. Within three days of admission, each patient was interviewed separately by a psychiatrist using DSM-IV TR, and a family physician using the Thai version of the CAM algorithm, with both sets of interviewers diagnosing for delirium.ResultsThe CAM algorithm tool, as used by family physicians, demonstrated a sensitivity of 91.9% and a specificity of 100.0%, with a PPV of 100.0% and an NPV of 90.6%. Interrater agreement between the family physicians and the psychiatrists was good (Cohens Kappa = 0.91, p < 0.0001). The mean of the time the family physicians spent using CAM algorithm was significantly briefer than that of the psychiatrists using DSM-IV TR.ConclusionsFamily physicians performed well when diagnosing delirium in elderly hospitalized Thai patients using the Thai version of the CAM algorithm, showing that this measurement tool is suitable for use by non-psychiatrically trained personnel, being short, quick, and easy to administer. However, proper training on use of the algorithm is required.

The Use of GDS-15 in Detecting MDD: A Comparison Between Residents in a Thai Long-Term Care Home and Geriatric Outpatients.

Background To assess the psychometric properties of the Thai version of the 15-item Geriatric Depression Scale (TGDS-15) when screening for major depression (MDD) among geriatric outpatients (GOs) and long-term care (LTC) home residents in Thailand. Methods This was a cross-sectional study of 156 geriatric outpatients and 81 LTC home residents. All 237 participants were given a Mini-Mental State Examination, a MDD diagnosis according to the Mini-International Neuropsychiatric Interview, and completed a TGDS-15 questionnaire. Sensitivity, specificity, overall accuracy, and positive and negative predictive values were calculated. A comparison between the two groups was carried out. Differential Item Functioning (DIF) using logistic regression and factor analytic study were also applied. Results Overall, 38.4% of the participants were found to have MDD. The TGDS-15 was found to perform better when used with the GOs than with the LTC home residents, revealing a sensitivity of 0.92 and a specificity of 0.87 in the GOs (cut-off score of ≥ 5), but a sensitivity of 100% and a specificity of 49% with the LTC home group (cut-off score of ≥ 8), when comparing only cognitively intact subjects. The negative predictive value (NPV) was very good for both groups, but the positive predictive value (PPV) for the GO group was much better than for those in the LTC group (83.3% vs. 31.2%). Seven uniform DIF items were found - 2 by gender and 4 by age. Cronbach’s alpha was higher for the GO group than for the LTC home residents. Factor analysis supported a two-factor solution, using the ‘depressed mood’ and ‘positive mood’ factors, which accounted for 46.55% of the total variance. Conclusions The TGDS-15 scale was effective at screening for MDD in elderly cognitively intact Thais, those in both GO and LTC settings, as the sensitivity and NPV were shown to be very good in both groups. However, in the LTC setting, the low specificity and PPV found leads to the need for a further assessment to be carried among the potentially depressed individuals, based on the GDS results. Taking the factor analytic study into account, a more suitable version of the GDS should be developed.

A Short Version of the Revised ‘Experience of Close Relationships Questionnaire’: Investigating Non-Clinical and Clinical Samples

Aim: This study seeks to investigate the psychometric properties of the short version of the revised ‘Experience of Close Relationships’ questionnaire, comparing non-clinical and clinical samples. Methods: In total 702 subjects participated in this study, of whom 531 were non-clinical participants and 171 were psychiatric patients. They completed the short version of the revised ‘Experience of Close Relationships’ questionnaire (ECR-R-18), the Perceived Stress Scale-10(PSS-10), the Rosenberg Self-Esteem Scale (RSES) and the UCLA Loneliness scale. A retest of the ECR-R-18 was then performed at four-week intervals. Then, confirmatory factor analyses were performed to test the validity of the new scale. Results: The ECR-R-18 showed a fair to good internal consistency (α 0.77 to 0.87) for both samples, and the test-retest reliability was found to be satisfactory (ICC = 0.75). The anxiety sub-scale demonstrated concurrent validity with PSS-10 and RSES, while the avoidance sub-scale showed concurrent validity with the UCLA Loneliness Scale. Confirmatory factor analysis using method factors yielded two factors with an acceptable model fit for both groups. An invariance test revealed that the ECR-R-18 when used on the clinical group differed from when used with the non-clinical group. Conclusions: The ECR-R-18 questionnaire revealed an overall better level of fit than the original 36 item questionnaire, indicating its suitability for use with a broader group of samples, including clinical samples. The reliability of the ECR-R- 18 might be increased if a modified scoring system is used and if our suggestions with regard to future studies are followed up.

The Group Cohesiveness Scale (GCS) for psychiatric inpatients.

PURPOSE This study aimed to examine the psychometric properties of the seven-item Group Cohesiveness Scale (GCS). DESIGN AND METHODS In total, 96 inpatients completed the GCS along with the Cohesion to Therapist Scale Questionnaire and the Group Benefit Questionnaire after participating in group therapy sessions. Construct and concurrent validities and internal consistency were analyzed. FINDINGS It yielded a Cronbachs alpha of .87, with a one-factor solution with excellent fit indices. A significant correlation was found between the GCSc, the Cohesion to Therapist Scale, and the Group Benefit Questionnaire. PRACTICE IMPLICATIONS The scale shows good internal consistency, and its brevity makes it suitable for use with psychiatric inpatients.

Social inhibition as a mediator of neuroticism and depression in the elderly

BackgroundA number of factors, such as demographics, cognitive function, personality and interpersonal relationship) play a role in late-life depression. This study investigates the influence of social inhibition on the inverse emotional stability (neuroticism) and depressive symptoms found in elderly Thai people.MethodsIn total, 123 elderly Thais aged 60 years of age or older were tested using the 64-item Inventory of Interpersonal Problems, Symptom Checklist-90, and the 16 Personality Factors Questionnaire. Hierarchical regression and path analyses were performed in order to identify the relationships among these variables.ResultsThe age of the participants ranged from 60 to 93 years old (mean = 71.7; SD = 6.2), and out of the group, 51.2% were male, 56.1% were married and 61.8% were on a low income. The average number of years spent in education among the participants was 7.6 (SD = 5.1). The variables found to be significantly associated with depression were age, intellect, social inhibition and possession of inverse emotional stability (neuroticism). Low levels of emotional stability were most strongly associated with depressive symptoms (standardized regression coefficients −0.29), but this effect was found to be reduced (mediated, to −0.26) by social inhibition. In total, 30% of the total variance could be explained by this model, and there was an excellent statistical fit.ConclusionsThe variables found to be significantly associated with depression were a younger age, as well as lower levels of intellectual skill, social inhibition and inversed emotional stability (neuroticism). It was found that a lack of emotional stability is, along with a younger age, the strongest predictor of depressive symptoms, but can be mediated by social inhibition.

Personality traits influencing somatization symptoms and social inhibition in the elderly.

Purpose Somatization is a common symptom among the elderly, and even though personality disorders have been found to be associated with somatization, personality traits have not yet been explored with regard to this symptom. The aim of this study is to investigate the relationship between personality traits and somatization, and social inhibition. Patients and methods As part of a cross-sectional study of a community sample, 126 elderly Thais aged 60 years or over completed self-reporting questionnaires related to somatization and personality traits. Somatization was elicited from the somatization subscale when using the Symptom Checklist SCL-90 instrument. Personality traits were drawn from the 16 Personality Factor Questionnaire and social inhibition was identified when using the inventory of interpersonal problems. In addition, path analysis was used to establish the influence of personality traits on somatization and social inhibition. Results Of the 126 participants, 51% were male, 55% were married, and 25% were retired. The average number of years in education was 7.6 (standard deviation =5.2). “Emotional stability” and “dominance” were found to have a direct effect on somatization, as were age and number of years in education, but not sex. Also, 35% of the total variance could be explained by the model, with excellent fit statistics. Dominance was found to have an indirect effect, via vigilance, on social inhibition, which was also influenced by number of years in education and emotional stability. Social inhibition was not found to have any effect on somatization, although hypothetically it should. Conclusion “Emotional stability”, “dominance”, and “vigilance”, as well as age and the number of years in education, were found to have an effect on somatization. Attention should be paid to these factors in the elderly with somatization.

level of agreement between self-rated and clinician-rated instruments when measuring major depressive disorder in the Thai elderly: a 1-year assessment as part of the ThAIsAD study

Purpose Whether self-reporting and clinician-rated depression scales correlate well with one another when applied to older adults has not been well studied, particularly among Asian samples. This study aimed to compare the level of agreement among measurements used in assessing major depressive disorder (MDD) among the Thai elderly and the factors associated with the differences found. Patients and methods This was a prospective, follow-up study of elderly patients diagnosed with MDD and receiving treatment in Thailand. The Mini International Neuropsychiatric Inventory (MINI), 17-item Hamilton Depression Rating Scale (HAMD-17), 30-item Geriatric Depression Scale (GDS-30), 32-item Inventory of Interpersonal Problems scale, Revised Experience of Close Relationships scale, ten-item Perceived Stress Scale (PSS-10), and Multidimensional Scale of Perceived Social Support were used. Follow-up assessments were conducted after 3, 6, 9, and 12 months. Results Among the 74 patients, the mean age was 68±6.02 years, and 86% had MDD. Regarding the level of agreement found between GDS-30 and MINI, Kappa ranged between 0.17 and 0.55, while for Gwet’s AC1 the range was 0.49 to 0.91. The level of agreement was found to be lowest at baseline, and increased during follow-up visits. The correlation between HAMD-17 and GDS-30 scores was 0.17 (P=0.16) at baseline, then 0.36 to 0.41 in later visits (P<0.01). The PSS-10 score was found to be positively correlated with GDS-30 at baseline, and predicted the level of disagreement found between the clinicians and patients when reporting on MDD. Conclusion The level of agreement between the GDS, MINI, and HAMD was found to be different at baseline when compared to later assessments. Patients who produced a low GDS score were given a high rating by the clinicians. An additional self-reporting tool such as the PSS-10 could, therefore, be used in such under-reporting circumstances.