Nahathai Wongpakaran

A comparison of Cohen’s Kappa and Gwet’s AC1 when calculating inter-rater reliability coefficients: a study conducted with personality disorder samples

BackgroundRater agreement is important in clinical research, and Cohen’s Kappa is a widely used method for assessing inter-rater reliability; however, there are well documented statistical problems associated with the measure. In order to assess its utility, we evaluated it against Gwet’s AC1 and compared the results.MethodsThis study was carried out across 67 patients (56% males) aged 18 to 67, with a mean SD of 44.13 ± 12.68 years. Nine raters (7 psychiatrists, a psychiatry resident and a social worker) participated as interviewers, either for the first or the second interviews, which were held 4 to 6 weeks apart. The interviews were held in order to establish a personality disorder (PD) diagnosis using DSM-IV criteria. Cohen’s Kappa and Gwet’s AC1 were used and the level of agreement between raters was assessed in terms of a simple categorical diagnosis (i.e., the presence or absence of a disorder). Data were also compared with a previous analysis in order to evaluate the effects of trait prevalence.ResultsGwet’s AC1 was shown to have higher inter-rater reliability coefficients for all the PD criteria, ranging from .752 to 1.000, whereas Cohen’s Kappa ranged from 0 to 1.00. Cohen’s Kappa values were high and close to the percentage of agreement when the prevalence was high, whereas Gwet’s AC1 values appeared not to change much with a change in prevalence, but remained close to the percentage of agreement. For example a Schizoid sample revealed a mean Cohen’s Kappa of .726 and a Gwet’s AC1of .853 , which fell within the different level of agreement according to criteria developed by Landis and Koch, and Altman and Fleiss.ConclusionsBased on the different formulae used to calculate the level of chance-corrected agreement, Gwet’s AC1 was shown to provide a more stable inter-rater reliability coefficient than Cohen’s Kappa. It was also found to be less affected by prevalence and marginal probability than that of Cohen’s Kappa, and therefore should be considered for use with inter-rater reliability analysis.

Reliability and Validity of the Multidimensional Scale of Perceived Social Support (MSPSS): Thai Version.

This study examines the Thai version of the Multidimensional Scale of Perceived Social Support (MSPSS) for its psychometric properties. Methods: In total 462 participants were recruited - 310 medical students from Chiang Mai University and 152 psychiatric patients, and they completed the Thai version of the MSPSS, the State Trait Anxiety Inventory (STAI), the Rosenberg Self-Esteem Scale (RSES) and the Thai Depression Inventory (TDI). Test-retest reliability was conducted over a four week period. Results: Factor analysis produced three-factor solutions for both patient (PG) and student groups (SG), and overall the model demonstrated adequate fit indices. The mean total score and the sub-scale score for the SG were statistically higher than those in the PG, except for ‘Significant Others’. The internal consistency of the scale was good, with a Cronbach’s alpha of 0.91 for the SG and 0.87 for the PG. After a four week retest for reliability exercise, the intra-class correlation coefficient (ICC) was found to be 0.84. The Thai-MSPSS was found to have a negative correlation with the STAI and the TDI, but was positively correlated with the RSES. Conclusion: The Thai MSPSS is a reliable and valid instrument to use.

Benzodiazepine prescribing behaviour and attitudes: a survey among general practitioners practicing in northern Thailand.

BackgroundOver-prescribing of benzodiazepines appears common in many countries, a better understanding of prescribing practices and attitudes may help develop strategies to reduce prescribing. This study aimed to evaluate benzodiazepine prescribing behaviour and attitudes in general practitioners practising in Chiang Mai and Lampoon, Thailand.MethodsQuestionnaire survey of general practitioners in community hospitals, to estimate: i) use of benzodiazepines for anxiety/insomnia, panic disorder, depression, essential hypertension, and uncomplicated low back pain and ii) views on the optimal duration of benzodiazepine use.ResultsFifty-five of 100 general practitioners returned the completed questionnaires. They reported use of benzodiazepines for anxiety/insomnia (n = 51, 93%), panic disorder (n = 43, 78%), depression (n = 26, 43%), essential hypertension (n = 15, 27 %) and uncomplicated low back pain (n = 10, 18%). Twenty-eight general practitioners would prescribe benzodiazepines for non-psychiatric conditions, 17 for use as muscle relaxants. Seventy-five per cent, 62% and 29% of the general practitioners agreed or totally agreed with the use of benzodiazepines for insomnia, anxiety and depression, respectively. Practitioners agreed that prescribing should be less than one week (80%); or from 1 week to 1 month (47%); or 1 to 4 months (16%); or 4 to 6 months (5%) or more than 6 months (2%). Twenty-five general practitioners (45%) accepted that they used benzodiazepines excessively in the past year.ConclusionA considerable proportion of general practitioners in Chiang Mai and Lampoon, Thailand inappropriately use benzodiazepines for physical illnesses, especially essential hypertension and uncomplicated low back pain. However, almost half of them thought that they overused benzodiazepines. General practitioners lack of time, knowledge and skills should be taken into account in improving prescribing behaviour and attitudes.

Prevalence of major depressive disorders and suicide in long-term care facilities: a report from northern Thailand.

Background:  Depression is a common mental illness among elderly Thais, but little is known about the occurrence of major depressive disorders (MDD) in long‐term care (LTC) facilities. This study investigated the prevalence of MDD and suicide risk among residents in an LTC facility in northern Thailand.

Diagnosing delirium in elderly Thai patients: utilization of the CAM algorithm.

BackgroundDelirium is a common illness among elderly hospitalized patients. However, under-recognition of the condition by non-psychiatrically trained personnel is prevalent. This study investigated the performance of family physicians when detecting delirum in elderly hospitalized Thai patients using the Thai version of the Confusion Assessment Method (CAM) algorithm.MethodsA Thai version of the CAM algorithm was developed, and three experienced Thai family physicians were trained in its use. The diagnosis of delirium was also carried out by four fully qualified psychiatrists using DSM-IV TR criteria, which can be considered the gold standard. Sixty-six elderly patients were assessed with MMSE Thai 2002, in order to evaluate whether they had dementia upon admission. Within three days of admission, each patient was interviewed separately by a psychiatrist using DSM-IV TR, and a family physician using the Thai version of the CAM algorithm, with both sets of interviewers diagnosing for delirium.ResultsThe CAM algorithm tool, as used by family physicians, demonstrated a sensitivity of 91.9% and a specificity of 100.0%, with a PPV of 100.0% and an NPV of 90.6%. Interrater agreement between the family physicians and the psychiatrists was good (Cohens Kappa = 0.91, p < 0.0001). The mean of the time the family physicians spent using CAM algorithm was significantly briefer than that of the psychiatrists using DSM-IV TR.ConclusionsFamily physicians performed well when diagnosing delirium in elderly hospitalized Thai patients using the Thai version of the CAM algorithm, showing that this measurement tool is suitable for use by non-psychiatrically trained personnel, being short, quick, and easy to administer. However, proper training on use of the algorithm is required.

Validity and reliability of the Thai version of the Confusion Assessment Method for the Intensive Care Unit (CAM-ICU).

Purpose The purpose of this study was to determine the validity and reliability of the Thai version of the Confusion Assessment Method for the Intensive Care Unit (CAM-ICU), when compared to the diagnoses made by delirium experts. Patients and methods This was a cross-sectional study conducted in both surgical intensive care and subintensive care units in Thailand between February–June 2011. Seventy patients aged 60 years or older who had been admitted to the units were enrolled into the study within the first 48 hours of admission. Each patient was randomly assessed as to whether they had delirium by a nurse using the Thai version of the CAM-ICU algorithm (Thai CAM-ICU) or by a delirium expert using the Diagnostic and Statistical Manual of Mental Disorders, Fourth Edition, Text Revision. Results The prevalence of delirium was found to be 18.6% (n=13) by the delirium experts. The sensitivity of the Thai CAM-ICU’s algorithms was found to be 92.3% (95% confidence interval [CI] =64.0%−99.8%), while the specificity was 94.7% (95% CI =85.4%−98.9%). The instrument displayed good interrater reliability (Cohen’s κ =0.81; 95% CI =0.64−0.99). The time taken to complete the Thai CAM-ICU was 1 minute (interquatile range, 1−2 minutes). Conclusion The Thai CAM-ICU demonstrated good validity, reliability, and ease of use when diagnosing delirium in a surgical intensive care unit setting. The use of this diagnostic tool should be encouraged for daily, routine use, so as to promote the early detection of delirium and its rapid treatment.

The Use of GDS-15 in Detecting MDD: A Comparison Between Residents in a Thai Long-Term Care Home and Geriatric Outpatients.

Background To assess the psychometric properties of the Thai version of the 15-item Geriatric Depression Scale (TGDS-15) when screening for major depression (MDD) among geriatric outpatients (GOs) and long-term care (LTC) home residents in Thailand. Methods This was a cross-sectional study of 156 geriatric outpatients and 81 LTC home residents. All 237 participants were given a Mini-Mental State Examination, a MDD diagnosis according to the Mini-International Neuropsychiatric Interview, and completed a TGDS-15 questionnaire. Sensitivity, specificity, overall accuracy, and positive and negative predictive values were calculated. A comparison between the two groups was carried out. Differential Item Functioning (DIF) using logistic regression and factor analytic study were also applied. Results Overall, 38.4% of the participants were found to have MDD. The TGDS-15 was found to perform better when used with the GOs than with the LTC home residents, revealing a sensitivity of 0.92 and a specificity of 0.87 in the GOs (cut-off score of ≥ 5), but a sensitivity of 100% and a specificity of 49% with the LTC home group (cut-off score of ≥ 8), when comparing only cognitively intact subjects. The negative predictive value (NPV) was very good for both groups, but the positive predictive value (PPV) for the GO group was much better than for those in the LTC group (83.3% vs. 31.2%). Seven uniform DIF items were found - 2 by gender and 4 by age. Cronbach’s alpha was higher for the GO group than for the LTC home residents. Factor analysis supported a two-factor solution, using the ‘depressed mood’ and ‘positive mood’ factors, which accounted for 46.55% of the total variance. Conclusions The TGDS-15 scale was effective at screening for MDD in elderly cognitively intact Thais, those in both GO and LTC settings, as the sensitivity and NPV were shown to be very good in both groups. However, in the LTC setting, the low specificity and PPV found leads to the need for a further assessment to be carried among the potentially depressed individuals, based on the GDS results. Taking the factor analytic study into account, a more suitable version of the GDS should be developed.

A Short Version of the Revised ‘Experience of Close Relationships Questionnaire’: Investigating Non-Clinical and Clinical Samples

Aim: This study seeks to investigate the psychometric properties of the short version of the revised ‘Experience of Close Relationships’ questionnaire, comparing non-clinical and clinical samples. Methods: In total 702 subjects participated in this study, of whom 531 were non-clinical participants and 171 were psychiatric patients. They completed the short version of the revised ‘Experience of Close Relationships’ questionnaire (ECR-R-18), the Perceived Stress Scale-10(PSS-10), the Rosenberg Self-Esteem Scale (RSES) and the UCLA Loneliness scale. A retest of the ECR-R-18 was then performed at four-week intervals. Then, confirmatory factor analyses were performed to test the validity of the new scale. Results: The ECR-R-18 showed a fair to good internal consistency (α 0.77 to 0.87) for both samples, and the test-retest reliability was found to be satisfactory (ICC = 0.75). The anxiety sub-scale demonstrated concurrent validity with PSS-10 and RSES, while the avoidance sub-scale showed concurrent validity with the UCLA Loneliness Scale. Confirmatory factor analysis using method factors yielded two factors with an acceptable model fit for both groups. An invariance test revealed that the ECR-R-18 when used on the clinical group differed from when used with the non-clinical group. Conclusions: The ECR-R-18 questionnaire revealed an overall better level of fit than the original 36 item questionnaire, indicating its suitability for use with a broader group of samples, including clinical samples. The reliability of the ECR-R- 18 might be increased if a modified scoring system is used and if our suggestions with regard to future studies are followed up.

The Group Cohesiveness Scale (GCS) for psychiatric inpatients.

PURPOSE This study aimed to examine the psychometric properties of the seven-item Group Cohesiveness Scale (GCS). DESIGN AND METHODS In total, 96 inpatients completed the GCS along with the Cohesion to Therapist Scale Questionnaire and the Group Benefit Questionnaire after participating in group therapy sessions. Construct and concurrent validities and internal consistency were analyzed. FINDINGS It yielded a Cronbachs alpha of .87, with a one-factor solution with excellent fit indices. A significant correlation was found between the GCSc, the Cohesion to Therapist Scale, and the Group Benefit Questionnaire. PRACTICE IMPLICATIONS The scale shows good internal consistency, and its brevity makes it suitable for use with psychiatric inpatients.

Social inhibition as a mediator of neuroticism and depression in the elderly

BackgroundA number of factors, such as demographics, cognitive function, personality and interpersonal relationship) play a role in late-life depression. This study investigates the influence of social inhibition on the inverse emotional stability (neuroticism) and depressive symptoms found in elderly Thai people.MethodsIn total, 123 elderly Thais aged 60 years of age or older were tested using the 64-item Inventory of Interpersonal Problems, Symptom Checklist-90, and the 16 Personality Factors Questionnaire. Hierarchical regression and path analyses were performed in order to identify the relationships among these variables.ResultsThe age of the participants ranged from 60 to 93 years old (mean = 71.7; SD = 6.2), and out of the group, 51.2% were male, 56.1% were married and 61.8% were on a low income. The average number of years spent in education among the participants was 7.6 (SD = 5.1). The variables found to be significantly associated with depression were age, intellect, social inhibition and possession of inverse emotional stability (neuroticism). Low levels of emotional stability were most strongly associated with depressive symptoms (standardized regression coefficients −0.29), but this effect was found to be reduced (mediated, to −0.26) by social inhibition. In total, 30% of the total variance could be explained by this model, and there was an excellent statistical fit.ConclusionsThe variables found to be significantly associated with depression were a younger age, as well as lower levels of intellectual skill, social inhibition and inversed emotional stability (neuroticism). It was found that a lack of emotional stability is, along with a younger age, the strongest predictor of depressive symptoms, but can be mediated by social inhibition.