Validating Screening Questionnaires for Internalizing and Externalizing Disorders against Clinical Interviews in 8 to 17-Year-Old Syrian Refugee Children

Abstract

Syrian children affected by the civil war are at increased risk of mental health problems, including depression, anxiety, post-traumatic stress disorder (PTSD), and externalizing behaviour problems. Screening questionnaires are designed to identify individual children who require further assessment and treatment, and also estimate the need for mental health services in a population. However, few questionnaires have been rigorously tested in this population. This study examined the reliability and validity of questionnaires for depression (Center for Epidemiological Studies Depression Scale for Children, CES-DC, self-report, 10item version), anxiety (Screen for Child Anxiety Related Emotional Disorders, SCARED, self-report, 18-item version), PTSD (Child PTSD Symptom Scale, CPSS, self-report), and internalizing and externalizing behavior problems (Strengths and Difficulties Questionnaire, SDQ, parent-report version) in a population sample of 8-17 year old Syrian children living in Informal Tented Settlements (ITS) in the Beqaa region of Lebanon. In addition, several ways of measuring functional impairment due to mental health problems were compared. These included selfand parent-report questionnaires (World Health Organization Disability Assessment Schedule, WHODAS-Child; SDQ Impact supplement, parent-report only) and an interviewer rating of severity (Clinical Global Impression–severity, CGI-s). Questionnaires were translated into Arabic and modified based on pilot testing with Syrian children. Responses from N=1006 children and caregivers were used for analysis, a subset of whom had additional clinical interview data (MINI KID + clinical judgement; N=119). The self-report questionnaires showed good internal consistency reliability with alpha>.80, though the parent-report SDQ and WHODAS-Child fell below this level. In terms of validity, the SDQ externalizing scale performed well in differentiating children with conduct problems from those without and it was possible to achieve a fair balance between sensitivity (82%) and specificity (71%). The CES-DC, CPSS, SDQ total difficulties, and WHODAS-Child (selfreport) achieved an acceptable level of validity, though it was harder to achieve a good balance between sensitivity and specificity. In most cases, at least 50% of those screening positive were false positives, meaning that a more in-depth follow up assessment would be required if these tools were used as screeners in a clinical setting. Furthermore, correction would be needed if used to estimate prevalence rates for mental disorders in this population. There was moderate convergent validity between measures of functional impairment, with self-report WHODAS-Child showing greater agreement with interviewer ratings when compared to parent-report measures (WHODAS and SDQ Impact). Measuring functional impairment and distress due to mental health problems should help to differentiate children with clinically significant mental health problems from those with subthreshold problems; however, more work will be required to establish how helpful the tools used here are in achieving that aim. Overview of SCARED: MENAT Measurement Library Criteria SCARED should have high evidence of internal consistency and diagnostic accuracy for use as a screening measure in clinical settings or as an epidemiological research measure. In testing with Syrian refugee children in Lebanon, SCARED subscales had only moderate evidence of internal consistency and the total scores showed modest ability to discriminate between children with and without anxiety disorders. This version of the SCARED is not currently recommended for the purposes of screening for or estimating prevalence of anxiety disorders in the Syrian refugee context. If interested in use of the measure, please contact the developer for further information. Criteria Indicators Notes Purpose Screening Requires high internal consistency; strong evidence of validity, including diagnostic accuracy, sensitivity, and specificity. May prioritize evidence of sensitivity. Epidemiological research Requires high internal consistency; strong evidence of validity, including diagnostic accuracy, sensitivity, & specificity. May prioritize a balance of sensitivity & specificity or the number of false positives & false negatives, although the latter is sample specific. Empirical evidence overall # of types of evidence 7 % of evidence meets criteria1 10% (green only); 50% (yellow and green) Evidence fit for purpose Yes for internal consistency and validity Confidence in evidence Sampling method Full sample: Purposive cluster sampling Clinical interview sample: Purposive sampling and use of sample weights to represent full sample Sample size Full sample: Large (N = 1006) Clinical interview sample: Small (N = 119) Missing data Small amount of missing data Rigor of method High Revisions Clear guidance on what to adjust/refine Yes 1Does not include sensitivity, specificity, positive predictive values (PPV) or negative predictive values (NPV) This technical working paper was developed by Fiona S. McEwen, Patricia Moghames, Tania Bosqui, Vanessa Kyrillos, Nicolas Chehade, Stephanie Saad, Diana Abdul Rahman, Cassandra Popham, Dahlia Saab, Georges Karam, Elie Karam, & Michael Pluess as members of the 3EA | MENAT Measurement Consortium, and reviewed by NYU Global Ties for Children. Suggested citation: McEwen, F. S., Moghames, P., Bosqui, T., Kyrillos, V., Chehade, N., Saad, S., Abdul Rahman, D., Popham, C., Saab, D., Karam, G., Karam, E., & Pluess, M. (2020, January). Validating screening questionnaires for internalizing and externalizing disorders against clinical interviews in 8-17 year-old Syrian refugee children. Technical working paper. London, UK: QMUL. Overview of SCARED Empirical Results Scales / subscales Is scale internally consistent? Is there evidence on the internal structure of the scale? Does scale predict disorder better than chance? (AUC) What % of cases are detected? (sensitivity) What % of noncases are identified? (specificity) What % of positive results true? (PPV) What % of negative results true? (NPV) Are there any other concerns?