Jessica Deighton

Measures of Social and Emotional Skills for Children and Young People A Systematic Review

This study presents the findings of a systematic review of measures of social and emotional skills for children and young people. The growing attention to this area in recent years has resulted in the development of a large number of measures to aid in the assessment of children and young people. These measures vary on a number of variables relating to implementation characteristics and psychometric properties. The methodology of the review followed the general principles of systematic reviewing, such as systematic search of databases, the adoption of predetermined set of inclusion and exclusion criteria, and a multistage filtering process. The review process resulted in the retention of 12 measures, which are presented and discussed in relation to key issues in this area, including difficulties with the underlying theory and frameworks for social and emotional skills, inconsistent terminology, the scope and distinctiveness of available measures, and more practical issues such as the type of respondent, location, and purpose of measurement.

Patient-reported outcomes in child and adolescent mental health services (CAMHS): Use of idiographic and standardized measures

Background There is increasing emphasis on use of patient-reported outcome measures (PROMs) in mental health but little research on the best approach, especially where there are multiple perspectives. Aims To present emerging findings from both standardized and idiographic child-, parent- and clinician-rated outcomes in child and adolescent mental health services (CAMHS) and consider their correlations. Method Outcomes were collected in CAMHS across the UK. These comprised idiographic measures (goal-based outcomes) and standardized measures (practitioner-rated Childrens Global Assessment Scale; child- and parent-rated Strengths and Difficulties Questionnaire). Results There was reliable positive change from the beginning of treatment to later follow-up according to all informants. Standardized clinician function report was correlated with standardized child difficulty report (r  =  − 0.26), standardized parent report (r  =  − 0.28) and idiographic joint client-determined goals (r  =  0.38) in the expected directions. Conclusions These results suggest that routine outcome monitoring is feasible, and suggest the possibility of using jointly agreed idiographic measures alongside particular perspectives on outcome as part of a PROMs approach.

A general psychopathology factor in early adolescence

BACKGROUND Recently, a general psychopathology dimension reflecting common aspects among disorders has been identified in adults. This has not yet been considered in children and adolescents, where the focus has been on externalising and internalising dimensions. AIMS To examine the existence, correlates and predictive value of a general psychopathology dimension in young people. METHOD Alternative factor models were estimated using self-reports of symptoms in a large community-based sample aged 11-13.5 years (N = 23 477), and resulting dimensions were assessed in terms of associations with external correlates and future functioning. RESULTS Both a traditional two-factor model and a bi-factor model with a general psychopathology bi-factor fitted the data well. The general psychopathology bi-factor best predicted future psychopathology and academic attainment. Associations with correlates and factor loadings are discussed. CONCLUSIONS A general psychopathology factor, which is equal across genders, can be identified in young people. Its associations with correlates and future functioning indicate that investigating this factor can increase our understanding of the aetiology, risk and correlates of psychopathology.

The Development of a School-Based Measure of Child Mental Health.

Early detection of child mental health problems in schools is critical for implementing strategies for prevention and intervention. The development of an effective measure of mental health and well-being for this context must be both empirically sound and practically feasible. This study reports the initial validation of a brief self-report measure for child mental health suitable for use with children as young as 8 years old (“Me & My School” [M&MS]). After factor analysis, and studies of measurement invariance, 2 subscales emerged: emotional difficulties and behavioral difficulties. These 2 subscales were highly correlated with corresponding constructs of the Strengths and Difficulties Questionnaire (SDQ) and showed correlations with attainment, deprivation, and educational needs similar to ones obtained between these demographic measures and the SDQ. Results suggest that this school-based self-report measure is psychometrically sound, and has the potential of contributing to school mental health surveys, evaluation of interventions, and recognition of mental health problems within schools.

Measuring mental health and wellbeing outcomes for children and adolescents to inform practice and policy: a review of child self-report measures

There is a growing appetite for mental health and wellbeing outcome measures that can inform clinical practice at individual and service levels, including use for local and national benchmarking. Despite a varied literature on child mental health and wellbeing outcome measures that focus on psychometric properties alone, no reviews exist that appraise the availability of psychometric evidence and suitability for use in routine practice in child and adolescent mental health services (CAMHS) including key implementation issues. This paper aimed to present the findings of the first review that evaluates existing broadband measures of mental health and wellbeing outcomes in terms of these criteria. The following steps were implemented in order to select measures suitable for use in routine practice: literature database searches, consultation with stakeholders, application of inclusion and exclusion criteria, secondary searches and filtering. Subsequently, detailed reviews of the retained measures’ psychometric properties and implementation features were carried out. 11 measures were identified as having potential for use in routine practice and meeting most of the key criteria: 1) Achenbach System of Empirically Based Assessment, 2) Beck Youth Inventories, 3) Behavior Assessment System for Children, 4) Behavioral and Emotional Rating Scale, 5) Child Health Questionnaire, 6) Child Symptom Inventories, 7) Health of the National Outcome Scale for Children and Adolescents, 8) Kidscreen, 9) Pediatric Symptom Checklist, 10) Strengths and Difficulties Questionnaire, 11) Youth Outcome Questionnaire. However, all existing measures identified had limitations as well as strengths. Furthermore, none had sufficient psychometric evidence available to demonstrate that they could reliably measure both severity and change over time in key groups. The review suggests a way of rigorously evaluating the growing number of broadband self-report mental health outcome measures against standards of feasibility and psychometric credibility in relation to use for practice and policy.

Satisfaction in child and adolescent mental health services: translating users' feedback into measurement.

The present research addressed gaps in our current understanding of validity and quality of measurement provided by patient reported experience measures. We established the psychometric properties of a freely available experience of service questionnaire (ESQ), based on responses from 7,067 families of patients across 41 UK providers of child and adolescent mental health services, using the two-level latent trait modeling. Responses to the ESQ were subject to strong ‘halo’ effects, which were thought to represent the overall positive or negative affect towards one’s treatment. Two strongly related constructs measured by the ESQ were interpreted as specific aspects of global satisfaction, namely satisfaction with care, and with environment. The Care construct was sensitive to differences between less satisfied patients, facilitating individual and service-level problem evaluation. The effects of nesting within service providers were strong, with parental reports being the most reliable source of data for the between-provider comparisons. We provide a scoring protocol for converting the hand-scored ESQ to the model-based population-referenced scores with supplied standard errors, which can be used for benchmarking services as well as individual evaluations.

From ‘reckless’ to ‘mindful’ in the use of outcome data to inform service-level performance management: perspectives from child mental health

In the recent report on patient safety in the National Health Service (NHS) in England, Don Berwick calls on the NHS to align the necessity for increased ‘accountability’ with the necessity to ‘abandon blame as a tool’ in order to develop a ‘transparent learning culture’.1 Sir Bruce Keogh, Medical Director NHS, and colleagues’ recent analysis of outlier hospitals based on mortality data marks a key step on this journey, but has led to high-profile debate about the risk of possible ‘reckless’ (Sir Bruce Keoghs term) use of data if appropriate parameters are not established.2 ,3 If these and other equivalent proxies for outcomes are to be used safely and effectively to support performance management and quality improvement in the ways envisioned by both Keogh and Berwick, it is crucial to establish clearly agreed operational procedures. Drawing on our experience of collecting and interpreting outcome data in the challenging context of child mental health across the UK, we suggest adoption of a MINDFUL framework involving consideration of m ultiple perspectives, i nterpreting differences in the light of current evidence base, focus on n egative differences when triangulated with other data, d irected discussions based on ‘what if this were a true difference’ (employing the 75–25% rule), use of f unnel plots as a starting point to consider outliers, appreciation of u ncertainty as a key contextual reality and the use of l earning collaborations to support appropriate implementation and action strategies. Any attempt to measure ‘impact’ of a service using a given ‘outcome’ is complex. The Keogh report acknowledges: “two different measures of mortality, HSMR [Hospital Standardised Mortality Ratio] and SHMI [Summary Hospital Level Mortality Indicator] generated two completely different lists of outlier trusts.” This was ‘solved’ by using both lists, but with a suggestion to move to …

Embedding mental health support in schools: learning from the Targeted Mental Health in Schools (TaMHS) national evaluation

The Targeted Mental Health in Schools (TaMHS) programme was a nationwide initiative that funded mental health provision in schools for pupils at risk of or already experiencing mental health problems. The implementation, impact and experience of this programme was evaluated using quantitative and qualitative methodology involving three main studies: (1) a 1-year RCT involving 8658 8–10 year olds and 6583 11–13 year olds, (2) a 3-year longitudinal study involving 3346 8–10 year olds and 2647 11–13 year olds and (3) qualitative interviews with 26 TaMHS workers, 31 school staff, 15 parents and 60 pupils. The RCT demonstrated that TaMHS led to reductions in behaviour problems but not emotional problems for 8–10 year olds. No impact was found for 11–13 year olds. The effects on behaviour problems in primary school were enhanced by the provision of evidence based self-help materials, but not by other area level support. The longitudinal study found information giving and good inter-agency working correlated with more positive outcomes for behavioural problems in secondary schools. The qualitative findings indicated that TaMHS was well received by all groups, though challenges to its implementation were noted. Overall, findings indicate the utility of targeted mental health provision in schools, particularly in primary settings. The implications for implementation are discussed.

Feedback from Outcome Measures and Treatment Effectiveness, Treatment Efficiency, and Collaborative Practice: A Systematic Review

Due to recent increases in the use of feedback from outcome measures in mental health settings, we systematically reviewed evidence regarding the impact of feedback from outcome measures on treatment effectiveness, treatment efficiency, and collaborative practice. In over half of 32 studies reviewed, the feedback condition had significantly higher levels of treatment effectiveness on at least one treatment outcome variable. Feedback was particularly effective for not-on-track patients or when it was provided to both clinicians and patients. The findings for treatment efficiency and collaborative practice were less consistent. Given the heterogeneity of studies, more research is needed to determine when and for whom feedback is most effective.

Systematic review of approaches to using patient experience data for quality improvement in healthcare settings

Objectives Explore how patient-reported experience measures (PREMs) are collected, communicated and used to inform quality improvement (QI) across healthcare settings. Design Systematic review. Setting Various primary and secondary care settings, including general practice, and acute and chronic care hospitals. Participants A full range of patient populations from (children through to the elderly) and staff (from healthcare practitioners to senior managers). Methods Scientific databases were searched (CINAHL, PsycINFO, MEDLINE and Cochrane Libraries) as was grey literature. Qualitative and quantitative studies describing collection of PREM data and subsequent QI actions in any healthcare setting were included. Risk of bias was assessed using established criteria. Of 5312 initial hits, 32 full texts were screened, and 11 were included. Results Patient experience data were most commonly collected through surveys and used to identify small areas of incremental change to services that do not require a change to clinician behaviour (eg, changes to admission processes and producing educational materials). While staff in most studies reported having made effective improvements, authors struggled to identify what those changes were or the impact they had. Conclusions Findings suggest there is no single best way to collect or use PREM data for QI, but they do suggest some key points to consider when planning such an approach. For instance, formal training is recommended, as a lack of expertise in QI and confidence in interpreting patient experience data effectively may continue to be a barrier to a successful shift towards a more patient-centred healthcare service. In the context of QI, more attention is required on how patient experience data will be used to inform changes to practice and, in turn, measure any impact these changes may have on patient experience.