Angela Sweeney

A model for developing outcome measures from the perspectives of mental health service users.

It is becoming increasingly recognized that conventionally derived outcome measures in mental health research are problematic. This is both because of the methodology used and because a ‘good’ outcome is framed from the perspective of clinicians and researchers. This paper describes a methodology for developing outcome measures for use in large studies entirely from the perspective of mental health service users. It is a mixed methods model starting with a participatory and qualitative methodology and proceeding to psychometric testing. At all stages, the researchers are themselves mental health service users. In the first phase of the model, focus groups are convened comprising people who have received the treatment or service being measured. The focus groups meet twice resulting in a draft mixed-methods questionnaire devised from thematic analysis of the focus group data. This is then taken to expert panels, again comprising individuals who have received the treatment or service being evaluated for refinement. Following this, a feasibility study is conducted with N ∼ 50 participants and changes made iteratively to the questionnaire in light of feedback. The final measure is subject to psychometric testing both to ensure it is robust and to explore similarities and differences with conventionally derived measures.

CHoice of Outcome In Cbt for psychosEs (CHOICE): The Development of a New Service User–Led Outcome Measure of CBT for Psychosis

Outcome measures for cognitive behavior therapy for psychosis (CBTp) have been derived from pharmacological studies, focusing on symptom change rather than outcomes such as distress or fulfillment. This study presents the development and psychometric properties of a new outcome measure (CHoice of Outcome In Cbt for psychosEs [CHOICE]), which reflects more strongly the aims of CBTp and the priorities of service users. Service users who had received CBTp participated in focus groups to discuss their outcome priorities, using a topic guide generated by a panel of experts in CBTp. A qualitative thematic analysis was undertaken to reach consensus on themes and generate items. Response scales were constructed for 3 dimensions: severity, satisfaction, and importance. The resulting questionnaire was piloted with service users who had not received CBTp, stratified by service type, ethnicity, and first language to ensure that it was user friendly and applicable prior to CBTp. The psychometric properties of the measure were then examined in a sample of 152 service users. Twenty-four items, and 2 of the dimensions (severity and satisfaction), were retained in the final measure. A factor analysis revealed a single psychological recovery factor interspersed throughout with both CBTp and recovery items. Test-retest reliability, construct validity, and sensitivity to change following CBTp were confirmed. The CHOICE measure is unique in being the first psychometrically adequate service user-led outcome measure of CBTp. It provides the opportunity to examine the evidence base for CBTp with an assessment approach that prioritizes service user definitions of recovery and CBT aims.

Developing a user-generated measure of continuity of care: brief report

Objective:  This paper describes a measure of continuity of care, establishes its reliability and tests it in a field trial sample for evidence of its validity. In contrast to others, this measure has been generated from the perspectives of service users. As continuity of care is a concern particularly for those with severe mental illness, we have confined our work to this population group.

Hearing the voices of service user researchers in collaborative qualitative data analysis: the case for multiple coding.

Health research is frequently conducted in multi‐disciplinary teams, with these teams increasingly including service user researchers. Whilst it is common for service user researchers to be involved in data collection – most typically interviewing other service users – it is less common for service user researchers to be involved in data analysis and interpretation. This means that a unique and significant perspective on the data is absent.

Understanding service user-defined continuity of care and its relationship to health and social measures: a cross-sectional study

BackgroundDespite the importance of continuity of care [COC] in contemporary mental health service provision, COC lacks a clearly agreed definition. Furthermore, whilst there is broad agreement that definitions should include service users’ experiences, little is known about this. This paper aims to explore a new construct of service user-defined COC and its relationship to a range of health and social outcomes.MethodsIn a cross sectional study design, 167 people who experience psychosis participated in structured interviews, including a service user-generated COC measure (CONTINU-UM) and health and social assessments. Constructs underlying CONTINU-UM were explored using factor analysis in order to understand service user-defined COC. The relationships between the total/factor CONTINU-UM scores and the health and social measures were then explored through linear regression and an examination of quartile results in order to assess whether service user-defined COC is related to outcome.ResultsService user-defined COC is underpinned by three sub-constructs: preconditions, staff-related continuity and care contacts, although internal consistency of some sub-scales was low. High COC as assessed via CONTINU-UM, including preconditions and staff-related COC, was related to having needs met and better therapeutic alliances. Preconditions for COC were additionally related to symptoms and quality of life. COC was unrelated to empowerment and care contacts unrelated to outcomes. Service users who had experienced a hospital admission experienced higher levels of COC. A minority of service users with the poorest continuity of care also had high BPRS scores and poor quality of life.ConclusionsService-user defined continuity of care is a measurable construct underpinned by three sub-constructs (preconditions, staff-related and care contacts). COC and its sub-constructs demonstrate a range of relationships with health and social measures. Clinicians have an important role to play in supporting service users to navigate the complexities of the mental health system. Having experienced a hospital admission does not necessarily disrupt the flow of care. Further research is needed to test whether increasing service user-defined COC can improve clinical outcomes. Using CONTINU-UM will allow researchers to assess service users’ experiences of COC based on the elements that are important from their perspective.

Evaluation of a community awareness programme to reduce delays in referrals to early intervention services and enhance early detection of psychosis

BackgroundReducing treatment delay and coercive pathways to care are accepted aims for Early Intervention Services (EIS) for people experiencing first episode psychosis but how to achieve this is unclear. A one-year community awareness programme was implemented in a London EIS team, targeting staff in non-health service community organisations. The programme comprised psycho-educational workshops and EIS link workers, and offering direct referral routes to EIS. Its feasibility and its impact on duration of untreated psychosis and pathways to EIS were evaluated.MethodsEvaluation comprised: pre and post questionnaires with workshop participants assessing knowledge and attitudes to psychosis and mental health services; and a comparison of new service users’ “service DUP”(time from first psychotic symptom to first contact with EIS) and pathways to care in the intervention year and preceding year. Focus groups sought stakeholders’ views regarding the benefits and limitations of the programme and what else might promote help-seeking.Results41 workshops at 36 community organisations were attended by 367 staff. 19 follow up workshops were conducted and 16 services were allocated an EIS link worker. Participants’ knowledge and attitudes to psychosis and attitudes to mental health services improved significantly following workshops. In the year of the intervention, only 6 of 110 new service users reached EIS directly via community organisations. For all new referrals accepted by EIS, in the intervention year compared to the previous year, there was no difference in mean or median service DUP. A clear impact on pathways to care could not be discerned. Stakeholders suggested that barriers to referral remained. These included: uncertainty about the signs of early psychosis, disengagement by young people when becoming unwell, and worries about stigma or coercive treatment from mental health services. More general, youth focused, mental health services were proposed.ConclusionsThe community awareness programme did not reduce treatment delays for people experiencing first episode psychosis. Further research is needed regarding effective means to reduce duration of untreated psychosis. Although EIS services are guided to promote access through community engagement, this may not be an effective use of their limited resources.Trial registrationCurrent Controlled Trial ISRCTN98260910 Registered 19th May 2010.