Tim Coombs

Assessing the value of existing recovery measures for routine use in Australian mental health services

Object: The concept of recovery has been recognized as important in the treatment of mental illness. A number of specific instruments exist which are designed to: (i) measure recovery at an individual level; and (ii) assess the recovery orientation of services. The current review aimed to identify these and evaluate their potential for routine use in Australian public sector mental health services. Method: We identified potential instruments by drawing on existing reviews, searching MEDLINE and PsycINFO, and consulting with experts. We used a hierarchical criterion-based approach to assess whether given instruments might be candidates for measuring recovery in the Australian context. Results: We identified 33 instruments: 22 designed to measure individuals’ recovery and 11 designed to assess the recovery orientation of services (or providers). Four of the former (Recovery Assessment Scale; Illness Management and Recovery Scales; Stages of Recovery Instrument; Recovery Process Inventory) and four of the latter (Recovery Oriented Systems Indicators Measure; Recovery Self Assessment; Recovery Oriented Practices Index; Recovery Promotion Fidelity Scale) were identified as promising candidates for routine use in Australian public sector mental health services. Conclusions: Further work is required, however, to determine which, if any, might best be used for this purpose; the possibility that modifications to existing instruments or the development of new instruments might be required should not be ruled out. It might be desirable to invest in two instruments: one designed to measure individuals’ recovery and one designed to measure the recovery orientation of services. If Australia were to go down this path, it would make sense to align indicators in each as far as possible, and to ensure that they were consistent with existing endeavours aimed at monitoring and improving recovery-focused aspects of service quality.

Adverse Incidents in Acute Psychiatric Inpatient Units: Rates, Correlates and Pressures:

Objective: This paper reports findings from a multicentre service evaluation project conducted in acute psychiatric inpatient units in NSW, Australia. Overall rates of aggression, absconding and early readmission are reported, as well as length-of-stay profiles and associations between these outcomes and selected sociodemographic and clinical characteristics routinely collected by health services. Method: Data from the 11 participating units were collected for a 12month period from multiple sources, including electronic medical records, routine clinical modules, incident forms, and shift based project-specific logs. For the current analyses, two admission-level datasets were used, comprising aggregated patient-level events (n=3242 admissions) and basic sociodemographic, clinical, admission and discharge information (n=5546 admissions by 3877 patients). Results: The participating units were under considerable strain: 23.3% of admissions were high acuity; 60.4% had previous hospital stays; 47.6% were involuntary; 25–30% involved adverse incidents; bed occupancy averaged 88.4%; median length of stay was 8days (mean=14.59days); and 17.4% had a subsequent early readmission. Reportable aggressive incidents (11.2% of admissions) were intermittent (averaging 0.55 incidents per month per occupied bed) and associated with younger age, personality disorder, less serious aggression, longer periods of hospitalization, and subsequent early readmission. Less serious aggressive incidents (15.0% of admissions) were maximal in the first 24h (averaging 3.73 incidents per month per occupied bed) and associated with younger age, involuntary status, bipolar and personality disorders, the absence of depression, and longer hospital stays. Absconding (15.7% of admissions) peaked in the second week following admission and was associated with drug and alcohol disorder, younger age, and longer periods of hospitalization. Conclusions: By examining relationships between a core set of risk factors and multiple short-term outcomes, we were able to identify several important patterns, which were suggestive of the need for a multi-level approach to intervention, shifting from a risk management focus during the early phase of hospitalization to a more targeted, therapeutic approach during the later phase. But the latter approach may not be achievable under current circumstances with existing resources.

Enhancing medication adherence: clinician outcomes from the Medication Alliance training program

Objective: Patient nonadherence to psychiatric medication is a key cause of relapse but clinicians do not appear to possess specific adherence skills. We sought to demonstrate that a brief training program on medication adherence strategies could improve the adherence skills, attitudes and knowledge of mental health clinicians. Methods: Twenty-three Tasmanian mental health workers were provided a 3 day training workshop on strategies to enhance patient adherence to medications (Medication Alliance). Pre- and post-training measures were taken of clinician knowledge about adherence strategies, ability to identify predictors of nonadherence, attitudes toward working with nonadherent patients, and optimism about treatment outcomes for patients. Videotapes of clinicians demonstrating key adherence therapy skills were also collected before and after training and blind-rated by two experienced therapists. Results: A series of paired samples t-tests indicated significant improvements in skills, knowledge and attitudes. Conclusions: Compared with similar studies in the UK, Medication Alliance was found to be an effective and efficient training program. However, there is a need for further research to assess maintenance of training effects over time and patient outcomes.

What does 'clinical significance' mean in the context of the Health of the Nation Outcome Scales?

Objective: The aim of this paper was to improve understanding of what ‘clinical significance’ means in relation to the Health of the Nation Outcome Scales (HoNOS) and its older persons and child/adolescent equivalents (the HoNOS65+ and HoNOSCA). Method: An anonymous, web-based survey was completed by 94 outcome measurement experts, most of whom had clinical responsibilities. Respondents were asked to indicate for acute inpatient and ambulatory settings: the rating on each item which represented a clinically significant problem; the relative importance of each item in determining overall clinical severity; and the items which would not be expected to improve between admission and review, admission and discharge, review and review, and review and discharge. Results: A score of 2 (‘mild problem but definitely present’) on each HoNOS/HoNOS65+/HoNOSCA item resonates with experts as being evidence of a clinically significant problem that requires active monitoring or intervention. In the main, all items on these instruments are viewed as equally important in making an overall judgement of clinical severity. The items making up the impairment and, to a lesser extent, social subscales are least likely to demonstrate change during the course of an episode of care, according to expert opinion. Generally, these findings apply across instruments and service settings. Conclusions: Overall, the findings provide support for the content validity and clinical utility of the HoNOS/HoNOS65+/HoNOSCA. Further exploration of the question of clinical significance as reflected in these instruments could take a number of forms.

Do adults in contact with Australia's public sector mental health services get better?

This paper describes the outcomes of episodes of care for adults in public sector mental health services across Australia, with a view to informing the debate on service quality. Health of the Nation Outcome Scales (HoNOS) change scores and effect sizes were calculated for 14,659 acute inpatient episodes and 23,692 community episodes. The results showed that people in contact with public sector mental health services generally do get better, although the magnitude of improvement depends on the setting and episode type. This confirmatory finding is particularly positive, given current community concerns about the quality and effectiveness of mental health services.

Modelling candidate effectiveness indicators for mental health services

Objective: Although Australia and the UK have both made efforts to systematize outcome measurement in mental health services, surprisingly little attention has been paid to how best to analyse routine outcome data in order to determine how services are performing. Methods: Outcome data collected in acute inpatient and ambulatory mental health services across Australia during the 2006–2007 financial year were used. three approaches to measuring effectiveness were explored: effect size (ES); the reliable change index (RCI); and standard error of measurement (SEM). Results: The most conservative results were produced by the RCI and the least conservative by the medium ES statistic and the SEM. By way of example, only 38.0% of inpatient admission–discharge periods of care showed significant improvement for adults when the RCI was used, whereas 67.4% and 72.9% did so when the medium ES and the SEM statistics were used, respectively. Conclusions: In any routine outcome measurement exercise, the degree of effectiveness demonstrated by services will depend on the specific statistical indicator used to judge effectiveness. Routine outcome measurement has the potential to answer a range of crucial performance-related questions, but only if the same metric is used. Discussion of the appropriate statistical approach to take to facilitate cross-service, cross-area and even cross-national comparisons warrants attention.

The multiple uses of routine mental health outcome measures in Australia and New Zealand: experiences from the field

Objective: The aim of this paper is to describe the way in which particular Australian and New Zealand mental health services are making use of routine outcome data to foster clinical improvements for consumers. Method: We invited individuals who are responsible for implementing outcome data collection across the two countries to describe exemplary practices occurring within their own services, and present the resultant information in the form of case studies. Results: Outcome measurement is being used to guide clinical decision-making, engage consumers in treatment, foster a collaborative approach to care planning and goal setting, review consumers’ progress with treatment, inform questions about consumers’ eligibility for given programs, assist with discharge planning, improve the evidence-base on which services are founded, and evaluate particular models of service delivery. Conclusions: A number of mental health services are deriving useful information from routine outcome measurement, and using this to guide clinical practice. The examples provided here may offer some lessons for other services wishing to make better use of outcome data, and there may be some opportunities for the sharing of resources or infrastructure. Services that are already using outcome data to inform their clinical activities, and services that are keen to do so, will need ongoing support.

A review of social inclusion measures

Background: Social inclusion is crucial to mental health and well-being and is emphasised in Australia’s Fourth National Mental Health Plan. There is a recognition that a measure of social inclusion would complement the suite of outcome measures that is currently used in public sector mental health services. This paper is an initial scope of candidate measures of social inclusion and considers their suitability for this purpose. Methods: We identified potential measures through searches of PsycINFO and Medline and a more general Internet search. We extracted descriptive and evaluative information on each measure identified and compared this information with a set of eight criteria. The criteria related to the measure’s inclusion of four domains of social inclusion outlined in Australia’s Fourth National Mental Health Plan, its usability within the public mental health sector and its psychometric properties. Results: We identified 10 candidate measures of social inclusion: the Activity and Participation Questionnaire (APQ-6); the Australian Community Participation Questionnaire (ACPQ); the Composite Measure of Social Inclusion (CMSI); the EMILIA Project Questionnaire (EPQ); the Evaluating Social Inclusion Questionnaire (ESIQ); the Inclusion Web (IW); the Social and Community Opportunities Profile (SCOPE); the Social Inclusion Measure (SIM); the Social Inclusion Questionnaire (SIQ); and the Staff Survey of Social Inclusion (SSSI). After comparison with the eight review criteria, we determined that the APQ-6 and the SCOPE–short form show the most potential for further testing. Conclusions: Social inclusion is too important not to measure. This discussion of individual-level measures of social inclusion provides a springboard for selecting an appropriate measure for use in public sector mental health services. It suggests that there are two primary candidates, but neither of these is quite fit-for-purpose in their current form. Further exploration will reveal whether one of these is suitable, whether another measure might be adapted for the current purpose or whether a new, specifically designed measure needs to be developed.

An examination of risk factors for readmission to acute adult mental health services within 28 days of discharge in the Australian setting

Objective: The aim of this paper was to identify risk factors associated with readmission within 28 days of discharge from eight Australian adult acute mental health inpatient services. Method: A detailed file audit was conducted comparing 222 patients readmitted within 28 days of discharge with 253 patients not readmitted during the same period. Results: There was an association between early readmission and having had contact with the service in the previous 12 months (51% vs 21%), having been admitted in the previous 12 months (65% vs 36%), and having been diagnosed with an emotionally unstable personality disorder (14% vs 4%). Those who were not readmitted had a significantly higher score in the Health of the Nation Outcome Scales on index admission than those who were readmitted. Those who had community team contact on the day of discharge and those who received follow up by the mental health team within 7 days of discharge were more likely to be readmitted (55% vs 45% and 29% vs 19% respectively). Conclusions: Significant determinants of early readmission identified were a history of recent previous admissions and a diagnosis of unstable personality disorder. The usefulness of this performance measure to Area Mental Health Services and clinicians is discussed.

Achievements in mental health outcome measurement in Australia: Reflections on progress made by the Australian Mental Health Outcomes and Classification Network (AMHOCN)

BackgroundAustralia’s National Mental Health Strategy has emphasised the quality, effectiveness and efficiency of services, and has promoted the collection of outcomes and casemix data as a means of monitoring these. All public sector mental health services across Australia now routinely report outcomes and casemix data. Since late-2003, the Australian Mental Health Outcomes and Classification Network (AMHOCN) has received, processed, analysed and reported on outcome data at a national level, and played a training and service development role. This paper documents the history of AMHOCN’s activities and achievements, with a view to providing lessons for others embarking on similar exercises.MethodWe conducted a desktop review of relevant documents to summarise the history of AMHOCN.ResultsAMHOCN has operated within a framework that has provided an overarching structure to guide its activities but has been flexible enough to allow it to respond to changing priorities. With no precedents to draw upon, it has undertaken activities in an iterative fashion with an element of ‘trial and error’. It has taken a multi-pronged approach to ensuring that data are of high quality: developing innovative technical solutions; fostering ‘information literacy’; maximising the clinical utility of data at a local level; and producing reports that are meaningful to a range of audiences.ConclusionAMHOCN’s efforts have contributed to routine outcome measurement gaining a firm foothold in Australia’s public sector mental health services.