Steve Sizmur

Influence of patients’ age and sex and the mode of administration on results from the NHS Friends and Family Test of patient experience:

Objective To investigate the impact of demographic factors (patients’ age and sex) and of the mode of administration of a national patient experience questionnaire in England: the NHS Friends and Family Test. Methods Secondary analysis of April–August 2013 data collected using a mixed mode approach from 38,998 inpatients and 29,610 emergency department attendees at 429 wards or units in 32 hospitals. Multilevel models were applied with responses from wards nested within hospitals and trusts. Age, sex and mode of administration were entered as main effects. Results There were consistent differences in response for patients and emergency department attendees related to their age and sex. Women gave less positive ratings than men, whilst the likelihood of positive responses increased with age except among the oldest age group (75 years and above). As regards mode of administration, online responses were significantly less positive than postcard responses: the mean differences in score were 22.0 points for inpatients (95% confidence interval 27.3 to 16.7) and 18.0 points for emergency department attendees (29.0 to 7.0). Telephone responses were significantly more positive than postcard responses, with a mean difference of 9.2 (1.6–16.8) in the emergency department setting. Conclusions Data from the Friends and Family Test are vulnerable to bias from demographic factors and from the mode of administration. Comparisons between organisations should be avoided. Scores may be useful at a local level where the test is implemented consistently and patients’ demographic characteristics remain stable. Improving the utility of the Friends and Family Test nationally requires a standardised method for administration and adjustment of results for demographic characteristics.

Do some trusts deliver a consistently better experience for patients? An analysis of patient experience across acute care surveys in English NHS trusts

Introduction Data were used from inpatient, outpatient and accident and emergency surveys in acute trusts in England to examine consistency in patient-reported experience across services, and factors associated with systematic variations in performance. Methods Standardised mean scores for six domains of patient experience were constructed for each survey for 145 non-specialist acute trusts. Hierarchical cluster analysis was used to investigate whether and how trust performance clusters. Multilevel regression analysis was used to determine trust characteristics associated with performance. Results Cluster analysis identified three groups: trusts that performed consistently above (30 trusts) or below (six trusts) average, and those with mixed performance. All the poor performing trusts were in London, none were foundation trusts or teaching hospitals, and they had the highest mean deprivation score and the lowest proportion of white inpatients and response rates. Foundation and teaching status, and the proportion of white inpatients, were positively associated with performance; deprivation and response rates showed less consistent positive associations. No regional effects were apparent after adjusting for independent variables. Conclusion The results have significant implications for quality improvement in the NHS. The finding that some NHS providers consistently perform better than others suggests that there are system-wide determinants of patient experience and the potential for learning from innovators. However, there is room for improvement overall. Given the large samples of these surveys, the messages could also have relevance for healthcare systems elsewhere.

The accident and emergency department questionnaire: a measure for patients’ experiences in the accident and emergency department—reply

With this letter we respond to the comments of Chance et al on our recent publication ‘Patient Experience in the Accident and Emergency Department’. We applied three different methods of grouping and summarised the items on the Accident and Emergency (A&E) department questionnaire presented to patients. The best score reliability was provided by the Principal Components Analysis. Chance et al acknowledge the importance of patients’ feedback but questioned whether this measure of patients’ experiences is valuable and valid for comparisons of the quality of care services, and whether selection bias affected the results. The purpose of our research was to examine alternative ways of obtaining a reliable composite, but we do not negate the important information from separate questions. Single questionnaire items are less reliable than well-constructed composites,1 but are able …

Impact of case-mix on comparisons of patient-reported experience in NHS acute hospital trusts in England

Objectives To examine the impact of patient-mix on National Health Service (NHS) acute hospital trust scores in two national NHS patient surveys. Methods Secondary analysis of 2012 patient survey data for 57,915 adult inpatients at 142 NHS acute hospital trusts and 45,263 adult emergency department attendees at 146 NHS acute hospital trusts in England. Changes in trust scores for selected questions, ranks, inter-trust variance and score-based performance bands were examined using three methods: no adjustment for case-mix; the current standardization method with weighting for age, sex and, for inpatients only, admission method; and a regression model adjusting in addition for ethnicity, presence of a long-term condition, proxy response (inpatients only) and previous emergency attendances (emergency department survey only). Results For both surveys, all the variables examined were associated with patients’ responses and affected inter-trust variance in scores, although the direction and strength of impact differed between variables. Inter-trust variance was generally greatest for the unadjusted scores and lowest for scores derived from the full regression model. Although trust scores derived from the three methods were highly correlated (Kendall’s tau coefficients 0.70–0.94), up to 14% of trusts had discordant ranks of when the standardization and regression methods were compared. Depending on the survey and question, up to 14 trusts changed performance bands when the regression model with its fuller case-mix adjustment was used rather than the current standardization method. Conclusions More comprehensive case-mix adjustment of patient survey data than the current limited adjustment reduces performance variation between NHS acute hospital trusts and alters the comparative performance bands of some trusts. Given the use of these data for high-impact purposes such as performance assessment, regulation, commissioning, quality improvement and patient choice, a review of the long-standing method for analysing patient survey data would be timely, and could improve rigour and comparability across the NHS. Performance comparisons need to be perceived as fair and scientifically robust to maintain confidence in publicly reported data, and to support their use by both the public and the NHS.

New patient-reported experience measure for children with allergic disease: development, validation and results from integrated care

Objectives To develop and validate a new allergy-specific patient-reported experience measure (PREM) for children and their parents, and to collect feedback in an integrated care setting. Design Two allergy-specific PREMs were produced using focus groups, cognitive testing, two prospective validation studies (collaboration: Royal College of Paediatrics and Child Health, Picker Institute Europe, Imperial College/London): ‘Your Allergy Care’, for children aged 8–16 years; ‘Your Childs Allergy Care’, for parents of children aged 0–7 years. Setting Community event, primary/secondary/tertiary allergy care settings. Main outcome measures Performance of PREMs in validation study; reported experience of allergy care. Participants 687 children with allergic conditions and their parents/carers. Results In total, 687 questionnaires were completed; 503/687 (253 child; 250 parent) for the final survey. In both surveys, demographic variations were not associated with differences in results. Although 71% of patients reported one or more allergic conditions (food allergy/eczema/hay fever/asthma), 62% required multiple visits before receiving final diagnosis. Overall, patient experience was good for communication with patient/parent, competence and confidence in ability, and 73% felt looked after ‘very well’ and 23% ‘quite well’. Areas for improvement included communication with nurseries/schools, more information on side effects, allergic conditions and allergen/irritant avoidance. Allergy care in primary/emergency care settings was associated with higher problem-scores (worse experience) than in specialist clinics. Conclusions These new PREMs will allow allergy-specific patient experience reporting for children and parents and help identification of priority areas for improvement and commissioning of care. Efforts towards better allergy care provision must be targeted at primary and emergency care settings and underpinned by improving communication between healthcare providers and the community.

Children and young people’s versus parents’ responses in an English national inpatient survey

Objective Despite growing interest in children and young people’s (CYP) perspectives on healthcare, they continue to be excluded from many patient experience surveys. This study investigated the feasibility of, and additional information gained by, measuring CYP experiences of a recent hospital admission. Design Cross-sectional analysis of national survey data. Setting Inpatients aged 8–15 years in eligible National Health Service hospitals, July–September 2014. Participants 6204 parents/carers completed the parent section of the survey. The CYP section of the survey was completed by CYP themselves (n=3592), parents (n=849) or jointly (n=1763). Main outcome measures Pain relief, involvement, quality of staff communication, perceived safety, ward environment, overall experience. Analyses Single-measures intraclass correlations (ICCs) were used to assess the concordance between CYP and parent responses about the same inpatient episode. Multilevel logistic regression models, adjusted for individual characteristics, were used to compare the odds of positive responses when the CYP section of the survey was completed by parents, by CYP themselves or jointly. Results The CYP section of the survey was completed independently by 57.8% of CYP. Agreement between CYP and parent responses was reasonably good for pain relief (ICC=0.61 (95% CI 0.58 to 0.63)) and overall experience (ICC=0.70 (95% CI 0.68 to 0.72)), but much lower for questions comparing professionals’ communication with CYP and with their parents (ICC range=0.28 (95% CI 0.24 to 0.32) to 0.51 (95% CI 0.47 to 0.54)). In the regression models, CYP were significantly less likely than parents to report feeling safe (adjusted OR (AOR)=0.54 (95% CI 0.38 to 0.76)), involvement in decisions (AOR=0.66 (95% CI 0.46 to 0.94)) or adequate privacy (AOR=0.68 (95% CI 0.52 to 0.89)). Conclusions Including CYP (8–15 years) in patient experience surveys is feasible and enhances what is known from parents’ responses.

The Relational Aspects of Care Questionnaire: item reduction and scoring using inpatient and accident and emergency data in England

Purpose The Relational Aspects of Care Questionnaire (RAC-Q) is an electronic instrument which has been developed to assess staff’s interactions with patients when delivering relational care to inpatients and those accessing accident and emergency (A&E) services. The aim of this study was to reduce the number of questionnaire items and explore scoring methods for “not applicable” response options. Patients and methods Participants (n=3928) were inpatients or A&E attendees across six participating hospital trusts in England during 2015–2016. The instrument, consisting of 20 questionnaire items, was administered by trained hospital volunteers over a period of 10 months. Items were subjected to exploratory factor analysis to confirm unidimensionality, and the number of items was reduced using a range of a priori psychometric criteria. Two alternative approaches to scoring were undertaken, one treated “not applicable” responses as missing data, while the second adopted a problem score approach where “not applicable” was considered “no problem with care.” Results Two short-form RAC-Qs with alternative scoring options were identified. The first (the RAC-Q-12) contained 12 items, while the second scoring option (the RAC-Q-14) contained 14 items. Scores from both short forms correlated highly with the full 20-item parent form score (RAC-Q-12, r=0.93 and RAC-Q-14, f=0.92), displayed high internal consistency (Cronbach’s α: RAC-Q-12=0.92 and RAC-Q-14=0.89) and had high levels of agreement (intraclass correlation coefficient [ICC]=0.97 for both scales). Conclusion The RAC-Q is designed to offer near-real-time feedback on staff’s interactions with patients when delivering relational care. The new short-form RAC-Qs and their respective method of scoring are reflective of scores derived using the full 20-item parent form. The new short-form RAC-Qs may be incorporated into inpatient surveys to enable the comparison of ward or hospital performance. Using either the RAC-Q-12 or the RAC-Q-14 offers a method to reduce missing data and response fatigue.

Differences in treatment approach between ethnic groups

Purpose – The mental health experience of people from ethnic minorities differs from that of the majority, including differential access to services and treatments. The 2014 National Health Service (NHS) Community Mental Health survey gathered data from 13,787 individuals in 57 NHS trusts in England, providing one means of monitoring such experience. The purpose of this paper is to analyse survey variables describing treatments offered to respondents for evidence of differential access or treatment experiences associated with ethnicity. Design/methodology/approach – Secondary analysis of survey data. Proportions for target variables were modelled using multilevel logit models. Ethnic background, age and gender were entered as independent variables. Findings – Respondents in most minority groups were more likely to be on the care programme approach (CPA) to provision than white British respondents and less likely to report receiving psychological treatments. Unmet need for psychological treatment was relatively high in certain Asian groups. Medication use was consistently high across respondents, but differences by ethnic background were evident. Research limitations/implications – The study was dependent on existing survey data of a relatively limited nature, and potentially subject to non-response bias. The survey excludes users of certain types of service, giving an incomplete cross-section. Originality/value – This represents a novel use of the data from the Community Mental Health survey, and complements evidence from a range of other sources. The findings mostly concur with other evidence but provide important new data in relation to medication, unmet needs in psychotherapy and use of the CPA. They remain suggestive of the complex nature of discrimination and/or unequal access and treatment in mental health services.