Sally Redfern

The validity and reliability of methods to assess the competence to practise of pre-registration nursing and midwifery students.

This paper reports findings from a study funded by the National Board for Nursing, Midwifery and Health Visiting for Scotland to test selected nursing and midwifery clinical competence assessment tools for reliability and validity. The study, which took place over two years from July 1997, involved comparing items in the selected tools with statutory competencies for nurses and midwives, collecting assessment data from a sample of 257 nursing and 43 midwifery students in four educational institutions and administering additional assessment measures (the Nursing Competencies Questionnaire (NCQ) (Bartlett et al., An evaluation of pre-registration nursing education: a literature review and comparative study of graduate outcomes, Oxford Centre for Health Care Research & Development, Oxford Brookes University, Oxford, 1998) and the Key Areas Assessment Instrument-KAAI) to the total student sample (and to their lecturers and practice assessors) at two time points which were six months apart. Our focus was the programme-specific clinical competence assessment tools but by testing these tools we also provide evidence on the validity of other methods of competence assessment. Validity of the methods was assessed, primarily, by calculating multivariate and univariate correlation coefficients between them. The NCQ and KAAI were analysed for internal consistency. The NCQ and the versions of KAAI for lecturers and practice assessors were found to have good internal consistency. The version of the KAAI tool developed for students showed reasonable internal consistency for nursing students, but less consistency for midwifery students. Correlational analysis of data collected on students showed that there is little or no relationship between most of the clinical competence assessment methods in current use, or between these methods and those introduced by the research team. This finding supports previous research, particularly in medical education and confirms that the different methods address different abilities.A clear finding from this study is that no single method is appropriate for assessing clinical competence. A multi-method UK-wide strategy for clinical competence assessment for nursing and midwifery is needed if we are to be sure that assessment reveals whether or not students have achieved the complex repertoire of knowledge, skills and attitudes required for competent practice.

Assessing competence to practise in nursing: a review of the literature

Recent reforms of nursing education have led to calls for assessment of clinical performance to contribute to academic qualifications that incorporate professional awards. Questions then follow concerning the psychometric quality of methods available for assessing competence and performance and the ability of the methods to distinguish between different levels of practice. The purpose of this review of the literature is to analyse methods of assessing competence to practise in nursing and draw conclusions on their reliability and validity. The methods reviewed include questionnaire rating scales, ratings by observation, criterion-referenced rating scales, simulations including the objective structured clinical examination (OSCE), Benners model of skill acquisition, reflection in and on practice, self-assessment and multi-method approaches. Methodological challenges to competence assessment are raised. Findings of the review are as follows. Questionnaire rating scales used to assess competence in nursing have not been rigorously tested for reliability and validity. The methodological limitations of observation can be overcome with use of criterion-referenced scales or simulations such as the OSCE or its variations, as has been found in research on competence assessment in medicine. The Barts Nursing OSCE, designed for the end of the common foundation programme of the nursing diploma course, has promise because it provides a close simulation to real life and is grounded in an interpretive approach to skill acquisition. Reflection on practice using portfolios, learning contracts and self-assessment is valid if based on rigorous analysis of critical incidents rather than simple description. Reflective approaches have the advantage of being located in real-life settings and can integrate theory with practice but they need skilled collaboration between teachers, employers and practitioners. The conclusion is that a multimethod approach enhances validity and ensures comprehensive assessment of the complex repertoire of skills required of students in nursing.

Using Realistic Evaluation to Evaluate a Practice-level Intervention to Improve Primary Healthcare for Patients with Long-term Mental Illness

Mental Health Link - a facilitated programme - aimed to develop systems within primary care and links with specialists to improve care for patients with long-term mental illness. A process evaluation based on Pawson and Tilley’s Realistic Evaluation complemented a randomized controlled trial. This article describes the method developed for this ‘realistic evaluation’, the mechanisms behind the integration of linked specialist workers and discusses practical and theoretical issues arising from the use of the realistic evaluation framework as a way of explaining the results of trials and service development. Retrospective interviews identified the important outcomes and were used to construct ‘Context-Mechanism-Outcome’ configurations. The 12 case studies represented what had happened. A second-level analysis using analytic induction developed ‘middle range theories’ designed to be of value to those developing care elsewhere. The intervention was successful in stimulating productive joint working, through case discussions, but often failed to ensure a review of progress.

Achieving change in health care practice

This study evaluated a practice development programme consisting of nine projects together known as STEP (South Thames Evidence-Based Practice Project). The aim of STEP was to establish and assess evidence-based practice in nursing and other health care practice areas. Objectives of the independent evaluation were to identify and assess outcomes from the process of change and investigate the association between these intermediate outcomes and patient outcomes. Outcomes were measured before and after the changes were introduced. Data collection methods included interviews with the change agents and other stakeholders, and a questionnaire to staff in each centre. Patient outcome data were collected from each centre. The findings revealed ‘dissemination’ of information to staff and ‘adherence’ by staff to new practice guidelines to be important intermediate outcomes in the process of change. The need emerged for a supportive organizational culture and commitment, recognition of the importance of change and a credible change agent. There was some evidence of linearity in the process of change in that a logical route appeared from dissemination of information to staff through adherence to the change guidelines to improvement in patient outcome in six of the nine centres. Linearity was less apparent in the other three centres, where the process of change seemed more dynamic and chaotic. These three centres were affected more than the others by organizational barriers to change. Our conclusions support the view that the linear model of change can work in settings with high levels of certainty but complexity theory is more likely to underpin the process of change in organizations characterized by uncertainty.

Care work and quality of care for older people: A review of the research literature

There is considerable research on work satisfaction and stress of care staff on the one hand and on quality of care and well-being of older people on the other. However, very little research in continuing-care settings for older people (nursing homes, residential homes and long-stay wards) links perceptions of workers (work satisfaction and stress) with the process of care (quality of care) and outcomes for residents/patients (well-being). This is a notable omission, given the emphasis of government policy on improving quality of health and social care services for vulnerable elderly people in the UK. The White Papers, The New NHS and Modernising Social Services and the National Priority Guidance for Health and Social Services for 2000–2003, all emphasize the importance of services that are responsive to local needs and which maintain and promote independence. The Centre for Policy on Ageing has been commissioned by the Department of Health to develop national standards for nursing and residential home care services for older people. This work will build upon recommendations of the Burgner Report and Achieving a Better Home Life, which identified areas for benchmarking. The national standards proposed under the forthcoming National Service Framework for Older People will underpin new legislation to regulate care services.

Care at home for people with dementia: as in a total institution?

This article examines three kinds of social relationship likely to be present when people with dementia are cared for at home by relatives or friends: custodial care, an intimate relationship, and home-life. Using Goffmans three defining aspects of custodial care – routinisation, surveillance and mortification of the self – the paper examines whether these characterised the care of people with dementia at home and, if so, whether they conflicted with the intimate relationship and with home-life. The study involved sustained observations and interviews with 20 people with dementia and their carers in and around London, and qualitative analysis of the data. It was found that all three aspects of custodial care were present although not fully realised, and that they led to difficulties, many of which were associated with the concurrent pursuit of an intimate relationship and home-life. In all cases, daily life was routinised partly to help accomplish care tasks but was found monotonous, while surveillance was usual but restrictive, and prevented both the carers and those with dementia from doing things that they wished to do. Those with dementia were distressed by the denial of their former identities, such as car-driver or home-maker, and by being treated like children. Both the carers and the people with dementia had various ways of balancing custodial care, their intimate relationships and home-life, such as combining routines with other activities, evading surveillance or carrying it out by indirect means, and there were many attempts to maintain some semblance of former identities.

Individualised patient care Its meaning and practice in a general setting

It is assumed that individualised patient care benefits both patients and nurses. This study set out to clarify what individualised patient care means to nurses and how they practise it as well as how it is experienced by patients. With some exceptions, individualised patient care was not practised widely in the seven wards that were used as case studies. Even in the wards where individualised patient care was more common, there were some examples of bad practice. Factors that facilitated individualised patient care were: the personal qualities of the nurses themselves; a shared understanding among the ward team of the goals of nursing care and what constitutes good practice; levels of staffing and skill mix; and effective leadership and management of nursing work [NTResearch 1996; 1: 1, 22-33]

An evaluation of nursing development units

The aims of this study, which is still in progress, are to assess the value that nursing development units add to nursing and health care and to describe the essence of an NDU. The areas investigated cover: resources and costs, research and audit activities, networking activities, staff morale, staff development and supervision, and clinical leadership. The research took a staged approach with four phases: consultation, profiling the NDUs, comparing NDUs with units without NDU status and case studies of five high-performing NDUs. This paper provides an overview of the study and findings from Phases 1-3. It was important for clinical leaders to be clinically credible, to have authority and to be free from day-to-day care provision and management. Quantitative differences between NDUs and comparison units emerged for research and dissemination activity (NDUs were more active), and sickness absence (more long-term sickness in NDUs) but not for audit and staff development activity, nor for the financial context. Data from the final phase will give a more detailed understanding of the significance of these differences and the different pathways that can be taken to achieving success as an NDU.

Accomplishing Care at Home for People With Dementia: Using Observational Methodology

The authors consider the problems and possibilities presented by using unstructured observation in the home setting. The findings of the original study are described elsewhere (Briggs, Askham, Norman, & Redfern 1998; National Health Service [NHS] Executive 1998). In this article, the authors discuss process issues (e.g., gaining and maintaining access to the research setting, disengaging, what can and cannot be observed, and the validity of the data and analysis), drawing on a study of the nature of care as a form of social organization in the homes of people with dementia. They show that observation is not only possible but, when combined with conversational interviewing, essential for understanding the processes of caring and what it means to live with dementia.

Perceptions and Management of Change by People with Dementia and Their Carers Living at Home

This article presents findings from a research study that examined, by observation and interview, the nature of care as a form of social organization in the homes of people with dementia whose care was at least partly provided by relatives or friends. Twenty situations were included in which the carers and people with dementia were interviewed. Each participating pair was observed for an average of nine hours. Change was perceived, on the whole, in negative terms for both caregiver and the person with dementia. The main categories of change to emerge were those to do with normal routines or practices, perceived deterioration in the condition of the person with dementia, and changes concerning living arrangements. Coping strategies used reflect those in the general coping literature. They included denial, attempts to restore the situation or regain control, making positive comparisons, attempting to understand dementia and acceptance.