Derek Kyte

Patient-Reported Outcome (PRO) Assessment in Clinical Trials: A Systematic Review of Guidance for Trial Protocol Writers

Background Evidence suggests there are inconsistencies in patient-reported outcome (PRO) assessment and reporting in clinical trials, which may limit the use of these data to inform patient care. For trials with a PRO endpoint, routine inclusion of key PRO information in the protocol may help improve trial conduct and the reporting and appraisal of PRO results; however, it is currently unclear exactly what PRO-specific information should be included. The aim of this review was to summarize the current PRO-specific guidance for clinical trial protocol developers. Methods and Findings We searched the MEDLINE, EMBASE, CINHAL and Cochrane Library databases (inception to February 2013) for PRO-specific guidance regarding trial protocol development. Further guidance documents were identified via Google, Google scholar, requests to members of the UK Clinical Research Collaboration registered clinical trials units and international experts. Two independent investigators undertook title/abstract screening, full text review and data extraction, with a third involved in the event of disagreement. 21,175 citations were screened and 54 met the inclusion criteria. Guidance documents were difficult to access: electronic database searches identified just 8 documents, with the remaining 46 sourced elsewhere (5 from citation tracking, 27 from hand searching, 7 from the grey literature review and 7 from experts). 162 unique PRO-specific protocol recommendations were extracted from included documents. A further 10 PRO recommendations were identified relating to supporting trial documentation. Only 5/162 (3%) recommendations appeared in ≥50% of guidance documents reviewed, indicating a lack of consistency. Conclusions PRO-specific protocol guidelines were difficult to access, lacked consistency and may be challenging to implement in practice. There is a need to develop easily accessible consensus-driven PRO protocol guidance. Guidance should be aimed at ensuring key PRO information is routinely included in appropriate trial protocols, in order to facilitate rigorous collection/reporting of PRO data, to effectively inform patient care.

Systematic evaluation of the patient-reported outcome (PRO) content of clinical trial protocols

Background Qualitative evidence suggests patient-reported outcome (PRO) information is frequently absent from clinical trial protocols, potentially leading to inconsistent PRO data collection and risking bias. Direct evidence regarding PRO trial protocol content is lacking. The aim of this study was to systematically evaluate the PRO-specific content of UK National Institute for Health Research (NIHR) Health Technology Assessment (HTA) programme trial protocols. Methods and Findings We conducted an electronic search of the NIHR HTA programme database (inception to August 2013) for protocols describing a randomised controlled trial including a primary/secondary PRO. Two investigators independently reviewed the content of each protocol, using a specially constructed PRO-specific protocol checklist, alongside the ‘Standard Protocol Items: Recommendations for Interventional Trials’ (SPIRIT) checklist. Disagreements were resolved through discussion with a third investigator. 75 trial protocols were included in the analysis. Protocols included a mean of 32/51 (63%) SPIRIT recommendations (range 16–41, SD 5.62) and 11/33 (33%) PRO-specific items (range 4–18, SD 3.56). Over half (61%) of the PRO items were incomplete. Protocols containing a primary PRO included slightly more PRO checklist items (mean 14/33 (43%)). PRO protocol content was not associated with general protocol completeness; thus, protocols judged as relatively ‘complete’ using SPIRIT were still likely to have omitted a large proportion of PRO checklist items. Conclusions The PRO components of HTA clinical trial protocols require improvement. Information on the PRO rationale/hypothesis, data collection methods, training and management was often absent. This low compliance is unsurprising; evidence shows existing PRO guidance for protocol developers remains difficult to access and lacks consistency. Study findings suggest there are a number of PRO protocol checklist items that are not fully addressed by the current SPIRIT statement. We therefore advocate the development of consensus-based supplementary guidelines, aimed at improving the completeness and quality of PRO content in clinical trial protocols.

Inconsistencies in quality of life data collection in clinical trials: a potential source of bias? Interviews with research nurses and trialists

Background Patient-reported outcomes (PROs), such as health-related quality of life (HRQL) are increasingly used to evaluate treatment effectiveness in clinical trials, are valued by patients, and may inform important decisions in the clinical setting. It is of concern, therefore, that preliminary evidence, gained from group discussions at UK-wide Medical Research Council (MRC) quality of life training days, suggests there are inconsistent standards of HRQL data collection in trials and appropriate training and education is often lacking. Our objective was to investigate these reports, to determine if they represented isolated experiences, or were indicative of a potentially wider problem. Methods And Findings We undertook a qualitative study, conducting 26 semi-structured interviews with research nurses, data managers, trial coordinators and research facilitators involved in the collection and entry of HRQL data in clinical trials, across one primary care NHS trust, two secondary care NHS trusts and two clinical trials units in the UK. We used conventional content analysis to analyze and interpret our data. Our study participants reported (1) inconsistent standards in HRQL measurement, both between, and within, trials, which appeared to risk the introduction of bias; (2), difficulties in dealing with HRQL data that raised concern for the well-being of the trial participant, which in some instances led to the delivery of non-protocol driven co-interventions, (3), a frequent lack of HRQL protocol content and appropriate training and education of trial staff, and (4) that HRQL data collection could be associated with emotional and/or ethical burden. Conclusions Our findings suggest there are inconsistencies in the standards of HRQL data collection in some trials resulting from a general lack of HRQL-specific protocol content, training and education. These inconsistencies could lead to biased HRQL trial results. Future research should aim to develop HRQL guidelines and training programmes aimed at supporting researchers to carry out high quality data collection.

Patient Reported Outcomes (PROs) in Clinical Trials: Is ‘In-Trial’ Guidance Lacking? A Systematic Review

Background Patient reported outcomes (PROs) are increasingly assessed in clinical trials, and guidelines are available to inform the design and reporting of such trials. However, researchers involved in PRO data collection report that specific guidance on ‘in-trial’ activity (recruitment, data collection and data inputting) and the management of ‘concerning’ PRO data (i.e., data which raises concern for the well-being of the trial participant) appears to be lacking. The purpose of this review was to determine the extent and nature of published guidelines addressing these areas. Methods and Findings Systematic review of 1,362 articles identified 18 eligible papers containing ‘in-trial’ guidelines. Two independent authors undertook a qualitative content analysis of the selected papers. Guidelines presented in each of the articles were coded according to an a priori defined coding frame, which demonstrated reliability (pooled Kappa 0.86–0.97), and validity (<2% residual category coding). The majority of guidelines present were concerned with ‘pre-trial’ activities (72%), for example, outcome measure selection and study design issues, or ‘post-trial’ activities (16%) such as data analysis, reporting and interpretation. ‘In-trial’ guidelines represented 9.2% of all guidance across the papers reviewed, with content primarily focused on compliance, quality control, proxy assessment and reporting of data collection. There were no guidelines surrounding the management of concerning PRO data. Conclusions The findings highlight there are minimal in-trial guidelines in publication regarding PRO data collection and management in clinical trials. No guidance appears to exist for researchers involved with the handling of concerning PRO data. Guidelines are needed, which support researchers to manage all PRO data appropriately and which facilitate unbiased data collection.

Patient-Reported Outcome Alerts: Ethical and Logistical Considerations in Clinical Trials

The assessment of patient-reported outcomes (PROs) in clinical trials poses a number of potential problems. What happens when a patient reports a severe symptom and no one is monitoring that information; for example, when questionnaires are not reviewed until the end of a study? Do hospitals or researchers face liability if a patient reports suicidal thoughts on a questionnaire? Investigators and institutional review boards (IRBs) are increasingly concerned about the risks to patients, and the potential legal liability, when PRO information is not reviewed to detect concerning levels of psychological distress or physical symptoms that may require an immediate response.1 These so-called PRO alerts may present as extreme scores on questionnaires, in additional information provided by a patient, or in a discussion between a patient and research personnel. However, in all cases, the overarching priority must be to ensure the safety and well-being of patients participating in clinical trials.

Guidelines for Inclusion of Patient-Reported Outcomes in Clinical Trial Protocols:: the SPIRIT-PRO Extension

Importance Patient-reported outcome (PRO) data from clinical trials can provide valuable evidence to inform shared decision making, labeling claims, clinical guidelines, and health policy; however, the PRO content of clinical trial protocols is often suboptimal. The SPIRIT (Standard Protocol Items: Recommendations for Interventional Trials) statement was published in 2013 and aims to improve the completeness of trial protocols by providing evidence-based recommendations for the minimum set of items to be addressed, but it does not provide PRO-specific guidance. Objective To develop international, consensus-based, PRO-specific protocol guidance (the SPIRIT-PRO Extension). Design, Setting, and Participants The SPIRIT-PRO Extension was developed following the Enhancing Quality and Transparency of Health Research (EQUATOR) Network’s methodological framework for guideline development. This included (1) a systematic review of existing PRO-specific protocol guidance to generate a list of potential PRO-specific protocol items (published in 2014); (2) refinements to the list and removal of duplicate items by the International Society for Quality of Life Research (ISOQOL) Protocol Checklist Taskforce; (3) an international stakeholder survey of clinical trial research personnel, PRO methodologists, health economists, psychometricians, patient advocates, funders, industry representatives, journal editors, policy makers, ethicists, and researchers responsible for evidence synthesis (distributed by 38 international partner organizations in October 2016); (4) an international Delphi exercise (n = 137 invited; October 2016 to February 2017); and (5) consensus meeting (n = 30 invited; May 2017). Prior to voting, consensus meeting participants were informed of the results of the Delphi exercise and given data from structured reviews evaluating the PRO protocol content of 3 defined samples of trial protocols. Results The systematic review identified 162 PRO-specific protocol recommendations from 54 sources. The ISOQOL Taskforce (n = 21) reduced this to 56 items, which were considered by 138 international stakeholder survey participants and 99 Delphi panelists. The final wording of the SPIRIT-PRO Extension was agreed on at a consensus meeting (n = 29 participants) and reviewed by external group of experts during a consultation period. Eleven extensions and 5 elaborations to the SPIRIT 2013 checklist were recommended for inclusion in clinical trial protocols in which PROs are a primary or key secondary outcome. Extension items focused on PRO-specific issues relating to the trial rationale, objectives, eligibility criteria, concepts used to evaluate the intervention, time points for assessment, PRO instrument selection and measurement properties, data collection plan, translation to other languages, proxy completion, strategies to minimize missing data, and whether PRO data will be monitored during the study to inform clinical care. Conclusions and Relevance The SPIRIT-PRO guidelines provide recommendations for items that should be addressed and included in clinical trial protocols in which PROs are a primary or key secondary outcome. Improved design of clinical trials including PROs could help ensure high-quality data that may inform patient-centered care.

An introduction to patient-reported outcome measures in ophthalmic research.

Clinical outcomes, such as quantifying the extent of visual field loss by automated perimetry, are valued highly by health professionals, but such measures do not capture the impact of the condition on a patient’s life. Patient-reported outcomes describe any report or measure of health reported by the patient, without external interpretation by a clinician or researcher. In this review, we discuss the value of the measures that capture this information (patient-reported outcome measures; PROMs), and why they are important to both the clinician and the researcher. We also consider issues around developing or selecting a PROM for ophthalmic research, the emerging challenges around conducting and reporting PROMs in clinical trials and highlight best practice for their use. Search terms for this review comprised: (1) (patient-reported outcomes OR patient-reported outcome measures) AND (2) randomised controlled trials AND (3) limited to ophthalmic conditions. These terms were expanded as follows: (((‘patients’(MeSH Terms) OR ‘patients’(All Fields) OR ‘patient’(All Fields)) AND (‘research report’(MeSH Terms) OR (‘research’(All Fields) AND ‘report’(All Fields)) OR ‘research report’(All Fields) OR ‘reported’(All Fields)) AND outcomes(All Fields)) OR ((‘patients’(MeSH Terms) OR ‘patients’(All Fields) OR ‘patient’(All Fields)) AND (‘research report’(MeSH Terms) OR (‘research’(All Fields) AND ‘report’(All Fields)) OR ‘research report’(All Fields) OR ‘reported’(All Fields) AND (‘outcome assessment (health care)’(MeSH Terms) OR (‘outcome’(All Fields) AND ‘assessment’(All Fields) AND ‘(health’(All Fields) AND ‘care)’(All Fields)) OR ‘outcome assessment (health care)’(All Fields) OR (‘outcome’(All Fields) AND ‘measures’(All Fields)) OR ‘outcome measures’(All Fields)))) AND (‘randomized controlled trial’(Publication Type) OR ‘randomized controlled trials as topic’(MeSH Terms) OR ‘randomised controlled trials’(All Fields) OR ‘randomized controlled trials’(All Fields)) AND (ophth*(All Fields)). The authors also utilised the extensive non-ophthalmic literature and online resources relating to PROs and PROMs to inform this review.

Putting patient-reported outcomes on the ‘Big Data Road Map’

The term ‘big data’ is used to describe large, complex datasets and the associated advances in technology and analytics. Within healthcare, this covers many different types of data including, for example: routinely collected clinic data in electronic health records; healthcare claims data, genomics data; public data releases, such as aggregated population health data or clinical trials results; data from automated sensors and smart devices; and information from healthcare forums and social media. In late 2013, the Association of the British Pharmaceutical Industry published its ‘Big data road map’, which sets out an action plan to direct progress in the use of big data over the next four years. The road map highlights the need to increase awareness and understanding of big data, build capability and capacity, create sustainable data ecosystems and accelerate high-value opportunities. These suggestions do not just apply to pharma, however, but also to healthcare more generally as clinicians, patients and policy-makers look to take advantage of the potential benefits of big data. There is, however, a risk that patient-reported outcome (PRO) data may be underrepresented in the big data ‘revolution’. PRO data are important in providing patients’ perspective on their own health. However, in contrast to countries such as the US and Netherlands, PROs are not routinely collected in UK clinical settings (other than for a small number of procedures, via the NHS PROMs programme). If this situation is not addressed, we risk continuing our overreliance on routinely collected clinical indicators to provide evidence for health policy: a potential backwards step in the move towards patient-centred care.

Oseltamivir: the real world data

To coincide with the publication of an updated systematic review by the Cochrane group the BMJ invited Nick Freemantle and colleagues to consider the current status of observational studies of oseltamivir and their influence on policy and practice

Putting patients at the heart of health-care research.

www.thelancet.com Vol 385 March 21, 2015 1073 important patient outcomes can be lost. Our systematic review of PRO guidance for trial protocol writers identified a confusing mix of 162 separate recommendations spread across 54 documents. Additionally, although the Standard Protocol Items: Recommendations for Interventional Trials (SPIRIT) group published guidelines in 2013, this guidance does not provide specific information about PROs. Clearly, consolidated PRO guide lines are needed and could be in the form of a SPIRIT PRO extension. However, a checklist alone is unlikely to be suffi cient to change practice. We need to develop comprehensive guidelines and web-based training resources that are accessible to a range of stakeholders including researchers, funding bodies, journal editors and reviewers, patient advocates involved in trial design, and ethics committees. Additionally, support from journal editors and funding agencies will be essential to help with the appropriate uptake of these proposed SPIRIT PRO guidelines. Development of a SPIRIT PRO extension and the supportive resources is being led by researchers at the University of Birmingham (UK) and University of Sydney (Australia), in collaboration with the International Society for Quality of Life Research, SPIRIT Executive, and international stakeholders. Improvements to PRO research are essential to provide high-quality data and inform patient care.