Sarah Gorst

The COMET Handbook: Version 1.0

The selection of appropriate outcomes is crucial when designing clinical trials in order to compare the effects of different interventions directly. For the findings to influence policy and practice, the outcomes need to be relevant and important to key stakeholders including patients and the public, health care professionals and others making decisions about health care. It is now widely acknowledged that insufficient attention has been paid to the choice of outcomes measured in clinical trials. Researchers are increasingly addressing this issue through the development and use of a core outcome set, an agreed standardised collection of outcomes which should be measured and reported, as a minimum, in all trials for a specific clinical area.Accumulating work in this area has identified the need for guidance on the development, implementation, evaluation and updating of core outcome sets. This Handbook, developed by the COMET Initiative, brings together current thinking and methodological research regarding those issues. We recommend a four-step process to develop a core outcome set. The aim is to update the contents of the Handbook as further research is identified.

Core Outcome Set–STAndards for Reporting: The COS-STAR Statement

Background Core outcome sets (COS) can enhance the relevance of research by ensuring that outcomes of importance to health service users and other people making choices about health care in a particular topic area are measured routinely. Over 200 COS to date have been developed, but the clarity of these reports is suboptimal. COS studies will not achieve their goal if reports of COS are not complete and transparent. Methods and Findings In recognition of these issues, an international group that included experienced COS developers, methodologists, journal editors, potential users of COS (clinical trialists, systematic reviewers, and clinical guideline developers), and patient representatives developed the Core Outcome Set–STAndards for Reporting (COS-STAR) Statement as a reporting guideline for COS studies. The developmental process consisted of an initial reporting item generation stage and a two-round Delphi survey involving nearly 200 participants representing key stakeholder groups, followed by a consensus meeting. The COS-STAR Statement consists of a checklist of 18 items considered essential for transparent and complete reporting in all COS studies. The checklist items focus on the introduction, methods, results, and discussion section of a manuscript describing the development of a particular COS. A limitation of the COS-STAR Statement is that it was developed without representative views of low- and middle-income countries. COS have equal relevance to studies conducted in these areas, and, subsequently, this guideline may need to evolve over time to encompass any additional challenges from developing COS in these areas. Conclusions With many ongoing COS studies underway, the COS-STAR Statement should be a helpful resource to improve the reporting of COS studies for the benefit of all COS users.

Choosing important health outcomes for comparative effectiveness research: An updated review and user survey.

Background A COS represents an agreed minimum set of outcomes that should be measured and reported in all trials of a specific condition. The COMET (Core Outcome Measures in Effectiveness Trials) initiative aims to collate and stimulate the development and application of COS, by including data on relevant studies within a publically available internet-based resource. In recent years, there has been an interest in increasing the development of COS. Therefore, this study aimed to provide an update of a previous review, and examine the quality of development of COS. A further aim was to understand the reasons why individuals are searching the COMET database. Methods A multi-faceted search strategy was followed, in order to identify studies that sought to determine which outcomes/domains to measure in clinical trials of a specific condition. Additionally, a pop up survey was added to the COMET website, to ascertain why people were searching the COMET database. Results Thirty-two reports relating to 29 studies were eligible for inclusion in the review. There has been an improvement in the description of the scope of a COS and an increase in the proportion of studies using literature/systematic reviews and the Delphi technique. Clinical experts continue to be the most common group involved in developing COS, however patient and public involvement has increased. The pop-up survey revealed the most common reasons for visiting the COMET website to be thinking about developing a COS and planning a clinical trial. Conclusions This update demonstrates that recent studies appear to have adopted a more structured approach towards COS development and public representation has increased. However, there remains a need for developers to adequately describe details about the scope of COS, and for greater public engagement. The COMET database appears to be a useful resource for both COS developers and users of COS.

Home telehealth uptake and continued use among heart failure and chronic obstructive pulmonary disease patients: a systematic review.

BackgroundHome telehealth has the potential to benefit heart failure (HF) and chronic obstructive pulmonary disease (COPD) patients, however large-scale deployment is yet to be achieved.PurposeThe aim of this review was to assess levels of uptake of home telehealth by patients with HF and COPD and the factors that determine whether patients do or do not accept and continue to use telehealth.MethodsThis research performs a narrative synthesis of the results from included studies.ResultsThirty-seven studies met the inclusion criteria. Studies that reported rates of refusal and/or withdrawal found that almost one third of patients who were offered telehealth refused and one fifth of participants who did accept later abandoned telehealth. Seven barriers to, and nine facilitators of, home telehealth use were identified.ConclusionsResearch reports need to provide more details regarding telehealth refusal and abandonment, in order to understand the reasons why patients decide not to use telehealth.

COS-STAR: a reporting guideline for studies developing core outcome sets (protocol)

BackgroundCore outcome sets can increase the efficiency and value of research and, as a result, there are an increasing number of studies looking to develop core outcome sets (COS). However, the credibility of a COS depends on both the use of sound methodology in its development and clear and transparent reporting of the processes adopted. To date there is no reporting guideline for reporting COS studies. The aim of this programme of research is to develop a reporting guideline for studies developing COS and to highlight some of the important methodological considerations in the process.Methods/DesignThe study will include a reporting guideline item generation stage which will then be used in a Delphi study. The Delphi study is anticipated to include two rounds. The first round will ask stakeholders to score the items listed and to add any new items they think are relevant. In the second round of the process, participants will be shown the distribution of scores for all stakeholder groups separately and asked to re-score. A final consensus meeting will be held with an expert panel and stakeholder representatives to review the guideline item list. Following the consensus meeting, a reporting guideline will be drafted and review and testing will be undertaken until the guideline is finalised. The final outcome will be the COS-STAR (Core Outcome Set-STAndards for Reporting) guideline for studies developing COS and a supporting explanatory document.DiscussionTo assess the credibility and usefulness of a COS, readers of a COS development report need complete, clear and transparent information on its methodology and proposed core set of outcomes. The COS-STAR guideline will potentially benefit all stakeholders in COS development: COS developers, COS users, e.g. trialists and systematic reviewers, journal editors, policy-makers and patient groups.

Improving the relevance and consistency of outcomes in comparative effectiveness research

Policy makers have clearly indicated--through heavy investment in the Patient Centered Outcomes Research Institute--that reporting outcomes that are meaningful to patients is crucial for improvement in healthcare delivery and cost reduction. Better interpretation and generalizability of clinical research results that incorporate patient-centered outcomes research can be achieved by accelerating the development and uptake of core outcome sets (COS). COS provide a standardized minimum set of the outcomes that should be measured and reported in all clinical trials of a specific condition. The level of activity around COS has increased significantly over the past decade, with substantial progress in several clinical domains. However, there are many important clinical conditions for which high-quality COS have not been developed and there are limited resources and capacity with which to develop them. We believe that meaningful progress toward the goals behind the significant investments in patient-centered outcomes research and comparative effectiveness research will depend on a serious effort to address these issues.

Choosing Important Health Outcomes for Comparative Effectiveness Research: An Updated Review and Identification of Gaps

Background The COMET (Core Outcome Measures in Effectiveness Trials) Initiative promotes the development and application of core outcome sets (COS), including relevant studies in an online database. In order to keep the database current, an annual search of the literature is undertaken. This study aimed to update a previous systematic review, in order to identify any further studies where a COS has been developed. Furthermore, no prioritization for COS development has previously been undertaken, therefore this study also aimed to identify COS relevant to the world’s most prevalent health conditions. Methods The methods used in this updated review followed the same approach used in the original review and the previous update. A survey was also sent to the corresponding authors of COS identified for inclusion in this review, to ascertain what lessons they had learnt from developing their COS. Additionally, the COMET database was searched to identify COS that might be relevant to the conditions with the highest global prevalence. Results Twenty-five reports relating to 22 new studies were eligible for inclusion in the review. Further improvements were identified in relation to the description of the scope of the COS, use of the Delphi technique, and the inclusion of patient participants within the development process. Additionally, 33 published and ongoing COS were identified for 13 of the world’s most prevalent conditions. Conclusion The development of a reporting guideline and minimum standards should contribute towards future improvements in development and reporting of COS. This study has also described a first approach to identifying gaps in existing COS, and to priority setting in this area. Important gaps have been identified, on the basis of global burden of disease, and the development and application of COS in these areas should be considered a priority.

COMET V meeting summary

On 20-21st May 2015, more than 150 people with an interest in core outcome sets (COS) gathered at the University of Calgary in Alberta, Canada for the fifth meeting of the COMET Initiative. This was the first annual COMET meeting in North America, and the COMET Management Group are grateful to Cochrane Canada for facilitating the meeting and an excellent joint session on the second day. As well as participants from Canada and the USA, people came from Australia, Brazil, Germany, Portugal and the UK. Over the next two days, the invited plenary talks were complemented by workshops, posters and contributed presentations. Theresa Radwell (Alberta Cancer Foundation) opened the meeting, welcoming all to Calgary and introducing the importance of engaging patients within research and outcome selection. Paula Williamson (COMET Management Group) then spoke about the COMET Initiative, emphasising that COMET is keen to avoid unnecessary duplication of effort and to facilitate the development of COS. The participants were then introduced to important methodological issues in COS through a series of presentations. John Marshall (St. Michaels Hospital, Toronto) provided a critical care perspective and highlighted that mortality is not always the most important outcome from a patient perspective. Amy Hoang-Kim (University of Toronto) presented a recommendation for a minimal set of core domains for use in distal radius fracture clinical practice and research. Moving on to nephrology, Jonathan Craig and Allison Tong (University of Sydney) overviewed existing standardized outcomes, with preliminary results showing how dialysis free time was the most important outcome to haemodialysis patients. One of the novel additions in COMET V was a panel discussion showing the importance of COS to different stakeholders. John Fletcher (Canadian Medical Association Journal) described the pros and cons of COS from an editors perspective. Jordi Pardo (OMERACT) outlined the OMERACT process for developing a COS. Carole Legare (Health Canada) identified the problems seen by regulators because of inconsistency of safety reporting. John Marshall (Canadian Critical Care Trials Group) spoke about challenging issues faced by the critical care research community. Mike Clarke (COMET) brought all of this together by highlighting the resources that are available through COMET to assist in the development and evaluation of COS. The ensuing discussion highlighted the benefits of COS for journals, how stakeholder involvement and international harmonisation are essential to COS development, the need to consider barriers to uptake of COS for researchers, and recognition of the need for a COS that is in no way restrictive. The afternoon began with David Moher (Ottawa Hospital Research Institute) speaking about the EQUATOR network, which aims to maximise the value of research by improving conduct and reporting. David highlighted how the evaluation of reporting guidleines and COS is critical. He was followed by a series of presentations which centred around outcomes for paediatric trials. Zafira Bhaloo (University of Alberta) emphasised how the reporting of primary outcomes in pediatric trials is inadequate and encouraged higher standards for reporting and informed selection of outcomes and their measures. Michele Hamm (University of Alberta) discussed how the use of social media to identify patient-centred outcomes in child health did not result in broad reach as a stakeholder engagement strategy. Mufiza Kapadia (The Hospital for Sick Children) ended the session by stressing the importance of involving parents and children in COS development. Alongside the 17 posters that were available for viewing throughout the first day, four of the people who had submitted abstracts had been selected to give a contributed talk. These began with Carina Benstom (University Hospital RWTH Aachen) who highlighted how the problems caused by inconsistent outcome measures in clinical trials are hardly recognised. Chris Hylton (PaCER) spoke about improvements in the results of patient experiences and outcome analysis, from involving patient and community engagement researchers. Sally Crowe (Crowe Associates Ltd) continued the patient theme, by speaking about how workshops offer context and depth for talking about outcomes. The final contributed talk came from Thomas Kelley (International Consortium for Health Outcomes Measurement) who explained how ICHOMs mission is to define global standard sets of outcome measures that really matter to patients for the most relevant medical conditions. Mike Clarke (COMET Management Group) closed the first day with a presentation about COMET in Canada, which highlighted what Canada can do to increase the use of COS in research, for example by helping to persuade funders that COS should be used in research. The second day (21st May) was shared with Cochrane Canada and the opening of their Annual Symposium. The opening of the joint session was marked by memories of Dave Sackett and his contribution to evidence based healthcare. Following a minutes silence for the many friends, colleagues and admirers of Dave to remember him, the scientific session began with Kay Dickersin (John Hopkins University) highlighting how groundwork needs to be laid in subject areas where there are a lack of COS and how new methods need to be explored for developing COS. Mike presented findings from a survey of outcomes in Cochrane Reviews, showing the wide variation in outcomes and the lack of COS, at least up to 2013. Holger Schunemann (McMaster University) closed the plenary session by providing an overview of the GRADE evidence to decision frameworks. Following a break, the participants headed off for one of three COMET workshops. Paula Williamson led the first of these, which focused on the methods for developing what to measure in COS. The workshop introduced methodological issues and considerations involved in developing COS. Workshop 2 was led by Mike Clarke and looked at how COS might be used for randomised trials and Cochrane Reviews. Bridget Young (University of Liverpool) led the third workshop providing an interactive opportunity for the participants to identify the challenges that researchers may encounter when planning to involve patients and carers in COS development. COMET V allowed a wide variety of stakeholders with an interest in COS development to meet and share experiences, findings, and plans with others. It brought together key scientists and consumers responsible for developing and implementing COS. Patient involvement emerged as a major focus of the meeting with an emphasis on engaging the relevant stakeholders early in the process of COS development. Thoughts were offered for how COMET can evolve both in Canada and the rest of the world. And, challenging questions were posed throughout the meeting, including: How can we ensure that COS are well developed in the first place? Is there a magic number of outcomes to be included in a COS and, if so, what is it? As COMET looks forward to COMET VI, it will seek to meet these challenges, guided by an International Advisory Group, which will include Peter Tugwell (University of Ottawa), one of the founders of OMERACT. The slides from COMET V presentations can be viewed at: http://www.comet-initiative.org/events/FifthCometMeeting.

Choosing important health outcomes for comparative effectiveness research: An updated systematic review and involvement of low and middle income countries

Background Core outcome sets (COS) comprise a minimum set of outcomes that should be measured and reported in all trials for a specific health condition. The COMET (Core Outcome Measures in Effectiveness Trials) Initiative maintains an up to date, publicly accessible online database of published and ongoing COS. An annual systematic review update is an important part of this process. Methods This review employed the same, multifaceted approach that was used in the original review and the previous two updates. This approach has identified studies that sought to determine which outcomes/domains to measure in clinical trials of a specific condition. This update includes an analysis of the inclusion of participants from low and middle income countries (LMICs) as identified by the OECD, in these COS. Results Eighteen publications, relating to 15 new studies describing the development of 15 COS, were eligible for inclusion in the review. Results show an increase in the use of mixed methods, including Delphi surveys. Clinical experts remain the most common stakeholder group involved. Overall, only 16% of the 259 COS studies published up to the end of 2016 have included participants from LMICs. Conclusion This review highlights opportunities for greater public participation in COS development and the involvement of stakeholders from a wider range of geographical settings, in particular LMICs.

The core outcome measures in effectiveness trials (COMET) initiative: five years on

Core outcome sets (COS) can reduce waste in research, through measurement of an agreed set of outcomes across all trials in a particular area of health. The COMET Initiative was launched in 2010 to (i) raise awareness of problems with outcomes in trials; (ii) encourage COS development and uptake; (iii) promote patient and public involvement in COS development; (iv) provide resources to facilitate this; and (v) encourage evidence-based COS development. This talk will review progress and challenges over our first five years. It will describe our work with multiple stakeholders to facilitate engagement, including funders, trialists, patient organisations, systematic reviewers, editors, industry, regulators, and guideline developers. The perception that COMET was a ‘UK thing’ has been dispelled, with the establishment of an International Advisory Group and meetings in Rome (2014) and Calgary (2015). Highlights include the development of a searchable repository of COS studies, and completion of the first comprehensive systematic review of published COS (recently updated to December 2014). Over 7800 searches have been done of the online repository, most visitors in 2014 were from outside the UK and we will present findings from a pop-up survey of why people search it. Promoting broader uptake of COS by researchers and securing funding for initiatives aimed at reducing waste in research remain major challenges. As awareness of the need for COS continues to grow and knowledge of the COMET Initiative increases, this is an appropriate moment to present our future proposals, including a research agenda, to the trials methodology community.