Justin Jagosh

Uncovering the Benefits of Participatory Research: Implications of a Realist Review for Health Research and Practice

Context Participatory research (PR) is the co-construction of research through partnerships between researchers and people affected by and/or responsible for action on the issues under study. Evaluating the benefits of PR is challenging for a number of reasons: the research topics, methods, and study designs are heterogeneous; the extent of collaborative involvement may vary over the duration of a project and from one project to the next; and partnership activities may generate a complex array of both short- and long-term outcomes. Methods Our review team consisted of a collaboration among researchers and decision makers in public health, research funding, ethics review, and community-engaged scholarship. We identified, selected, and appraised a large-variety sample of primary studies describing PR partnerships, and in each stage, two team members independently reviewed and coded the literature. We used key realist review concepts (middle-range theory, demi-regularity, and context-mechanism-outcome configurations [CMO]) to analyze and synthesize the data, using the PR partnership as the main unit of analysis. Findings From 7,167 abstracts and 591 full-text papers, we distilled for synthesis a final sample of twenty-three PR partnerships described in 276 publications. The link between process and outcome in these partnerships was best explained using the middle-range theory of partnership synergy, which demonstrates how PR can (1) ensure culturally and logistically appropriate research, (2) enhance recruitment capacity, (3) generate professional capacity and competence in stakeholder groups, (4) result in productive conflicts followed by useful negotiation, (5) increase the quality of outputs and outcomes over time, (6) increase the sustainability of project goals beyond funded time frames and during gaps in external funding, and (7) create system changes and new unanticipated projects and activities. Negative examples illustrated why these outcomes were not a guaranteed product of PR partnerships but were contingent on key aspects of context. Conclusions We used a realist approach to embrace the heterogeneity and complexity of the PR literature. This theory-driven synthesis identified mechanisms by which PR may add value to the research process. Using the middle-range theory of partnership synergy, our review confirmed findings from previous PR reviews, documented and explained some negative outcomes, and generated new insights into the benefits of PR regarding conflicts and negotiation between stakeholders, program sustainability and advancement, unanticipated project activity, and the generation of systemic change.

Testing the reliability and efficiency of the pilot Mixed Methods Appraisal Tool (MMAT) for systematic mixed studies review

BACKGROUND Systematic literature reviews identify, select, appraise, and synthesize relevant literature on a particular topic. Typically, these reviews examine primary studies based on similar methods, e.g., experimental trials. In contrast, interest in a new form of review, known as mixed studies review (MSR), which includes qualitative, quantitative, and mixed methods studies, is growing. In MSRs, reviewers appraise studies that use different methods allowing them to obtain in-depth answers to complex research questions. However, appraising the quality of studies with different methods remains challenging. To facilitate systematic MSRs, a pilot Mixed Methods Appraisal Tool (MMAT) has been developed at McGill University (a checklist and a tutorial), which can be used to concurrently appraise the methodological quality of qualitative, quantitative, and mixed methods studies. OBJECTIVES The purpose of the present study is to test the reliability and efficiency of a pilot version of the MMAT. METHODS The Center for Participatory Research at McGill conducted a systematic MSR on the benefits of Participatory Research (PR). Thirty-two PR evaluation studies were appraised by two independent reviewers using the pilot MMAT. Among these, 11 (34%) involved nurses as researchers or research partners. Appraisal time was measured to assess efficiency. Inter-rater reliability was assessed by calculating a kappa statistic based on dichotomized responses for each criterion. An appraisal score was determined for each study, which allowed the calculation of an overall intra-class correlation. RESULTS On average, it took 14 min to appraise a study (excluding the initial reading of articles). Agreement between reviewers was moderate to perfect with regards to MMAT criteria, and substantial with respect to the overall quality score of appraised studies. CONCLUSION The MMAT is unique, thus the reliability of the pilot MMAT is promising, and encourages further development.

A realist evaluation of community-based participatory research: partnership synergy, trust building and related ripple effects

BackgroundCommunity-Based Participatory Research (CBPR) is an approach in which researchers and community stakeholders form equitable partnerships to tackle issues related to community health improvement and knowledge production. Our 2012 realist review of CBPR outcomes reported long-term effects that were touched upon but not fully explained in the retained literature. To further explore such effects, interviews were conducted with academic and community partners of partnerships retained in the review. Realist methodology was used to increase the understanding of what supports partnership synergy in successful long-term CBPR partnerships, and to further document how equitable partnerships can result in numerous benefits including the sustainability of relationships, research and solutions.MethodsBuilding on our previous realist review of CBPR, we contacted the authors of longitudinal studies of academic-community partnerships retained in the review. Twenty-four participants (community members and researchers) from 11 partnerships were interviewed. Realist logic of analysis was used, involving middle-range theory, context-mechanism-outcome configuration (CMOcs) and the concept of the ‘ripple effect’.ResultsThe analysis supports the central importance of developing and strengthening partnership synergy through trust. The ripple effect concept in conjunction with CMOcs showed that a sense of trust amongst CBPR members was a prominent mechanism leading to partnership sustainability. This in turn resulted in population-level outcomes including: (a) sustaining collaborative efforts toward health improvement; (b) generating spin-off projects; and (c) achieving systemic transformations.ConclusionThese results add to other studies on improving the science of CBPR in partnerships with a high level of power-sharing and co-governance. Our results suggest sustaining CBPR and achieving unanticipated benefits likely depend on trust-related mechanisms and a continuing commitment to power-sharing. These findings have implications for building successful CBPR partnerships to address challenging public health problems and the complex assessment of outcomes.

Assessing the outcomes of participatory research: protocol for identifying, selecting, appraising and synthesizing the literature for realist review

BackgroundParticipatory Research (PR) entails the co-governance of research by academic researchers and end-users. End-users are those who are affected by issues under study (e.g., community groups or populations affected by illness), or those positioned to act on the knowledge generated by research (e.g., clinicians, community leaders, health managers, patients, and policy makers). Systematic reviews assessing the generalizable benefits of PR must address: the diversity of research topics, methods, and intervention designs that involve a PR approach; varying degrees of end-user involvement in research co-governance, both within and between projects; and the complexity of outcomes arising from long-term partnerships.MethodsWe addressed the above mentioned challenges by adapting realist review methodology to PR assessment, specifically by developing inductively-driven identification, selection, appraisal, and synthesis procedures. This approach allowed us to address the non-uniformity and complexity of the PR literature. Each stage of the review involved two independent reviewers and followed a reproducible, systematic coding and retention procedure. Retained studies were completed participatory health interventions, demonstrated high levels of participation by non-academic stakeholders (i.e., excluding studies in which end-users were not involved in co-governing throughout the stages of research) and contained detailed descriptions of the participatory process and context. Retained sets are being mapped and analyzed using realist review methods.ResultsThe librarian-guided search string yielded 7,167 citations. A total of 594 citations were retained after the identification process. Eighty-three papers remained after selection. Principle Investigators (PIs) were contacted to solicit all companion papers. Twenty-three sets of papers (23 PR studies), comprising 276 publications, passed appraisal and are being synthesized using realist review methods.DiscussionThe systematic and stage-based procedure addressed challenges to PR assessment and generated our robust understanding of complex and heterogeneous PR practices. To date, realist reviews have focussed on evaluations of relatively uniform interventions. In contrast our PR search yielded a wide diversity of partnerships and research topics. We therefore developed tools to achieve conceptual clarity on the PR field, as a beneficial precursor to our theoretically-driven synthesis using realist methods. Findings from the ongoing review will be provided in forthcoming publications.

Assessing the benefits of participatory research: a rationale for a realist review

Participatory research (PR) experts believe that increased community and stakeholder participation in research augments program pertinence, quality, outcome, sustainability, uptake, and transferability. There is, however, a dearth of assessments and measurement tools to demonstrate the contribution of participation in health research and interventions. One systematic review of PR, conducted for the Agency for Health Research and Quality (AHRQ), provided no conclusive evidence concerning the benefits of community participation to enhance research and health outcomes. To overcome methodological gaps and barriers of the AHRQ review, we propose to conduct a systematic realist review, which can be understood as a theory-driven qualitative review capable of capturing the often complex, diffuse and obtuse evidence concerning participation. Reviewing how PR mechanisms and contextual factors mediate and moderate outcomes, the review will generate and test hypotheses (middle-range theories) conceptualizing the benefits of participation and will portray the manner and circumstances in which participation influences outcomes.

Protocol—the RAMESES II study: developing guidance and reporting standards for realist evaluation

Introduction Realist evaluation is an increasingly popular methodology in health services research. For realist evaluations (RE) this project aims to: develop quality and reporting standards and training materials; build capacity for undertaking and critically evaluating them; produce resources and training materials for lay participants, and those seeking to involve them. Methods To achieve our aims, we will: (1) Establish management and governance infrastructure; (2) Recruit an interdisciplinary Delphi panel of 35 participants with diverse relevant experience of RE; (3) Summarise current literature and expert opinion on best practice in RE; (4) Run an online Delphi panel to generate and refine items for quality and reporting standards; (5) Capture ‘real world’ experiences and challenges of RE—for example, by providing ongoing support to realist evaluations, hosting the RAMESES JISCmail list on realist research, and feeding problems and insights from these into the deliberations of the Delphi panel; (6) Produce quality and reporting standards; (7) Collate examples of the learning and training needs of researchers, students, reviewers and lay members in relation to RE; (8) Develop, deliver and evaluate training materials for RE and deliver training workshops; and (9) Develop and evaluate information and resources for patients and other lay participants in RE (eg, draft template information sheets and model consent forms) and; (10) Disseminate training materials and other resources. Planned outputs: (1) Quality and reporting standards and training materials for RE. (2) Methodological support for RE. (3) Increase in capacity to support and evaluate RE. (4) Accessible, plain-English resources for patients and the public participating in RE. Discussion The realist evaluation is a relatively new approach to evaluation and its overall place in the is not yet fully established. As with all primary research approaches, guidance on quality assurance and uniform reporting is an important step towards improving quality and consistency.

Lost and Found in Translation: An Ecological Approach to Bilingual Research Methodology

Translation issues emerged from a qualitative study, conducted in French and English, that gathered patient perspectives on a newly implemented undergraduate medical curriculum entitled Physicianship: The Physician as Professional and Healer. French-speaking participants were interviewed using a translated interview guide, originally developed in English. A major finding that francophone participants contested the idea of the physician-healer in a manner not witnessed among the anglophone participants. Consultation with multilingual health professionals was undertaken to explore whether the contestation was the result of poor translation of the word healer. This process confirmed that no appropriate French equivalent could be found. With hindsight, the authors emphasize the importance of pretesting translated research instrumentation. An ecological perspective on language equivalency is also emphasized, in which emergent linguistic discrepancies are viewed as opportunities for learning about the culture-language relationship.

A realist review of shared medical appointments: How, for whom, and under what circumstances do they work?

BackgroundShared medical appointments (SMAs) are doctor-patient visits in which groups of patients are seen by one or more health care providers in a concurrent session. There is a growing interest in understanding the potential benefits of SMAs in various contexts to improve clinical outcomes and reduce healthcare costs. This study builds upon the existing evidence base that suggests SMAs are indeed effective. In this study, we explored how they are effective in terms of the underlying mechanisms of action and under what circumstances.MethodsRealist review methodology was used to synthesize the literature on SMAs, which included a broad search of 800+ published articles. 71 high quality primary research articles were retained to build a conceptual model of SMAs and 20 of those were selected for an in depth analysis using realist methodology (i.e.,middle-range theories and and context-mechanism-outcome configurations).ResultsNine main mechanisms that serve to explain how SMAs work were theorized from the data immersion process and configured in a series of context-mechanism-outcome configurations (CMOs). These are: (1) Group exposure in SMAs combats isolation, which in turn helps to remove doubts about one’s ability to manage illness; (2) Patients learn about disease self-management vicariously by witnessing others’ illness experiences; (3) Patients feel inspired by seeing others who are coping well; (4) Group dynamics lead patients and providers to developing more equitable relationships; (5) Providers feel increased appreciation and rapport toward colleagues leading to increased efficiency; (6) Providers learn from the patients how better to meet their patients’ needs; (7) Adequate time allotment of the SMA leads patients to feel supported; (8) Patients receive professional expertise from the provider in combination with first-hand information from peers, resulting in more robust health knowledge; and (9) Patients have the opportunity to see how the physicians interact with fellow patients, which allows them to get to know the physician and better determine their level of trust.ConclusionsNine overarching mechanisms were configured in CMO configurations and discussed as a set of complementary middle-range programme theories to explain how SMAs work. It is anticipated that this innovative work in theorizing SMAs using realist review methodology will provide policy makers and SMA program planners adequate conceptual grounding to design contextually sensitive SMA programs in a wide variety of settings and advance an SMA research agenda for varied contexts.

Uncovering the Benefits of Participatory Research: Implications of a Realist Review for Health Research and Practice: A Realist Review for Health Research and Practice

Context Participatory research (PR) is the co-construction of research through partnerships between researchers and people affected by and/or responsible for action on the issues under study. Evaluating the benefits of PR is challenging for a number of reasons: the research topics, methods, and study designs are heterogeneous; the extent of collaborative involvement may vary over the duration of a project and from one project to the next; and partnership activities may generate a complex array of both short- and long-term outcomes. Methods Our review team consisted of a collaboration among researchers and decision makers in public health, research funding, ethics review, and community-engaged scholarship. We identified, selected, and appraised a large-variety sample of primary studies describing PR partnerships, and in each stage, two team members independently reviewed and coded the literature. We used key realist review concepts (middle-range theory, demi-regularity, and context-mechanism-outcome configurations [CMO]) to analyze and synthesize the data, using the PR partnership as the main unit of analysis. Findings From 7,167 abstracts and 591 full-text papers, we distilled for synthesis a final sample of twenty-three PR partnerships described in 276 publications. The link between process and outcome in these partnerships was best explained using the middle-range theory of partnership synergy, which demonstrates how PR can (1) ensure culturally and logistically appropriate research, (2) enhance recruitment capacity, (3) generate professional capacity and competence in stakeholder groups, (4) result in productive conflicts followed by useful negotiation, (5) increase the quality of outputs and outcomes over time, (6) increase the sustainability of project goals beyond funded time frames and during gaps in external funding, and (7) create system changes and new unanticipated projects and activities. Negative examples illustrated why these outcomes were not a guaranteed product of PR partnerships but were contingent on key aspects of context. Conclusions We used a realist approach to embrace the heterogeneity and complexity of the PR literature. This theory-driven synthesis identified mechanisms by which PR may add value to the research process. Using the middle-range theory of partnership synergy, our review confirmed findings from previous PR reviews, documented and explained some negative outcomes, and generated new insights into the benefits of PR regarding conflicts and negotiation between stakeholders, program sustainability and advancement, unanticipated project activity, and the generation of systemic change.

Uncovering the benefits of participatory research

Context Participatory research (PR) is the co-construction of research through partnerships between researchers and people affected by and/or responsible for action on the issues under study. Evaluating the benefits of PR is challenging for a number of reasons: the research topics, methods, and study designs are heterogeneous; the extent of collaborative involvement may vary over the duration of a project and from one project to the next; and partnership activities may generate a complex array of both short- and long-term outcomes. Methods Our review team consisted of a collaboration among researchers and decision makers in public health, research funding, ethics review, and community-engaged scholarship. We identified, selected, and appraised a large-variety sample of primary studies describing PR partnerships, and in each stage, two team members independently reviewed and coded the literature. We used key realist review concepts (middle-range theory, demi-regularity, and context-mechanism-outcome configurations [CMO]) to analyze and synthesize the data, using the PR partnership as the main unit of analysis. Findings From 7,167 abstracts and 591 full-text papers, we distilled for synthesis a final sample of twenty-three PR partnerships described in 276 publications. The link between process and outcome in these partnerships was best explained using the middle-range theory of partnership synergy, which demonstrates how PR can (1) ensure culturally and logistically appropriate research, (2) enhance recruitment capacity, (3) generate professional capacity and competence in stakeholder groups, (4) result in productive conflicts followed by useful negotiation, (5) increase the quality of outputs and outcomes over time, (6) increase the sustainability of project goals beyond funded time frames and during gaps in external funding, and (7) create system changes and new unanticipated projects and activities. Negative examples illustrated why these outcomes were not a guaranteed product of PR partnerships but were contingent on key aspects of context. Conclusions We used a realist approach to embrace the heterogeneity and complexity of the PR literature. This theory-driven synthesis identified mechanisms by which PR may add value to the research process. Using the middle-range theory of partnership synergy, our review confirmed findings from previous PR reviews, documented and explained some negative outcomes, and generated new insights into the benefits of PR regarding conflicts and negotiation between stakeholders, program sustainability and advancement, unanticipated project activity, and the generation of systemic change.