Pierre-Olivier Bédard

Are the h-index and some of its alternatives discriminatory of epistemological beliefs and methodological preferences of faculty members? The case of social scientists in Quebec

This exploratory study aims at answering the following research question: Are the h-index and some of its derivatives discriminatory when applied to rank social scientists with different epistemological beliefs and methodological preferences? This study reports the results of five Tobit and two negative binomial regression models taking as dependent variable the h-index and six of its derivatives, using a dataset combining bibliometric data collected with the PoP software with cross-sectional data of 321 Quebec social scientists in Anthropology, Sociology, Social Work, Political Science, Economics and Psychology. The results reveal an epistemological/methodological effect making positivists and quantitativists globally more productive than constructivists and qualitativists.

Persistent misunderstandings about evidence-based (sorry: informed!) policy-making

BackgroundThe field of research on knowledge mobilization and evidence-informed policy-making has seen enduring debates related to various fundamental assumptions such as the definition of ‘evidence’, the relative validity of various research methods, the actual role of evidence to inform policy-making, etc. In many cases, these discussions serve a useful purpose, but they also stem from serious disagreement on methodological and epistemological issues.DiscussionThis essay reviews the rationale for evidence-informed policy-making by examining some of the common claims made about the aims and practices of this perspective on public policy. Supplementing the existing justifications for evidence-based policy making, we argue in favor of a greater inclusion of research evidence in the policy process but in a structured fashion, based on methodological considerations. In this respect, we present an overview of the intricate relation between policy questions and appropriate research designs.SummaryBy closely examining the relation between research questions and research designs, we claim that the usual points of disagreement are mitigated. For instance, when focusing on the variety of research designs that can answer a range of policy questions, the common critical claim about ‘RCT-based policy-making’ seems to lose some, if not all of its grip.

The Mobilization of Scientific Evidence by Public Policy Analysts

Research on knowledge mobilization in policy making has been largely focused on identifying relevant factors having an effect on the uptake of evidence by actors and organizations. However, evidence on the magnitude of those effects remains limited and existing methods allowing for this have been scarcely used in this field. In this article, we first provide a rationale for greater investigation of substantive effect sizes, using methods such as mediation analysis and conditional probabilities. Using cross-sectional data from Québec (Canada) government policy analysts, we test an absorptive capacity model and describe direct, specific indirect, and total effects estimated from a path analysis. The results show that some factors have considerable effects, such as physical access and individual field of training, whereas some mediated relations are worth considering. Finally, we discuss some practical implications with regard to policy making and policy analysis but also the methodological standards of empirical research in this field.

A cross-sectional survey of supports for evidence-informed decision-making in healthcare organisations: a research protocol

BackgroundThis protocol builds on the development of a) a framework that identified the various supports (i.e. positions, activities, interventions) that a healthcare organisation or health system can implement for evidence-informed decision-making (EIDM) and b) a qualitative study that showed the current mix of supports that some Canadian healthcare organisations have in place and the ones that are perceived to facilitate the use of research evidence in decision-making. Based on these findings, we developed a web survey to collect cross-sectional data about the specific supports that regional health authorities and hospitals in two Canadian provinces (Ontario and Quebec) have in place to facilitate EIDM.Methods/designThis paper describes the methods for a cross-sectional web survey among 32 regional health authorities and 253 hospitals in the provinces of Quebec and Ontario (Canada) to collect data on the current mix of organisational supports that these organisations have in place to facilitate evidence-informed decision-making. The data will be obtained through a two-step survey design: a 10-min survey among CEOs to identify key units and individuals in regard to our objectives (step 1) and a 20-min survey among managers of the key units identified in step 1 to collect information about the activities performed by their unit regarding the acquisition, assessment, adaptation and/or dissemination of research evidence in decision-making (step 2). The study will target three types of informants: CEOs, library/documentation centre managers and all other key managers whose unit is involved in the acquisition, assessment, adaptation/packaging and/or dissemination of research evidence in decision-making. We developed an innovative data collection system to increase the likelihood that only the best-informed respondent available answers each survey question. The reporting of the results will be done using descriptive statistics of supports by organisation type and by province.DiscussionThis study will be the first to collect and report large-scale cross-sectional data on the current mix of supports health system organisations in the two most populous Canadian provinces have in place for evidence-informed decision-making. The study will also provide useful information to researchers on how to collect organisation-level data with reduced risk of self-reporting bias.