Alexander D. Rushforth

Accounting for Impact? The Journal Impact Factor and the Making of Biomedical Research in the Netherlands

The range and types of performance metrics has recently proliferated in academic settings, with bibliometric indicators being particularly visible examples. One field that has traditionally been hospitable towards such indicators is biomedicine. Here the relative merits of bibliometrics are widely discussed, with debates often portraying them as heroes or villains. Despite a plethora of controversies, one of the most widely used indicators in this field is said to be the Journal Impact Factor (JIF). In this article we argue that much of the current debates around researchers’ uses of the JIF in biomedicine can be classed as ‘folk theories’: explanatory accounts told among a community that seldom (if ever) get systematically checked. Such accounts rarely disclose how knowledge production itself becomes more-or-less consolidated around the JIF. Using ethnographic materials from different research sites in Dutch University Medical Centers, this article sheds new empirical and theoretical light on how performance metrics variously shape biomedical research on the ‘shop floor.’ Our detailed analysis underscores a need for further research into the constitutive effects of evaluative metrics.

To intervene or not to intervene; is that the question? On the role of scientometrics in research evaluation

Recent high‐profile statements, criticisms, and boycotts organized against certain quantitative indicators (e.g., the DORA declaration) have brought misuses of performance metrics to the center of attention. A key concern captured in these movements is that the metrics appear to carry authority even where established agents of quality control have explicitly outlined limits to their validity and reliability as measurement tools. This raises a number of challenging questions for those readers of this journal who are implicated in questions of indicator “production” and, by extension, “effects.” In this opinion piece we wish to critically engage the question of how producers of indicators can come to terms with their role as (partly) responsible parties in the current age of evaluative bibliometrics. We do so through the illuminating case of the professional scientometrics community.

Indicators as Judgment Devices: An Empirical Study of Citizen Bibliometrics in Research Evaluation.

A researcher’s number of publications has been a fundamental merit in the competition for academic positions since the late 18th century. Today, the simple counting of publications has been supplem ...

Maximising value from a United Kingdom Biomedical Research Centre: study protocol.

BackgroundBiomedical Research Centres (BRCs) are partnerships between healthcare organisations and universities in England. Their mission is to generate novel treatments, technologies, diagnostics and other interventions that increase the country’s international competitiveness, to rapidly translate these innovations into benefits for patients, and to improve efficiency and reduce waste in healthcare. As NIHR Oxford BRC (Oxford BRC) enters its third 5-year funding period, we seek to (1) apply the evidence base on how best to support the various partnerships in this large, multi-stakeholder research system and (2) research how these partnerships play out in a new, ambitious programme of translational research.MethodsOrganisational case study, informed by the principles of action research. A cross-cutting theme, ‘Partnerships for Health, Wealth and Innovation’ has been established with multiple sub-themes (drug development, device development, business support and commercialisation, research methodology and statistics, health economics, bioethics, patient and public involvement and engagement, knowledge translation, and education and training) to support individual BRC research themes and generate cross-theme learning.The ‘Partnerships’ theme will support the BRC’s goals by facilitating six types of partnership (with patients and citizens, clinical services, industry, across the NIHR infrastructure, across academic disciplines, and with policymakers and payers) through a range of engagement platforms and activities. We will develop a longitudinal progress narrative centred around exemplar case studies, and apply theoretical models from innovation studies (Triple Helix), sociology of science (Mode 2 knowledge production) and business studies (Value Co-creation). Data sources will be the empirical research studies within individual BRC research themes (who will apply separately for NHS ethics approval), plus documentary analysis and interviews and ethnography with research stakeholders. This study has received ethics clearance through the University of Oxford Central University Research Ethics Committee.DiscussionWe anticipate that this work will add significant value to Oxford BRC. We predict accelerated knowledge translation; closer alignment of the innovation process with patient priorities and the principles of responsible, ethical research; reduction in research waste; new knowledge about the governance and activities of multi-stakeholder research partnerships and the contexts in which they operate; and capacity-building that reflects the future needs of a rapidly-evolving health research system.

Portfolios of Worth: Capitalizing on Basic and Clinical Problems in Biomedical Research Groups

How are “interesting” research problems identified and made durable by academic researchers, particularly in situations defined by multiple evaluation principles? Building on two case studies of research groups working on rare diseases in academic biomedicine, we explore how group leaders arrange their groups to encompass research problems that latch onto distinct evaluation principles by dividing and combining work into “basic-oriented” and “clinical-oriented” spheres of inquiry. Following recent developments in the sociology of (e)valuation comparing academics to capitalist entrepreneurs in pursuit of varying kinds of worth, we argue that the metaphor of the portfolio is helpful in analyzing how group leaders manage these different research lines as “alternative investment options” from which they were variously hoping to capitalize. We argue portfolio development is a useful concept for exploring how group leaders fashion “entrepreneurial” practices to manage and exploit tensions between multiple matrices of (e)valuation and conclude with suggestions for how this vocabulary can further extend analysis of epistemic capitalism within science and technology studies.