Sarah de Rijcke

Bibliometrics: The Leiden Manifesto for research metrics

Nutzen Sie diese zehn Grundsatze um Forschung zu bewerten, drangen Diana Hicks, Paul Wouters und Kollegen.

Accounting for Impact? The Journal Impact Factor and the Making of Biomedical Research in the Netherlands

The range and types of performance metrics has recently proliferated in academic settings, with bibliometric indicators being particularly visible examples. One field that has traditionally been hospitable towards such indicators is biomedicine. Here the relative merits of bibliometrics are widely discussed, with debates often portraying them as heroes or villains. Despite a plethora of controversies, one of the most widely used indicators in this field is said to be the Journal Impact Factor (JIF). In this article we argue that much of the current debates around researchers’ uses of the JIF in biomedicine can be classed as ‘folk theories’: explanatory accounts told among a community that seldom (if ever) get systematically checked. Such accounts rarely disclose how knowledge production itself becomes more-or-less consolidated around the JIF. Using ethnographic materials from different research sites in Dutch University Medical Centers, this article sheds new empirical and theoretical light on how performance metrics variously shape biomedical research on the ‘shop floor.’ Our detailed analysis underscores a need for further research into the constitutive effects of evaluative metrics.

To intervene or not to intervene; is that the question? On the role of scientometrics in research evaluation

Recent high‐profile statements, criticisms, and boycotts organized against certain quantitative indicators (e.g., the DORA declaration) have brought misuses of performance metrics to the center of attention. A key concern captured in these movements is that the metrics appear to carry authority even where established agents of quality control have explicitly outlined limits to their validity and reliability as measurement tools. This raises a number of challenging questions for those readers of this journal who are implicated in questions of indicator “production” and, by extension, “effects.” In this opinion piece we wish to critically engage the question of how producers of indicators can come to terms with their role as (partly) responsible parties in the current age of evaluative bibliometrics. We do so through the illuminating case of the professional scientometrics community.

Image as Interface: Consequences for Users of Museum Knowledge

Photographs of objects are ubiquitous in the work and presentation of museums, whether in collection-management infrastructure or in Web-based communication. This article examines the use of images in these settings and traces how they function as interfaces and tools in the production of museum knowledge. Because images are not only the main material presented but also become multilayered objects on which to act in order to access or produce knowledge, they play a key role in the involvement of users with museums. This development is analyzed in the context of the Tropenmuseum (an ethnographic museum in Amsterdam, the Netherlands) based on an ethnographic study of visual practices at and about the museum. Drawing on science and technology studies and new media studies, our theoretically driven analysis demonstrates how images as interfaces provide networked contexts for museum knowledge. The various dimensions of images as interfaces in museums are explored through the questions: How are users engaged by these interfaces? Which skills and strategies are needed for this engagement? What are the consequences of visually mediated interfaces for users of digital knowledge in/about/from museums, archives, and other collections? These developments are discussed in terms of their consequences for how museums view their role.

Taking a good look at why scientific images don't speak for themselves

This review focuses on what scientific images can do, by considering three books in which they are central. We problematize the assumption that images can simply show and that the viewer can simply see what is at hand. Images are neither self-explanatory nor transparent, but rather partake in a specific visual culture. If they are to serve as bridges (e.g. to popularize scientific results), then they are only effective insofar as cultural conventions are shared. Felice Frankels how-to book demonstrates the complexity of making photographs transparent, while keeping these processes backstage. Joseph Dumits anthropological analysis of brain scans focuses on tensions between uses of scans inside and outside the lab. In Matthews and McQuains book, pairings of scans with Shakespearean theatre stand in for neuroscientific and artistic approaches to human nature. The attempt to join these very different spheres shows the dangers of going beyond shared understandings of visual material.

The graphic strategy: the uses and functions of illustrations in Wundt's Grundzüge

Illustrations played an important role in the articulation of Wundt’s experimental program. Focusing on the woodcuts of apparatus and experimental designs in the six editions of his Grundzüge der physiologischen Psychologie (published between 1873 and 1911), we investigate the uses and functions of illustrations in the experimental culture of the physiological and psychological sciences. We will first present some statistics on the increasing number of illustrations Wundt included in each new edition of his handbook. Next we will show how Wundt managed to introduce the material and literary technologies of physiology into the ‘new psychology’. The distribution of Wundt’s material technology will be further demonstrated by highlighting the crucial role of technicians and instrument-makers. We will use Shapin and Schaffer’s notions of the ‘three technologies’ and ‘virtual witnessing’, combined with Latour’s concept of ‘immutable mobiles’, as analytical tools to explore the strategic aspects of Wundt’s illustrations.

From Eminent Men to Excellent Universities: University Rankings as Calculative Devices

Global university rankings have become increasingly important ‘calculative devices’ for assessing the ‘quality’ of higher education and research. Their ability to make characteristics of universities ‘calculable’ is here exemplified by the first proper university ranking ever, produced as early as 1910 by the American psychologist James McKeen Cattell. Our paper links the epistemological rationales behind the construction of this ranking to the sociopolitical context in which Cattell operated: an era in which psychology became institutionalized against the backdrop of the eugenics movement, and in which statistics of science became used to counter a perceived decline in ‘great men.’ Over time, however, the ‘eminent man,’ shaped foremost by heredity and upbringing, came to be replaced by the excellent university as the emblematic symbol of scientific and intellectual strength. We also show that Cattell’s ranking was generative of new forms of the social, traces of which can still be found today in the enactment of ‘excellence’ in global university rankings.

The Drawbacks of Project Funding for Epistemic Innovation: Comparing Institutional Affordances and Constraints of Different Types of Research Funding

Abstract Over the past decades, science funding shows a shift from recurrent block funding towards project funding mechanisms. However, our knowledge of how project funding arrangements influence the organizational and epistemic properties of research is limited. To study this relation, a bridge between science policy studies and science studies is necessary. Recent studies have analyzed the relation between the affordances and constraints of project grants and the epistemic properties of research. However, the potentially very different affordances and constraints of funding arrangements such as awards, prizes and fellowships, have not yet been taken into account. Drawing on eight case studies of funding arrangements in high performing Dutch research groups, this study compares the institutional affordances and constraints of prizes with those of project grants and their effects on organizational and epistemic properties of research. We argue that the prize case studies diverge from project-funded research in three ways: 1) a more flexible use, and adaptation of use, of funds during the research process compared to project grants; 2) investments in the larger organization which have effects beyond the research project itself; and 3), closely related, greater deviation from epistemic and organizational standards. The increasing dominance of project funding arrangements in Western science systems is therefore argued to be problematic in light of epistemic and organizational innovation. Funding arrangements that offer funding without scholars having to submit a project-proposal remain crucial to support researchers and research groups to deviate from epistemic and organizational standards.

Working with Research Integrity—Guidance for Research Performing Organisations: The Bonn PRINTEGER Statement

AbstractThis document presents the Bonn PRINTEGER Consensus Statement: Working with Research Integrity—Guidance for research performing organisations. The aim of the statement is to complement existing instruments by focusing specifically on institutional responsibilities for strengthening integrity. It takes into account the daily challenges and organisational contexts of most researchers. The statement intends to make research integrity challenges recognisable from the work-floor perspective, providing concrete advice on organisational measures to strengthen integrity. The statement, which was concluded February 7th 2018, provides guidance on the following key issues: § 1.Providing information about research integrity§ 2.Providing education, training and mentoring§ 3.Strengthening a research integrity culture§ 4.Facilitating open dialogue§ 5.Wise incentive management§ 6.Implementing quality assurance procedures§ 7.Improving the work environment and work satisfaction§ 8.Increasing transparency of misconduct cases§ 9.Opening up research§ 10.Implementing safe and effective whistle-blowing channels§ 11.Protecting the alleged perpetrators§ 12.Establishing a research integrity committee and appointing an ombudsperson§ 13.Making explicit the applicable standards for research integrity

Towards best practices for authorship and research evaluation: Effects of performance metrics and the Leiden Manifesto

Watch the VIDEO here. New forms of evaluation are reconfiguring science in ways we are only beginning to understand. In this talk, I will address a key challenge in social-scientific research regarding how evaluations are implicated in scientific understandings of the world. I will address how public sector transformations (including a stronger emphasis on national economic goals such as innovation and growth) are introducing highly particular notions of ‘good’ performance and uses of evaluative metrics. Using empirical material and results from recent projects, the talk will reflect on how these trends are affecting knowledge production processes in different fields. I will also suggest ways to think more creatively and responsibly about the affordances of evaluation and indicators in academic settings.