Gunnar Sivertsen

The measurement of international scientific collaboration

A growing science policy interest in international scientific collaboration has brought about a multitude of studies which attempt to measure the extent of international scientific collaboration between countries and to explore intercountry collaborative networks. This paper attempts to clarify the methodology that is being used or can be used for this purpose and discusses the adequacy of the methods. The paper concludes that, in an analysis of collaborative links, it is essential to use both absolute and relative measures. The latter normalize differences in country size. Each yields a different type of information. Absolute measures yield an answer to questions such as which countries are central in the international network of science, whether collaborative links reveal a centre — periphery relationship, and which countries are the most important collaborative partners of another country. Relative measures provide answers to questions of the intensity of collaborative links.

The effect of highly cited papers on national citation indicators

Citation distributions are extremely skewed. This paper addresses the following question: To what extent are national citation indicators influenced by a small minority of highly cited articles? This question has not been studied before at the level of national indicators. Using the scientific production of Norway as a case, we find that the average citation rates in major subfields are highly determined by one or only a few highly cited papers. Furthermore, there are large annual variations in the influence of highly cited papers on the average citation rate of the subfields. We conclude that an analysis of the underlying data for national indicators may be useful in creating awareness towards the occurrence of particular articles with great influence on what is normally considered an indicator of “national performance”, and that the common interpretation of the indicator on research policy level needs to be informed by this fact.

Are female researchers less cited? A large-scale study of Norwegian scientists

Numerous studies have shown that female scientists tend to publish significantly fewer publications than do their male colleagues. In this study, we have analyzed whether similar differences also can be found in terms of citation rates. Based on a large-scale study of 8,500 Norwegian researchers and more than 37,000 publications covering all areas of knowledge, we conclude that the publications of female researchers are less cited than are those of men, although the differences are not large. The gender differences in citation rates can be attributed to differences in productivity. There is a cumulative advantage effect of increasing publication output on citation rates. Since the women in our study publish significantly fewer publications than do men, they benefit less from this effect. The study also provides results on how publication and citation rates vary according to scientific position, age, and discipline.

Comprehensive bibliographic coverage of the social sciences and humanities in a citation index: an empirical analysis of the potential

A well-designed and comprehensive citation index for the social sciences and humanities has many potential uses, but has yet to be realised. Significant parts of the scholarly production in these areas are not published in international journals, but in national scholarly journals, in book chapters or in monographs. The potential for covering these literatures more comprehensively can now be investigated empirically using a complete publication output data set from the higher education sector of an entire country (Norway). We find that while the international journals in the social sciences and humanities are rather small and more dispersed in specialties, representing a large but not unlimited number of outlets, the domestic journal publishing, as well as book publishing on both the international and domestic levels, show a concentration of many publications in few publication channels. These findings are promising for a more comprehensive coverage of the social sciences and humanities.

Taking scholarly books into account: current developments in five European countries

For academic book authors and the institutions assessing their research performance, the relevance of books is undisputed. In spite of this, the absence of comprehensive international databases covering the items and information needed for the assessment of this type of publication has urged several European countries to develop custom-built information systems for the registration of scholarly books, as well as weighting and funding allocation procedures. For the first time, these systems make the assessment of books as a research output feasible. The present paper summarizes the main features of the registration and/or assessment systems developed in five European countries/regions (Spain, Denmark, Flanders, Finland and Norway), focusing on the processes involved in the collection and processing of data on book publications, their weighting, as well as the application in the context of research assessment and funding.

The effect of booming countries on changes in the relative specialization index (RSI) on country level

The Relative Specialization Index (RSI) is an indicator that measures the research profile of a country by comparing the share of a given field in the publications of a given country with the share of the same field in the world total of publications. If measured over time, this indicator may be influenced in the world total by the increased representation of certain other countries with different research profiles. As a case, we study the effect on the RSI for The Netherlands of the increased representation of China in the ISI Web of Science. Although the booming of China is visible in the RSI for The Netherlands, especially in the last decade and in fields where the countries have opposite specializations, the basic research profile as measured by the RSI remains the same. We conclude that the indicator is robust with regard to booming countries, and that it may suffice to observe the general changes in the research profile of the database if the RSI for a country is studied over time.

Publication-Based Funding: The Norwegian Model

The ‘Norwegian Model’ attempts to comprehensively cover all the peer-reviewed scholarly literatures in all areas of research—including the preferred formats and languages of scholarly publishing in the humanities—in one single weighted indicator which makes the research efforts comparable across departments and faculties within and between research institutions. This article describes the main components of the model and how it has been implemented, as well as the effects and experiences in three of the countries that are making use of the model, and where it has been evaluated: Belgium (Flanders), Denmark and Norway. The article concludes with a discussion of the model from the perspective of the humanities.

Patterns of internationalization and criteria for research assessment in the social sciences and humanities

AbstractThis article investigates the developments during the last decades in the use of languages, publication types and publication channels in the social sciences and humanities (SSH). The purpose is to develop an understanding of the processes of internationalization and to apply this understanding in a critical examination of two often used general criteria in research evaluations in the SSH. One of them is that the coverage of a publication in Scopus or Web of Science is seen in itself as an expression of research quality and of internationalization. The other is that a specific international language, English, and a specific type of publication, journal articles, are perceived as supreme in a general hierarchy of languages and publication types. Simple distinctions based on these criteria are contrary to the heterogeneous publication patterns needed in the SSH to organize their research adequately, present their results properly, reach their audiences efficiently, and thereby fulfil their missions. Research quality, internationalization, and societal relevance can be promoted in research assessment in the SSH without categorical hierarchies of publications. I will demonstrate this by using data from scholarly publishing in the SSH that go beyond the coverage in the commercial data sources in order to give a more comprehensive representation of scholarly publishing in the SSH.

Science deserves to be judged by its contents, not by its wrapping: Revisiting Seglen's work on journal impact and research evaluation

The scientific foundation for the criticism on the use of the Journal Impact Factor (JIF) in evaluations of individual researchers and their publications was laid between 1989 and 1997 in a series of articles by Per O. Seglen. His basic work has since influenced initiatives such as the San Francisco Declaration on Research Assessment (DORA), the Leiden Manifesto for research metrics, and The Metric Tide review on the role of metrics in research assessment and management. Seglen studied the publications of only 16 senior biomedical scientists. We investigate whether Seglen’s main findings still hold when using the same methods for a much larger group of Norwegian biomedical scientists with more than 18,000 publications. Our results support and add new insights to Seglen’s basic work.

How can differences in international university rankings be explained

University rankings are typically presenting their results as league tables with more emphasis on final scores and positions, than on the clarification of why the universities are ranked as they are. Finding out the latter is often not possible, because final scores are based on weighted indicators where raw data and the processing of these are not publically available. In this study we use a sample of Scandinavian universities, explaining what is causing differences between them in the two most influential university rankings: Times Higher Education and the Shanghai-ranking. The results show that differences may be attributed to both small variations on what we believe are not important indicators, as well as substantial variations on what we believe are important indicators. The overall aim of this paper is to provide a methodology that can be used in understanding universities’ different ranks in global university rankings.