Ciriaco Andrea D’Angelo

Research collaboration and productivity: is there correlation?

The incidence of extramural collaboration in academic research activities is increasing as a result of various factors. These factors include policy measures aimed at fostering partnership and networking among the various components of the research system, policies which are in turn justified by the idea that knowledge sharing could increase the effectiveness of the system. Over the last two decades, the scientific community has also stepped up activities to assess the actual impact of collaboration intensity on the performance of research systems. This study draws on a number of empirical analyses, with the intention of measuring the effects of extramural collaboration on research performance and, indirectly, verifying the legitimacy of policies that support this type of collaboration. The analysis focuses on the Italian academic research system. The aim of the work is to assess the level of correlation, at institutional level, between scientific productivity and collaboration intensity as a whole, both internationally and with private organizations. This will be carried out using a bibliometric type of approach, which equates collaboration with the co-authorship of scientific publications.

University–industry collaboration in Italy: a bibliometric examination

Abstract This work investigates public–private research collaboration between Italian universities and domestic industry, applying a bibliometric type of approach. The study is based on an exhaustive listing of all co-authored publications in international journals that are jointly realized by Italian university scientists and researchers in the private sector; this listing permits the development of a national mapping system for public–private collaboration that is unique for its extensive and representative character. It is shown that, in absolute terms, most collaborations occur in medicine and chemistry, while it is industrial and information engineering that shows the highest percentage of co-authored articles out of all articles in the field. In addition, the investigation empirically examines and tests several hypotheses concerning the qualitative–quantitative impact of collaboration on the scientific production of individual university researchers. The analyses demonstrate that university researchers who collaborate with those in the private sector show research performance that is superior to that of colleagues who are not involved in such collaboration. But the impact factor of journals publishing academic articles co-authored by industry is generally lower than that concerning co-authorships with other entities. Finally, a further specific elaboration also reveals that publications with public–private co-authorship do not show a level of multidisciplinarity that is significantly different from that of other publications.

Gender differences in research productivity: a bibliometric analysis of the Italian academic system

The literature dedicated to the analysis of the difference in research productivity between the sexes tends to agree in indicating better performance for men. Through bibliometric examination of the entire population of research personnel working in the scientific-technological disciplines of Italian university system, this study confirms the presence of significant differences in productivity between men and women. The differences are, however, smaller than reported in a large part of the literature, confirming an ongoing tendency towards decline, and are also seen as more noticeable for quantitative performance indicators than other indicators. The gap between the sexes shows significant sectorial differences. In spite of the generally better performance of men, there are scientific sectors in which the performance of women does not prove to be inferior.

The measurement of Italian universities’ research productivity by a non parametric-bibliometric methodology

This paper presents a methodology for measuring the technical efficiency of research activities. It is based on the application of data envelopment analysis to bibliometric data on the Italian university system. For that purpose, different input values (research personnel by level and extra funding) and output values (quantity, quality and level of contribution to actual scientific publications) are considered. Our study aims at overcoming some of the limitations connected to the methodologies that have so far been proposed in the literature, in particular by surveying the scientific production of universities by authors’ name.

Revisiting the scaling of citations for research assessment

Over the past decade, national research evaluation exercises, traditionally conducted using the peer review method, have begun opening to bibliometric indicators. The citations received by a publication are assumed as proxy for its quality, but they require standardization prior to use in comparative evaluation of organizations or individual scientists: the citation data must be standardized, due to the varying citation behavior across research fields. The objective of this paper is to compare the effectiveness of the different methods of normalizing citations, in order to provide useful indications to research assessment practitioners. Simulating a typical national research assessment exercise, he analysis is conducted for all subject categories in the hard sciences and is based on the Thomson Reuters Science Citation Index-Expanded®. Comparisons show that the citations average is the most effective scaling parameter, when the average is based only on the publications actually cited.

Gender differences in research collaboration

The debate on the role of women in the academic world has focused on various phenomena that could be at the root of the gender gap seen in many nations. However, in spite of the ever more collaborative character of scientific research, the issue of gender aspects in research collaborations has been treated in a marginal manner. In this article we apply an innovative bibliometric approach based on the propensity for collaboration by individual academics, which permits measurement of gender differences in the propensity to collaborate by fields, disciplines and forms of collaboration: intramural, extramural domestic and international. The analysis of the scientific production of Italian academics shows that women researchers register a greater capacity to collaborate in all the forms analyzed, with the exception of international collaboration, where there is still a gap in comparison to male colleagues.

The importance of accounting for the number of co-authors and their order when assessing research performance at the individual level in the life sciences

Accurate measurement of research productivity should take account of both the number of co-authors of every scientific work and of the different contributions of the individuals. For researchers in the life sciences, common practice is to indicate such contributions through position in the authors list. In this work, we measure the distortion introduced to bibliometric ranking lists for scientific productivity when the number of co-authors or their position in the list is ignored. The field of observation consists of all Italian university professors working in the life sciences, with scientific production examined over the period 2004–2008. The outcomes of the study lead to a recommendation against using indicators or evaluation methods that ignore the different authors’ contributions to the research results.

Assessing the varying level of impact measurement accuracy as a function of the citation window length

With the passage of more time from the original date of publication, the measure of the impact of scientific works using subsequent citation counts becomes more accurate. However the measurement of individual and organizational research productivity should ideally refer to a period with closing date just prior to the evaluation exercise. Therefore it is necessary to compromise between accuracy and timeliness. This work attempts to provide an order of magnitude for the error in measurement that occurs with decreasing the time lapse between date of publication and citation count. The analysis is conducted by scientific discipline on the basis of publications indexed in the Thomson Reuters Italian National Citation Report.

Citations versus journal impact factor as proxy of quality: could the latter ever be preferable?

In recent years bibliometricians have paid increasing attention to research evaluation methodological problems, among these being the choice of the most appropriate indicators for evaluating quality of scientific publications, and thus for evaluating the work of single scientists, research groups and entire organizations. Much literature has been devoted to analyzing the robustness of various indicators, and many works warn against the risks of using easily available and relatively simple proxies, such as journal impact factor. The present work continues this line of research, examining whether it is valid that the use of the impact factor should always be avoided in favour of citations, or whether the use of impact factor could be acceptable, even preferable, in certain circumstances. The evaluation was conducted by observing all scientific publications in the hard sciences by Italian universities, for the period 2004–2007. Performance sensitivity analyses were conducted with changing indicators of quality and years of observation.

A field-standardized application of DEA to national-scale research assessment of universities

The current work proposes an application of DEA methodology for measurement of technical and allocative efficiency of university research activity. The analysis is based on bibliometric data from the Italian university system for the five-year period 2004–2008. Technical and allocative efficiency is measured with input being considered as a universitys research staff, classified according to academic rank, and with output considered as the field-standardized impact of the research product realized by these staff. The analysis is applied to all scientific disciplines of the so-called hard sciences, and conducted at subfield level, thus at a greater level of detail than ever before achieved in national-scale research assessments.