Ciriaco Andrea D'Angelo

A heuristic approach to author name disambiguation in bibliometrics databases for large-scale research assessments

National exercises for the evaluation of research activity by universities are becoming regular practice in ever more countries. These exercises have mainly been conducted through the application of peer-review methods. Bibliometrics has not been able to offer a valid large-scale alternative because of almost overwhelming difficulties in identifying the true author of each publication. We will address this problem by presenting a heuristic approach to author name disambiguation in bibliometric datasets for large-scale research assessments. The application proposed concerns the Italian university system, comprising 80 universities and a research staff of over 60,000 scientists. The key advantage of the proposed approach is the ease of implementation. The algorithms are of practical application and have considerably better scalability and expandability properties than state-of-the-art unsupervised approaches. Moreover, the performance in terms of precision and recall, which can be further improved, seems thoroughly adequate for the typical needs of large-scale bibliometric research assessments.

The relationship between scientists' research performance and the degree of internationalization of their research

Policy makers, at various levels of governance, generally encourage the development of research collaboration. However the underlying determinants of collaboration are not completely clear. In particular, the literature lacks studies that, taking the individual researcher as the unit of analysis, attempt to understand if and to what extent the researcher’s scientific performance might impact on his/her degree of collaboration with foreign colleagues. The current work examines the international collaborations of Italian university researchers for the period 2001–2005, and puts them in relation to each individual’s research performance. The results of the investigation, which assumes co-authorship as proxy of research collaboration, show that both research productivity and average quality of output have positive effects on the degree of international collaboration achieved by a scientist.

Evaluating research: from informed peer review to bibliometrics

National research assessment exercises are becoming regular events in ever more countries. The present work contrasts the peer-review and bibliometrics approaches in the conduct of these exercises. The comparison is conducted in terms of the essential parameters of any measurement system: accuracy, robustness, validity, functionality, time and costs. Empirical evidence shows that for the natural and formal sciences, the bibliometric methodology is by far preferable to peer-review. Setting up national databases of publications by individual authors, derived from Web of Science or Scopus databases, would allow much better, cheaper and more frequent national research assessments.

How do you define and measure research productivity

Productivity is the quintessential indicator of efficiency in any production system. It seems it has become a norm in bibliometrics to define research productivity as the number of publications per researcher, distinguishing it from impact. In this work we operationalize the economic concept of productivity for the specific context of research activity and show the limits of the commonly accepted definition. We propose then a measurable form of research productivity through the indicator “Fractional Scientific Strength (FSS)”, in keeping with the microeconomic theory of production. We present the methodology for measure of FSS at various levels of analysis: individual, field, discipline, department, institution, region and nation. Finally, we compare the ranking lists of Italian universities by the two definitions of research productivity.

Assessment of sectoral aggregation distortion in research productivity measurements

The reliability and correctness of the results of the assessment of research productivity in universities by means of bibliometric techniques depend on the level of data aggregation used for the analysis by disciplinary area. The variability among the university research fields, the varying prolificacy of the scientific disciplines and the different levels of representativeness by discipline of the journals covered in source databases are the main causes leading to distortions. Such effects can be reduced considerably by using a level of data aggregation by discipline which is as homogeneous and uniform as possible. This study compares the research productivity data of Italian universities at two different levels of aggregation of output and input data, thus showing and assessing the scale of the distortions that may result. Copyright , Beech Tree Publishing.

National-scale research performance assessment at the individual level

There is an evident and rapid trend towards the adoption of evaluation exercises for national research systems for purposes, among others, of improving allocative efficiency in public funding of individual institutions. However the desired macroeconomic aims could be compromised if internal redistribution of government resources within each research institution does not follow a consistent logic: the intended effects of national evaluation systems can result only if a “funds for quality” rule is followed at all levels of decision-making. The objective of this study is to propose a bibliometric methodology for: (i) large-scale comparative evaluation of research performance by individual scientists, research groups and departments within research institution, to inform selective funding allocations; and (ii) assessment of strengths and weaknesses by field of research, to inform strategic planning and control. The proposed methodology has been applied to the hard science disciplines of the Italian university research system for the period 2004–2006.

Research productivity: Are higher academic ranks more productive than lower ones?

This work analyses the links between individual research performance and academic rank. A typical bibliometric methodology is used to study the performance of all Italian university researchers active in the hard sciences, for the period 2004–2008. The objective is to characterize the performance of the ranks of full (FPs), associate and assistant professors (APs), along various dimensions, in order to verify the existence of performance differences among the ranks in general and for single disciplines.

National research assessment exercises: a comparison of peer review and bibliometrics rankings

Development of bibliometric techniques has reached such a level as to suggest their integration or total substitution for classic peer review in the national research assessment exercises, as far as the hard sciences are concerned. In this work we compare rankings lists of universities captured by the first Italian evaluation exercise, through peer review, with the results of bibliometric simulations. The comparison shows the great differences between peer review and bibliometric rankings for excellence and productivity.

Career advancement and scientific performance in universities

Many governments have placed priority on excellence in higher education as part of their policy agendas. Processes for recruitment and career advancement in universities thus have a critical role. The efficiency of faculty selection processes can be evaluated by comparing the subsequent performance of competition winners against that of the losers and the pre-existing staff of equal academic rank. Our study presents an empirical analysis concerning the recruitment procedures for associate professors in the Italian university system. The results of a bibliometric analysis of the hard science areas reveal that new associate professors are on average more productive than the incumbents. However a number of crucial concerns emerge, in particular concerning occurrence of non-winner candidates that are more productive than the winners over the subsequent triennium, and cases of winners that are completely unproductive. Beyond the implications for the Italian case, the analysis offers considerations for all decision-makers regarding the ex post evaluation of the efficiency of the recruitment process and the desirability of providing selection committees with bibliometric indicators in support of evaluation (i.e. informed peer review).

An Individual‐Level Assessment of the Relationship between Spin‐Off Activities and Research Performance in Universities

One of the most problematic aspects in the creation of spin‐offs by university personnel concerns the relationship between entrepreneurial activity and research activity by researcher‐entrepreneurs. The literature has expressed varying and opposing views as to the nature of the relationship, but very little has been produced to empirically legitimate one position or another. The present work proposes to address this shortcoming by exploring the relationship existing between academic spin‐off generation and the research performance of enterprise founders. The study investigates whether, and to what extent, scientific performance by academic entrepreneurs is different than that of their colleagues, and if the involvement in entrepreneurial activity has an influence on the individuals research activity. The research questions are answered by considering all spin‐offs generated by Italian universities over the period 2001–2008 and evaluating, through a bibliometric approach, the scientific performance of founders relative to that of their colleagues who carry out research in the same field. The data show better scientific performance by the researcher‐entrepreneurs than that of their colleagues, and in addition, although there are some variations across fields, the creation of a spin‐off does not seem, on average, to have negative effects on the scientific performance of the founders.