Per Ahlgren

Requirements for a cocitation similarity measure, with special reference to Pearson's correlation coefficient

Author cocitation analysis (ACA), a special type of cocitation analysis, was introduced by White and Griffith in 1981. This technique is used to analyze the intellectual structure of a given scientific field. In 1990, McCain published a technical overview that has been largely adopted as a standard. Here, McCain notes that Pearsons correlation coefficient (Pearsons r) is often used as a similarity measure in ACA and presents some advantages of its use. The present article criticizes the use of Pearsons r in ACA and sets forth two natural requirements that a similarity measure applied in ACA should satisfy. It is shown that Pearsons r does not satisfy these requirements. Real and hypothetical data are used in order to obtain counterexamples to both requirements. It is concluded that Pearsons r is probably not an optimal choice of a similarity measure in ACA. Still, further empirical research is needed to show if, and in that case to what extent, the use of similarity measures in ACA that fulfill these requirements would lead to objectively better results In full-scale studies. Further, problems related to incomplete cocitation matrices are discussed.

Document-document similarity approaches and science mapping: experimental comparison of five approaches

This paper treats document–document similarity approaches in the context of science mapping. Five approaches, involving nine methods, are compared experimentally. We compare text-based approaches, the citation-based bibliographic coupling approach, and approaches that combine text-based approaches and bibliographic coupling. Forty-three articles, published in the journal Information Retrieval, are used as test documents. We investigate how well the approaches agree with a ground truth subject classification of the test documents, when the complete linkage method is used, and under two types of similarities, first-order and second-order. The results show that it is possible to achieve a very good approximation of the classification by means of automatic grouping of articles. One text-only method and one combination method, under second-order similarities in both cases, give rise to cluster solutions that to a large extent agree with the classification.

The effects and their stability of field normalization baseline on relative performance with respect to citation impact : a case study of 20 natural science departments

In this paper we study the effects of field normalization baseline on relative performance of 20 natural science departments in terms of citation impact. Impact is studied under three baselines: journal, ISI/Thomson Reuters subject category, and Essential Science Indicators field. For the measurement of citation impact, the indicators item-oriented mean normalized citation rate and Top-5% are employed. The results, which we analyze with respect to stability, show that the choice of normalization baseline matters. We observe that normalization against publishing journal is particular. The rankings of the departments obtained when journal is used as baseline, irrespective of indicator, differ considerably from the rankings obtained when ISI/Thomson Reuters subject category or Essential Science Indicators field is used. Since no substantial differences are observed when the baselines Essential Science Indicators field and ISI/Thomson Reuters subject category are contrasted, one might suggest that people without access to subject category data can perform reasonable normalized citation impact studies by combining normalization against journal with normalization against Essential Science Indicators field.

Bibliographic coupling, common abstract stems and clustering: a comparison of two document-document similarity approaches in the context of science mapping

This paper deals with two document-document similarity approaches in the context of science mapping: bibliographic coupling and a text approach based on the number of common abstract stems. We used 43 articles, published in the journal Information Retrieval, as test articles. An information retrieval expert performed a classification of these articles. We used the cosine measure for normalization, and the complete linkage method was used for clustering the articles. A number of articles pairs were ranked (1) according to descending normalized coupling strength, and (2) according to descending normalized frequency of common abstract stems. The degree of agreement between the two obtained rankings was low, as measured by Kendall’s tau. The agreement between the two cluster solutions, one for each approach, was fairly low, according to the adjusted Rand index. However, there were examples of perfect agreement between the coupling solution and the stems solution. The classification generated by the expert contained larger groups compared to the coupling and stems solutions, and the agreement between the two solutions and the classification was not high. According to the adjusted Rand index, though, the stems solution was a better approximation of the classification than the coupling solution. With respect to cluster quality, the overall Silhouette value was slightly higher for the stems solution. Examples of homogeneous cluster structures, as well as negative Silhouette values, were found with regard to both solutions. The expert classification indicates that the field of information retrieval, as represented by one volume of articles published in Information Retrieval, is fairly heterogeneous regarding research themes, since the classification is associated with 15 themes. The complete linkage method, in combination with the upper tail rule, gave rise to a fairly good approximation of the classification with respect to the number of identified groups, especially in case of the stems approach.

Field normalized citation rates, field normalized journal impact and Norwegian weights for allocation of university research funds

We compared three different bibliometric evaluation approaches: two citation-based approaches and one based on manual classification of publishing channels into quality levels. Publication data for two universities was used, and we worked with two levels of analysis: article and department. For the article level, we investigated the predictive power of field normalized citation rates and field normalized journal impact with respect to journal level. The results for the article level show that evaluation of journals based on citation impact correlate rather well with manual classification of journals into quality levels. However, the prediction from field normalized citation rates to journal level was only marginally better than random guessing. At the department level, we studied three different indicators in the context of research fund allocation within universities and the extent to which the three indicators produce different distributions of research funds. It turned out that the three distributions of relative indicator values were very similar, which in turn yields that the corresponding distributions of hypothetical research funds would be very similar.

Experimental comparison of first and second-order similarities in a scientometric context

The measurement of similarity between objects plays a role in several scientific areas. In this article, we deal with document–document similarity in a scientometric context. We compare experimentally, using a large dataset, first-order with second-order similarities with respect to the overall quality of partitions of the dataset, where the partitions are obtained on the basis of optimizing weighted modularity. The quality of a partition is defined in terms of textual coherence. The results show that the second-order approach consistently outperforms the first-order approach. Each difference between the two approaches in overall partition quality values is significant at the 0.01 level.

The correlation between citation-based and expert-based assessments of publication channels: SNIP and SJR vs. Norwegian quality assessments

We study the correlation between citation-based and expert-based assessments of journals and series, which we collectively refer to as sources. The source normalized impact per paper (SNIP), the Scimago Journal Rank 2 (SJR2) and the raw impact per paper (RIP) indicators are used to assess sources based on their citations, while the Norwegian model is used to obtain expert-based source assessments. We first analyze – within different subject area categories and across such categories – the degree to which RIP, SNIP and SJR2 values correlate with the quality levels in the Norwegian model. We find that sources at higher quality levels on average have substantially higher RIP, SNIP, and SJR2 values. Regarding subject area categories, SNIP seems to perform substantially better than SJR2 from the field normalization point of view. We then compare the ability of RIP, SNIP and SJR2 to predict whether a source is classified at the highest quality level in the Norwegian model or not. SNIP and SJR2 turn out to give more accurate predictions than RIP, which provides evidence that normalizing for differences in citation practices between scientific fields indeed improves the accuracy of citation indicators.

Evaluation of retrieval effectiveness with incomplete relevance data: Theoretical and experimental comparison of three measures

This paper investigates two relatively new measures of retrieval effectiveness in relation to the problem of incomplete relevance data. The measures, Bpref and RankEff, which do not take into account documents that have not been relevance judged, are compared theoretically and experimentally. The experimental comparisons involve a third measure, the well-known mean uninterpolated average precision. The results indicate that RankEff is the most stable of the three measures when the amount of relevance data is reduced, with respect to system ranking and absolute values. In addition, RankEff has the lowest error-rate.

Retrieval evaluation with incomplete relevance data: a comparative study of three measures

Retrieval evaluation with incomplete relevance data : A comparative study of three measures (poster abstract)

Geographical distance in bibliometric relations within epistemic communities

Scientists collaborate increasingly on a global scale. Does this trend also hold for other bibliometric relations such as direct citations, cocitations and shared references? This study examines citation-based relations in publications published in the journal Scientometrics from 1981 to 2010. Different measures of Mean Geographical Distance (MGD) are tested. If we take all citation links into consideration, there is no indication of MGD increase, but when we look at maximum distances of each relation, a weak tendency of increasing MGD could be observed. One major factor behind the lack of growth of mean distances is the form of the distribution of citation links over distances. Our data suggest that the interactions might grow simultaneously for both short and long distances.