Werner Marx

What factors determine citation counts of publications in chemistry besides their quality

A number of bibliometric studies point out that citation counts are a function of many variables besides scientific quality. In this paper our aim is to investigate these factors that usually impact the number of citation counts, using an extensive data set from the field of chemistry. The data set contains roughly 2000 manuscripts that were submitted to the journal Angewandte Chemie International Edition (AC-IE) as short communications, reviewed by external reviewers, and either published in AC-IE or, if not accepted for publication by AC-IE, published elsewhere. As the reviewers’ ratings of the importance of the manuscripts’ results are also available to us, we can examine the extent to which certain factors that previous studies demonstrated to be generally correlated with citation counts increase the impact of papers, controlling for the quality of the manuscripts (as measured by reviewers’ ratings of the importance of the findings) in the statistical analysis. As the results show, besides being associated with quality, citation counts are correlated with the citation performance of the cited references, the language of the publishing journal, the chemical subfield, and the reputation of the authors. In this study no statistically significant correlation was found between citation counts and number of authors.

Citation analysis using online databases: Feasibilities and shortcomings

Extensive citation analysis with the Science Citation Index (SCI) has become possible through expanded search capabilities introduced by STN International a few years ago. STN enhanced its retrieval language with some important features, originally developed for statistical analysis of patents. Most important are an expanded select command and several functions to list the search results. The publications to be evaluated may be selected either in the SCI, or in a number of other bibliographic databases offered by the host. With the help of these features, the basic methods to appropriately measure the impact of scientific activities are demonstrated. Furthermore, possible shortcomings as well as the risks when interpreting the results of such studies are discussed.

Convergent validity of bibliometric Google Scholar data in the field of chemistry—Citation counts for papers that were accepted by Angewandte Chemie International Edition or rejected but published elsewhere, using Google Scholar, Science Citation Index, Scopus, and Chemical Abstracts

Examining a comprehensive set of papers (n=1837) that were accepted for publication by the journal Angewandte Chemie International Edition (one of the prime chemistry journals in the world) or rejected by the journal but then published elsewhere, this study tested the extent to which the use of the freely available database Google Scholar (GS) can be expected to yield valid citation counts in the field of chemistry. Analyses of citations for the set of papers returned by three fee-based databases – Science Citation Index, Scopus, and Chemical Abstracts – were compared to the analysis of citations found using GS data. Whereas the analyses using citations returned by the three fee-based databases show very similar results, the results of the analysis using GS citation data differed greatly from the findings using citations from the fee-based databases. Our study therefore supports, on the one hand, the convergent validity of citation analyses based on data from the fee-based databases and, on the other hand, the lack of convergent validity of the citation analysis based on the GS data.

The history of the stretched exponential function

The original article of Rudolf Kohlrausch (1854) on stretched exponentials and their application to describe relaxation phenomena has been often misquoted in the literature after its rediscovery around 1984. We discuss here this fact and attempt to set the record straight.

How good is research really? Measuring the citation impact of publications with percentiles increases correct assessments and fair comparisons.

Bibliometrics increasingly determine the allocation of jobs and funding in science. Bibliometricians must therefore develop and adopt reliable measures of quality that truly reflect a scientists contribution to his or her field.

How to evaluate individual researchers working in the natural and life sciences meaningfully? A proposal of methods based on percentiles of citations

Although bibliometrics has been a separate research field for many years, there is still no uniformity in the way bibliometric analyses are applied to individual researchers. Therefore, this study aims to set up proposals how to evaluate individual researchers working in the natural and life sciences. 2005 saw the introduction of the h index, which gives information about a researcher’s productivity and the impact of his or her publications in a single number (h is the number of publications with at least h citations); however, it is not possible to cover the multidimensional complexity of research performance and to undertake inter-personal comparisons with this number. This study therefore includes recommendations for a set of indicators to be used for evaluating researchers. Our proposals relate to the selection of data on which an evaluation is based, the analysis of the data and the presentation of the results.

Detecting the historical roots of research fields by reference publication year spectroscopy RPYS

We introduce the quantitative method named “Reference Publication Year Spectroscopy” (RPYS). With this method one can determine the historical roots of research fields and quantify their impact on current research. RPYS is based on the analysis of the frequency with which references are cited in the publications of a specific research field in terms of the publication years of these cited references. The origins show up in the form of more or less pronounced peaks mostly caused by individual publications that are cited particularly frequently. In this study, we use research on graphene and on solar cells to illustrate how RPYS functions, and what results it can deliver.

Methods for the generation of normalized citation impact scores in bibliometrics: Which method best reflects the judgements of experts?

Evaluative bibliometrics compare the citation impact of researchers, research groups and institutions with each other across time scales and disciplines. Both factors, discipline and period – have an influence on the citation count which is independent of the quality of the publication. Normalizing the citation impact of papers for these two factors started in the mid-1980s. Since then, a range of different methods have been presented for producing normalized citation impact scores. The current study uses a data set of over 50,000 records to test which of the methods so far presented correlate better with the assessment of papers by peers. The peer assessments come from F1000Prime – a post-publication peer review system of the biomedical literature. Of the normalized indicators, the current study involves not only cited-side indicators, such as the mean normalized citation score, but also citing-side indicators. As the results show, the correlations of the indicators with the peer assessments all turn out to be very similar. Since F1000 focuses on biomedicine, it is important that the results of this study are validated by other studies based on datasets from other disciplines or (ideally) based on multi-disciplinary datasets.

On the causes of subject-specific citation rates in Web of Science

It is well known in bibliometrics that the average number of citations per paper differs greatly between the various disciplines. The differing citation culture (in particular the different average number of references per paper and thereby the different probability of being cited) is widely seen as the cause of this variation. Based on all Web of Science (WoS) records published in 1990, 1995, 2000, 2005, and 2010 we demonstrate that almost all disciplines show similar numbers of references in the appendices of their papers. Our results suggest that the average citation rate is far more influenced by the extent to which the papers (cited as references) are included in WoS as linked database records. For example, the comparatively low citation rates in the humanities are not at all the result of a lower average number of references per paper but are caused by the low fraction of linked references which refer to papers published in the core journals covered by WoS.

Referenced Publication Years Spectroscopy applied to iMetrics: Scientometrics, Journal of Informetrics, and a relevant subset of JASIST

We have developed a (freeware) routine for “Referenced Publication Years Spectroscopy” (RPYS) and apply this method to the historiography of “iMetrics,” that is, the junction of the journals Scientometrics, Informetrics, and the relevant subset of JASIST (approx. 20%) that shapes the intellectual space for the development of information metrics (bibliometrics, scientometrics, informetrics, and webometrics). The application to information metrics (our own field of research) provides us with the opportunity to validate this methodology, and to add a reflection about using citations for the historical reconstruction. The results show that the field is rooted in individual contributions of the 1920s to 1950s (e.g., Alfred J. Lotka), and was then shaped intellectually in the early 1960s by a confluence of the history of science (Derek de Solla Price), documentation (e.g., Michael M. Kesslers “bibliographic coupling”), and “citation indexing” (Eugene Garfield). Institutional development at the interfaces between science studies and information science has been reinforced by the new journal Informetrics since 2007. In a concluding reflection, we return to the question of how the historiography of science using algorithmic means—in terms of citation practices—can be different from an intellectual history of the field based, for example, on reading source materials.