Eugene Garfield

Is citation analysis a legitimate evaluation tool

A comprehensive discussion on the use of citation analysis to rate scientific performance and the controversy surrounding it. The general adverse criticism that citation counts include an excessive number of negative citations (citations to incorrect results worthy of attack), self-citations (citations to the works of the citing authors), and citations to methodological papers is analyzed. Included are a discussion of measurement problems such as counting citations for multiauthored papers, distinguishing between more than one person with the same last name (homographs), and what it is that citation analysis actually measures. It is concluded that as the scientific enterprise becomes larger and more complex, and its role in society more critical, it will become more difficult, expensive and necessary to evaluate and identify the largest contributors. When properly used, citation analysis can introduce a useful measure of objectivity into the evaluation process at relatively low financial cost.

Algorithmic procedure for finding semantically related journals

Using citations, papers and references as parameters a relatedness factor (RF) is computed for a series of journals. Sorting these journals by the RF produces a list of journals most closely related to a specified starting journal. The method appears to select a set of journals that are semantically most similar to the target journal. The algorithmic procedure is illustrated for the journal Genetics. Inter-journal citation data needed to calculate the RF were obtained from the 1996 ISI Journal Citation Reports on CD-ROM©. Out of the thousands of candidate journals in JCR©, 30 have been selected. Some of them are different from the journals in the JCR category for genetics and heredity. The new procedure is unique in that it takes varying journal sizes into account.

Historiographic Mapping of Knowledge Domains Literature

To better understand the topic of this colloquium, we have created a series of databases related to knowledge domains (dynamic systems [small world/Milgram], information visualization [Tufte], co-citation [Small], bibliographic coupling [Kessler], and scientometrics [Scientometrics]). I have used a software package called HistCiteTM which generates chronological maps of subject (topical) collections resulting from searches of the ISI Web of Science1 or ISI citation indexes (SCI, SSCI, and/or AHCI) on CD-ROM. When a marked list is created on WoS, an export file is created which contains all cited references for each source document captured. These bibliographic collections, saved as ASCII files, are processed by HistCite in order to generate chronological and other tables as well as historiographs which highlight the most-cited works in and outside the collection. HistCite also includes a module for detecting and editing errors or variations in cited references as well as a vocabulary analyzer which generates both ranked word lists and word pairs used in the collection. Ideally the system will be used to help the searcher quickly identify the most significant work on a topic and trace its year-by-year historical development. In addition to the collections mentioned above, historiographs based on collections of papers that cite the Watson-Crick 1953 classic paper identifying the helical structure of DNA were created. Both year-by-year as well as month-by-month displays of papers from 1953 to 1958 were necessary to highlight the publication activity of those years.

Why do we need algorithmic historiography

This article discusses the rationale for creating historiographs of scholarly topics using a new program called HistCiteTM, which produces a variety of analyses to aid the historian identify key events (papers), people (authors), and journals in a field. By creating a genealogic profile of the evolution, the program aids the scholar in evaluating the paradigm involved.

From The Science of Science to Scientometrics Visualizing the History of Science with HistCite Software

While ISSI was founded in 1993, Scientometrics and Bibliometrics are now at least half a century old. Indeed, the field can be traced to early quantitative studies in the early 20th century. In the 1930s, it evolved to the “science of science.” The publication of J.D. Bernals Social Function of Science in 1939 was a key transition point but the field lay dormant until after World War II, when D.J.D. Prices books Science Since Babylon and Little Science, Big Science were published in 1961 and 1963. His role as the “Father of Scientometrics” is clearly evident by using the HistCite software to visualize his impact as well as the subsequent impact of the journal Scientometrics on the growth of the field. Scientometrics owes its name to V.V. Nalimov, the author of Naukometriya, and to Tibor Braun who adapted the neologism for the journal. The primordial paper on citation indexing by Garfield which appeared in Science 1955 became a bridge between Bernal and Price. The timeline for the evolution of Scientometrics is demonstrated by a HistCite tabulation of the ranked citation index of the 100,000 references cited in the 3000 papers citing Price.

The evolution of the Science Citation Index

The Science Citation Index was proposed over 50 years ago to facilitate the dissemination and retrieval of scien- tific literature. Its unique search engine, based on citation searching, was not widely adopted until it was made available online in 1972. Its by product, Journal Citation Reports, be- came available in 1975 and included its rankings by impact factor. Impact factors were not widely implemented until about a decade ago, when they began to be used as surrogates for expected citation frequencies for recently published papers—a highly controversial application of scientometrics in evaluating scientists and institutions. Here, the inventor of both the SCI and its companion, Social Sciences Citation Index, review the history of these instruments and discusses their more recent use in graphically visualizing microhistories of scholarly topics. In an example thereof, the patented HistCite software for algo- rithmic historiographic analysis is used to follow the genealogy of the Watson-Crick discovery of the double-helix structure of DNA and its relationship to the work of Heidelberger, Avery, and others.

Rank-normalized impact factor: a way to compare journal performance across subject categories

It is well known that uninformed science administrators often use ISIs journal impact factors without taking into account the inherent citation characteristics of individual scientific disciplines. A rank normalized impact factor (rnlF) is proposed which involves use of order statistics for the complete set of journals within each JCR category. We believe the normalization procedure provides reliable and easily interpretable values. For any journal j, its rnlF is designated as rnlF1and equals (K–R1+ 1)/K, where R1 is the descending rank of journal j in its JCR category and K is the number of journals in the category. Note: JCR impact factor listings are published in descending order. The proposed rnlF is compared with normalized impact factors proposed by earlier authors. The efficacy of the rnlF is illustrated in the cases of seven highly-cited scientists, one each from seven different fields.

Of nobel class: A citation perspective on high impact research authors

The purpose of this paper was to determine if quantitative rankings of highly cited research authors confirm Nobel prize awards. Six studies covering different time periods and author sample sizes were reviewed. The number of Nobel laureates at the time each study was published was tabulated, as was the number of high impact authors who later became laureates. The Nobelists and laureates-to-be were also compared with non-Nobelists to see if they differed in terms of impact and productivity. The results indicate that high rankings by citation frequency identify researchersof Nobel class — that is, a small set of authors that includes a high proportion of actual Nobelists and laureates-to-be. Also, the average impact (citations per author) of Nobelists and laureates-to-be is sufficiently high to distinguish them from non-Nobelists in these rankings. In conclusion, a simple, quantitative, and objective algorithm based on citation data can effectively corroborate —and even forecast — a complex, qualitative, and subjective selection process based on human judgement.

From Citation Indexes to Informetrics: Is the Tail Now Wagging the Dog ?

This article provides a synoptic review and history of citation indexes and their evolution into research evaluation tools including a discussion of the use of bibliometric data for evaluating U.S. institutions (academic departments) by the National Research Council (NRC). The review covers the origin and uses of journal impact factors, validation studies of citation analysis, information retrieval and dissemination (current awareness), citation consciousness, historiography and science mapping, Citation Classics,® and the history of contemporary science. Retrieval of information by cited reference searching is illustrated, especially as it applies to avoiding duplicated research. The fifteen-year cumulative impacts of journals and the percentage of uncitedness, the emergence of scientometrics, old boy networks, and citation frequency distributions are discussed. The paper concludes with observations about the future of citation indexing

Language Use in International Research: A Citation Analysis

The fact that English is the internationally accepted language of research communication raises the issue of a language barrier in two senses. First, those whose native language is not English risk being unaware of—and overlooked by—mainstream international research unless they learn to read, write, and publish in English. Second, native English-speaking researchers risk being ignorant of significant findings reported in foreign languages, especially the Japanese and Russian literature, unless they become proficient in at least one other language. The Institute for Scientific Information (ISI) data base is used to answer three basic questions bearing on this issue: (1) who writes in what languages; (2) who cites what languages; and (3) who cites what nations.