Delayed Recognition; the Co-citation Perspective
DDelayed Recognition; the Co-citation Perspective
Wenxi Zhao Dmitriy Korobskiy George ChackoNetelabs, NET ESolutions Corporation (an NTT DATA Company)McLean, VA 22102July 1, 2020
Abstract
A Sleeping Beauty is a publication that is apparently unrecognized for some periodof time before experiencing sudden recognition by citation. Various reasons, includingresistance to new ideas, have been attributed to such delayed recognition. We examinethis phenomenon in the special case of co-citations, which represent new ideas gen-erated through the combination of existing ones. Using relatively stringent selectioncriteria derived from the work of others, we analyze a very large dataset of over 940 mil-lion unique co-cited article pairs, and identified 1,196 cases of delayed co-citations. Wefurther classify these 1,196 cases with respect to amplitude, rate of citation, and disci-plinary origin and discuss alternative approaches towards identifying such instances.
The term ‘Sleeping Beauty’ has been used to describe an article that is not well cited inthe early years after its publication but experiences a sharp increase in the rate at which itis subsequently cited (van Raan, 2004). An implication is that the new concept presentedin such an article is ‘ahead of its time’ and resistance to it delays recognition. Other causesfor resistance and delayed recognition have been postulated that include (i) informationoverload from the large amount of information available, (ii) modest communication skillsof authors, (iii) insufficient promotion of ideas, (iv) conflict with existing theory and ex-perimental data, (v) the author’s position in the social hierarchy of science, (vi) multiplediscovery, (vii) the management structures of scientific institutions, (viii), and the conser-vative nature of establishments (Barber, 1961; Merton, 1963; Cole, 1970; Garfield, 1970,1980). The Sleeping Beauty phenomenon, and variants of it, have been studied and debatedwith some degree of agreement that a fraction of the scientific literature exhibits citationkinetics that suggest delayed but eventual recognition of new ideas (Glnzel et al., 2003;Glnzel & Garfield, 2004; van Raan, 2004; Redner, 2005; Braun et al., 2010; Li, 2014; Keet al., 2015; Li & Ye, 2016; Song et al., 2018; Sugimoto & Mostafa, 2018; Ye & Bornmann,2018; van Raan & Winnink, 2019). 1 a r X i v : . [ c s . D L ] J un arious approaches have been used to identify Sleeping Beauties and variants of it.Depth of sleep, length of sleep, and awake intensity as variables (van Raan, 2004), theGini coefficient to examine later years of citation history (Li et al., 2014), a parameter-freebeauty coefficient (Ke et al., 2015), positional measures (Costas et al., 2010), and thecitation angle by Ye & Bornmann (2018). While earlier studies examined small datasets,subsequent ones considered large samples of the literature, for example, 22 million publi-cations in Ke et al. (2015).The research cited above has focused on single publications, however, new ideas alsoresult from combining two previously independent ones. The recognition of such noveltythrough combination can be examined by co-citation analysis (Marshakova-Shaikevich,1973; Small, 1973; Uzzi et al., 2013; Boyack & Klavans, 2014; Wang et al., 2017; Bradleyet al., 2020). Tracing co-citations, therefore, provides another lens with which to studydelayed recognition. In precedent is a somewhat related use of co-citation analysis byZong et al. (2018); Teixeira et al. (2017) who sought to identify the so-called ‘princes’ thatawaken Sleeping Beauties.The measurement of delayed recognition by co-citation has been briefly explored byDevarakonda et al. (2020) in a study of 33.6 million reference pairs. The authors usedsimplified criteria derived from prior Sleeping Beauty studies on single publications (Keet al., 2015; van Raan, 2004; van Raan & Winnink, 2019), reported 24 co-cited pairs all inthe 99th percentile of co-citation frequencies, and proposed the term delayed co-citations for such cases. This initial exploration, albeit at scale, only considered reference pairs whereeach member of a pair was in the 99th percentile of highly cited articles in Scopus. In thisarticle, we extend the work delayed co-citation to a much larger dataset, approximately 940million pairs of articles. We refine the criteria in Devarakonda et al. (2020) and identifyco-cited article pairs that exhibit delayed recognition using modifications of the techniquesof van Raan (2004); van Raan & Winnink (2019) and Ke et al. (2015). We also ask whetherdelayed co-citations are derived from Sleeping Beauty publications. We have previously described a dataset of 33.6 million cited pairs each belonging to thetop 1% of cited articles in the Scopus bibliography (Devarakonda et al., 2020, Figure 2). Inthe present study, we include all co-cited pairs from references cited by articles publishedin Scopus in the 11 year period, 1985-1995, not only those drawn from the top 1% ofcited articles. We developed methods to manage the expected volume of data using acombination of SQL, Cypher, and Python. Our code for parsing and updating ScopusXML data, a PostgreSQL schema for Scopus data, SQL, Cypher, and Python scripts usedin this study are freely available from a Github repository (Korobskiy et al., 2019).To assemble and analyze a working dataset, we first exported 95,524,693 publicationrecords from Scopus (all citation types) as a citation graph consisting of an edgelist and a2odelist, imported these data into a graph database (Neo4j) treating publications as nodesand citations as edges. After creating indexes to improve performance, we selected allpublications of citation type ‘article’ published in the years 1985-1995 (inclusive of both)that had at least five cited references each. In counting references, we only consideredreferences with complete Scopus records. Incomplete references and those with crypticplaceholder identifiers were removed from the dataset. We also filtered rare cases in thedata where a publication cites itself, or if the publication date of a cited reference wasmissing or greater than the publication date of its citing article. Selection of publicationswith at least 5 references was performed after curating references.After initial comparison of SQL vs Cypher, we chose, on the basis of simplicity andperformance, to use Cypher queries in Neo4j to generate all pairwise (cid:0) n (cid:1) combinations ofan article’s cited references. We de-duplicated these pairs across all articles to assemble adataset of ∼
940 million pairs (940,357,633 pairs), roughly 28 times larger than the datasetin Devarakonda et al. (2020). We then calculated the frequency of co-cited pairs by dividingthe data and processing batches in parallel using Neo4j and the GNU Parallel utility. Aftertuning experiments on a test set of 1 million pairs using a Neo4j 4.0 in a Centos 7.5 virtualmachine with 128 Gb of RAM and 16 vCPUs in the Microsoft Azure environment, we setthe batch size to 1,000 pairs and the degree of parallelization to 15 cores. Under theseconditions, it took roughly 11 min to compute co-citation frequencies for a batch of 1,000pairs. We divided these 940 million pairs into 9 subsets of around 100 million pairs eachand processed them at the rate of approximately 19 hours per subset.In illustration, the simple Cypher query for calculating co-citation frequencies of pairsin Neo4j is shown below. The input to the query is a csv file containing two columns ofarticle identifiers with each row representing a co-cited pair.
UNWIND $ i n p u t d a t a AS rowMATCH ( a : P u b l i c a t i o n { n o d e i d : row . c i t e d 1 } ) < −− (p) −− > (b : P u b l i c a t i o n { n o d e i d : row . c i t e d 2 } )RETURN row . c i t e d 1 AS c i t e d 1 , row . c i t e d 2 AS c i t e d 2 ,count ( p ) AS s c o p u s f r e q u e n c y ; Frequencies thus calculated, were loaded back into PostgreSQL. For kinetic analysis,we selected all pairs with a co-citation frequency > = 100 and calculated the kinetics ofcitation accumulation from the first possible year of co-citation for each pair through theyear 2018, again in Neo4j. Finally, for continuity, we set zero as the frequency for all yearsbetween the first possible year of co-citation and the last co-cited year (2018), with missingfrequency counts. Minor differences between the data in Devarakonda et al. (2020) are dueto more current data in Scopus in our study, and computing kinetic data through 2018in this study. We compared small samples between the two datasets and confirmed thatthese minor differences in co-citation frequencies could be bridged by including citationsfrom publications in 2019 and later.After generating a dataset of 940 million pairs, we applied three relatively conservativeconditions to identify co-cited pairs of interest: (i) a minimum peak (annual) co-citation3requency for a pair of at least 20 (ii) a minimum total co-citation frequency of at least100 (iii) a requirement both members of a co-cited pair should be published no earlierthan 1970. We then identified delayed co-citation cases by setting two more conditions:(i) a minimum sleeping duration of 10 years as measured from the first possible year ofco-citation (the more recent publication year of the two articles), (ii) during this sleepingperiod of 10 years or more, the average co-citation frequency should be at most 1 with nomore than 2 co-citations in any one year.We also calculated the slope between the co-citation frequency of the awakening yearand the peak frequency and modified the Beauty Coefficient (Ke et al., 2015; Devarakondaet al., 2020), which was designed to measure kinetics in single publications, to be relevantto co-citations by treating the first possible year of co-citation equivalently to the year ofpublication for a single article (Devarakonda et al., 2020).To identify, single Sleeping Beauty publications, we narrowed the criteria of van Raan& Winnink (2019) to consider only a single sleeping period of 10 years or greater; depthof sleep (average citation rate during sleep) of at most 1; an awakening period of 5 years;and an average co-citation frequency during the awakening period (which is defined asawakening citation intensity by van Raan) of at least 5. We also calculated the BeautyCoefficient (Ke et al., 2015) for all single publications for comparison. In this study of delayed co-citations, we first examined cited references from 3,433,578publications in the Scopus database. The criteria for selection of these publications werethat they were classified as ‘article’, that they were published in the period 1985-1995,and they contained at least 5 cited references each. We generated all possible co-citedpairs for the references in these articles and de-duplicated them across articles. since thesame reference pair can occur in more than one article. Then we measured the co-citationfrequency of each pair across the entire Scopus database by counting all co-citation eventsfrom the first possible year of co-citation onwards, Fig 1,Table 1).The data in Fig 1 show a highly skewed distribution of co-citation frequencies across alarge dataset. Roughly 84% of the pairs have a total co-citation frequency of 2 or less, andthe 99th percentile is 16 although each pair had at least 10 years to accumulate co-citations.Even for a pair of articles from the most recent year in our data, 1995, this frequency of 16corresponds to less than one co-citation per year on average. Thus, only a small fraction ofpairs in these data have co-citation frequencies greater than 2 per year. One might considerthat the reasons advanced for delayed recognition described in the Introduction could alsocontribute to such modest recognition or even acknowledgment of non-merit.Beyond a high level understanding of the distribution of co-citation frequencies, how-ever, we are interested in frequently co-cited publications, which are derived from highlycited publications (Small, 1973), and are of interest to the community. Thus, we subset4 .000.250.500.751.00 8 128 2048 32768
Co−citation Frequency F ( C o − c i t a t i on F r equen cy ) Figure 1: Frequencies of ∼
940 million co-cited pairs drawn from Scopus 1985-1995. Pair-wise combinations, (cid:0) n (cid:1) , of references from articles indexed in Scopus (1985-1995), weregenerated as described in Materials and Methods. Total co-citation frequencies for thesepairs, ranged from 1 to 52,471 with a median frequency of 1. The empirical cumulativedistribution function (ECDF) was calculated from 940,357,633 co-citation frequencies andplotted against co-citation frequencies on a log scale.the data using a conservative threshold of 100 for total co-citation frequency along witha peak annual co-citation frequency of at least 20. These criteria are analogous to thoseproposed by van Raan (van Raan, 2004) and Redner (Redner, 2005). After applying thesetwo further restrictions, the number of co-cited pairs is reduced to 51,613 (approximately0.055% of the total number of pairs).To find cases of delayed co-citation, we applied the following conditions to these 51,613pairs: (i) a co-cited paper should have slept for at least 10 years and received no more than2 co-citations in each year during this sleeping period, which is defined as as the number ofyears from the first possible co-cited year to the first year that the pair receives more than2 co-citations. To be considered as a Sleeping Beauty, the awakening period that follows5able 1: Distribution of 940 million Co-citation Frequencies. The count of co-cited pairsin each frequency class as well as the percentage relative to the total number of 940,357,633is shown. Counts include the lower bound in each class and exclude the upper bound. Addlegend details f Interval Count Percentage < = 2 790,189,114 84.032 -4 82,022,893 8.724 -8 41,772,728 4.448 -16 17,749,436 1.8916-32 6,429,234 0.6832-64 1,704,908 0.1864-128 385,923 0.041128-256 81,164 0.0086256-512 17,150 0.0018512-1024 3,777 0.000401024-2048 948 0.00010 > > =10 (Devarakonda et al., 2020).A logical question is whether any of these 1,267 individual publications would be clas-sified as Sleeping Beauties. Applying van Raan’s criteria (Materials and Methods), weidentify 128 of these 1,267 publications as Sleeping Beauties. Interestingly, 27 of the 1,196delayed co-citation pairs were cases where both members were Sleeping Beauties. Of these,the 1978 article by Rassias titled ‘On the stability of the linear mapping in Banach spaces’was a member of four different pairs. Thus, delayed recognition can occur without a re-quirement that at least one member of a co-cited pair with delayed recognition shouldhave Sleeping Beauty characteristics. These observations also suggest that while high-referencing fields such as biology (Small & Greenlee, 1980) might be advantaged by ourselection criteria, the thresholds we set do not entirely exclude other fields. Accordingly,continuing this work with field normalization of co-citation frequencies, to the extent pos-sible, is warranted. 6able 2: Summary Statistics of 1,196 Delayed Co-citation Pairs. Criteria for selection werea minimum sleeping period of 10 years and a minimum peak of 20 citations in any year.Total Frequency Sleep Duration Slope Beauty Coefficient*Min 20.00 10.00 0.21 34.21Q1 22.00 11.00 1.23 89.40Median 26.00 14.00 1.7000 128.53Mean 34.06 15.11 2.40 167.633rd Qu 36.00 17.00 2.67 190.93Max 296.00 38.00 38.00 1678.62In contrast to co-citation frequencies for delayed co-citations (Fig. 2), which range from20-260; citation counts for the 1,267 publications that contribute to these 1,196 delayedco-citations range from 121 to 190,832 with 72 of these publications having citation countsof greater than 10,000.However, other co-citation frequencies do exceed the seemingly modest frequenciesnoted for delayed co-citations. For example, Becke (1993) and Lee et al. (1988), a pairof articles from the field of physical chemistry, have been co-cited over 51,000 times butdo not exhibit delayed citation kinetics. It should also be noted that these articles haveindividually been cited over 70,000 times each. Similarly, 1,357 pairs from the data shownin Fig 1 have co-citation frequencies greater than 1,000.We observe (Fig 1), that the 90th, 95th, and 99th percentiles of co-citation frequenciesin our dataset are 4, 6, and 16 respectively. In comparison. the 90th, 95th, and 99thpercentile of citation frequencies of ∼ further.An appealing alternative approach for delayed co-citations and Sleeping Beauties is theBeauty Coefficient. We have previously modified (Devarakonda et al., 2020) the BeautyCoefficient (Ke et al., 2015) designed to measure kinetics in single publications, to beuseful to the case of co-cited pairs. We computed the Beauty Coefficient for these 1,196pairs observing a range of 34.21-1678.62. These data are summarized in Table 2. Givenco-citation frequencies being generally lower than citation frequencies, the top 15 BeautyCoefficient values of the 1,196 delayed co-citations range from 712.47-1678.62, which appearcomparable to the top 15 described by Ke, all above 2,000.Ke and colleagues comment that parameterized approaches in preceding studies havesuffered from being somewhat arbitrary. The comment is fair, but arbitrariness may nothave impeded discovery, for example Redner’s work on the physics literature (Redner,2005) with its selection threshold of 250 citations. Further, while the Beauty Coefficient isparameter free, the choice of selection threshold is left to the user leaving the door open9or arbitrary selection thresholds. We consider this a strength of the measure since it canbe used in contextual studies. The approach of van Raan is also intuitive and flexible butdoes not consider the maximum number of citations received as an important parameterto be tuned. The cases with a sleeping period of ten years, and a citation rate of 5 forthe next 5 years, would satisfy requirements for a Sleeping Beauty but are perhaps lessnoteworthy.Finally, to ask which fields these 1,196 delayed co-citations are found in, we mappedthem to the All Science Journal Classification (ASJC) maintained by Scopus, which consistsof 27 major subject area categories. The data are represented in Figure 3 but should beinterpreted in the light of these subject area labels being derived from journals and that anarticle may have more than one label. Even so, the data suggest that delayed co-citations, aswe define them in our dataset are largely drawn from the domain of biochemistry, genetics,and molecular biology followed by physics, computer science, chemistry, and engineering.10igure 3: Disciplinary composition of 1,196 Delayed Co-citations. Each node represents amajor subject area in the Scopus ASJC classification. Node size is scaled to the numberarticles in a given subject area. Edge thickness indicates the number of pairs that haveone member in one each of the two nodes connected by the edge. Major subject areas areabbreviated in the graphic: MTH (Mathematics);
IMM (Immunology and Microbiology); HP (Health Professions); GEN (General);
ENS (Environmental Science);
ENG (Engi-neering);
EPS (Earth & Planetary Sciences);
DCS (Decision Sciences);
MAT (MaterialSciences);
CEN (Chemical Engineering);
PSY (Psychology);
PHY (Physics and Astron-omy);
NEU (Neuroscience); CS ( Computer Science); A&H (Arts and Humanities); SS (Social Sciences); MED (Medicine);
EGY (Energy);
CHE (Chemistry);
ABS (Agricul-tural & Biological Sciences);
BGMB (Biochemistry, Genetics & Molecular Biology);
BMA (Business, Management, and Accounting);
EEF (Economics, Econometrics and Finance)
PTP (Pharmacology, Toxicology & Pharmaceutics)11
CONCLUSION
In a large-scale exploration of the kinetics of co-citation (more than 940 million uniquearticle pairs), we have identified 1,196 cases of delayed co-citation using criteria largelyderived from the work of van Raan and Ke. We acknowledge that our selection criteria,while guided by positional statistics and intuitive preference, suffers from some degree ofarbitrariness. With all bibliometric data, coverage and data quality also influence discov-ery. Thus, we have tried to identify co-cited pairs of higher frequency since the trends insuch cases are more likely to be reproducible across other data sources. Relaxing theseconditions, will identify additional cases. Our goal was to identify a set of delayed co-citedpairs that can be studied, in the longer term, to understand the reasons for the patterns ofcitation. This future task will require a greater understanding of the fields in which suchdelayed co-citations occurred and ideally should be coupled to qualitative techniques.
Conflict of Interest Statement
Data used in this study derive from the ERNIE project, which involves a collaborationwith Elsevier. The content of this publication is solely the responsibility of the authorsand does not necessarily represent the official views of the National Institutes of Healthor Elsevier. Elsevier staff did not have a role in design, manuscript-writing, or review andinterpretation of results.
Author Contributions
Wenxi Zhao: Conceptualization; Methodology; Investigation; WritingReview and Editing.Dmitriy Korobskiy: Methodology; Writing Review and Editing; George Chacko: Concep-tualization; Methodology; Investigation; WritingOriginal Draft; WritingReview and Edit-ing; Funding Acquisition, Resources; Supervision.
Funding
Research and development reported in this publication was partially supported by federalfunds from the National Institute on Drug Abuse (NIDA), National Institutes of Health,U.S. Department of Health and Human Services, under Contract Nos. HHSN271201700053C(N43DA-17-1216) and HHSN271201800040C (N44DA-18-1216).
Acknowledgments
We thank our Elsevier colleagues for their support of the ERNIE project.12 ata Availability Statement
Data used in this study are restricted by a license from Elsevier Inc. Interested personswith a license for these data can use the code on our Github repository (Korobskiy et al.,2019) to reproduce our findings.
References
Barber, B. (1961). Resistance by scientists to scientific discovery.
Science , , 596–602.Becke, A. D. (1993). Densityfunctional thermochemistry. III. The role of exact exchange. The Journal of Chemical Physics , (7), 5648–5652. Publisher: American Institute ofPhysics.Boyack, K., & Klavans, R. (2014). Atypical combinations are confounded by disciplinaryeffects. In International Conference on Science and Technology Indicators , (pp. 49–58).Leiden, Netherlands: CWTS-Leiden University.Bradley, J., Devarakonda, S., Davey, A., Korobskiy, D., Liu, S., Lakhdar-Hamina, D.,Warnow, T., & Chacko, G. (2020). Co-citations in context: Disciplinary heterogeneityis relevant.
Quantitative Science Studies , (pp. 1–13).Braun, T., Glnzel, W., & Schubert, A. (2010). On Sleeping Beauties, Princes and othertales of citation distributions .
Research Evaluation , (3), 195–202.Cole, S. (1970). Professional Standing and the Reception of Scientific Discoveries. AmericanJournal of Sociology , (2), 286–306.URL Costas, R., van Leeuwen, T. N., & van Raan, A. F. (2010). Is scientific literature subject toa ’sell-by-date’ ? a general methodology to analyze the ’durability’of scientific documents.
Journal of the American Society for Information Science and Technology , (2), 329–339.Devarakonda, S., Bradley, J., Korobskiy, D., Warnow, T., & Chacko, G. (2020). Preprint:Frequently co-cited publications: Features and kinetics. ArXiv . http://arxiv.org/abs/2005.04793 , In Press (Quantitative Science Studies).URL http://arxiv.org/abs/2005.04793 Garfield, E. (1970). Would Mendels work have been ignored if the Science Citation Indexwas available 100 years ago?
Essays of an Information Scientist , , 69–70.Garfield, E. (1980). Premature Discovery or Delayed Recognition - Why? Essays of anInformation Scientist , , 488–493. 13lnzel, W., & Garfield, E. (2004). The myth of delayed recognition. Scientist , (11), 8.Glnzel, W., Schlemmer, B., & Thijs, B. (2003). Better late than never? on the chanceto become highly cited only beyond the standard bibliometric time horizon. KatholiekeUniversiteit Leuven, Open Access publications from Katholieke Universiteit Leuven , .Ke, Q., Ferrara, E., Radicchi, F., & Flammini, A. (2015). Defining and identifying SleepingBeauties in science. Proceedings of the National Academy of Sciences , (24), 7426–7431.Korobskiy, D., Davey, A., Liu, S., Devarakonda, S., & Chacko, G. (2019). Enhanced Re-search Network Informatics Environment (ERNIE). Github repository, NET ESolutionsCorporation.URL https://github.com/NETESOLUTIONS/ERNIE Lee, C., Yang, W., & Parr, R. G. (1988). Development of the Colle-Salvetti correlation-energy formula into a functional of the electron density.
Physical Review B , (2),785–789. Publisher: American Physical Society.Li, J. (2013). Citation curves of all-elements-sleeping-beauties: flash in the pan first andthen delayed recognition. Scientometrics , , 595–601.Li, J. (2014). Citation curves of all-elements-sleeping-beauties: flash in the pan first andthen delayed recognition. Scientometrics , , 595–601.Li, J., Shi, D., Zhao, S. X., & Ye, F. Y. (2014). A study of the “heartbeat spectra” for“sleeping beauties”. Journal of Informetrics , , 493–502.Li, J., & Ye, F. Y. (2016). Distinguishing sleeping beauties in science. Scientometrics , (2), 821–828.Marshakova-Shaikevich, I. (1973). System of Document Connections Based on References. Scientific and Technical Information Serial of VINITI , (2), 3–8.URL http://garfield.library.upenn.edu/marshakova/marshakovanauchtechn1973.pdf Merton, R. (1963). Resistance to the systematic study of multiple discoveries in science.
European Journal of Sociology , (2), 37–282.Redner, S. (2005). Citation statistics from 110 years of physical review. Physics Today , , 49–54.Small, H. (1973). Co-citation in the scientific literature: A new measure of the relationshipbetween two documents. Journal of the American Society for Information Science , (4),265–269. 14mall, H., & Greenlee, E. (1980). Citation context analysis of a co-citation cluster:Recombinant-DNA. Scientometrics , (4), 277–301.Song, Y., Situ, F., Zhu, H., & Lei, J. (2018). To be the prince to wake up sleeping beauty:the rediscovery of the delayed recognition studies. Scientometrics , (1), 9–24.Sugimoto, C., & Mostafa, J. (2018). A Note of Concern and Context: On CarefulUse ofTerminologies. Journal of the Association for Information Science and Technology , ,347348.Teixeira, A. A., Vieira, P. C., & Abreu, A. P. (2017). Sleeping beauties and their princesin innovation studies. Scientometrics , (2), 541–580.Uzzi, B., Mukherjee, S., Stringer, M., & Jones, B. (2013). Atypical Combinations andScientific Impact. Science , (6157), 468–472.van Raan, A. F. J. (2004). Sleeping Beauties in Science. Scientometrics , (3), 467–472.van Raan, A. F. J., & Winnink, J. J. (2019). The occurrence of Sleeping Beauty publicationsin medical research: Their scientific impact and technological relevance. PLOS ONE , (10), 1–34.Visser, M. (1999). An exotic class of Kaluza-Klein models. ArXiv . ArXiv:hep-th/9910093.Wang, J., Veugelers, R., & Stephan, P. (2017). Bias against novelty in science: A cautionarytale for users of bibliometric indicators.
Research Policy , (8), 1416–1436.Ye, F. Y., & Bornmann, L. (2018). smart girls versus sleeping beauties in the sciences: Theidentification of instant and delayed recognition by using the citation angle. Journal ofthe Association for Information Science and Technology , (3), 359–367.Zong, Z., Liu, X., & Fang, H. (2018). Sleeping beauties with no prince based on theco-citation criterion. Scientometrics ,117