Cui Prodest? Reciprocity of collaboration measured by Russian Index of Science Citation
CCui Prodest? Reciprocity of collaboration measured by Russian Index of Science Citation
Vladimir Pislyakov , Olga Moskaleva and Mark Akoev [email protected] Library, National Research University Higher School of Economics, Myasnitskaya 20, Moscow, 101000 (Russia) [email protected] Saint Petersburg State University, Universitetskaya emb. 7-9, St. Petersburg, 199034 (Russia) [email protected] Ural Federal University, Mira str. 19, Ekaterinburg, 620002 (Russia)
Abstract
Scientific collaboration is often not perfectly reciprocal. Scientifically strong countries/institutions/laboratories may help their less prominent partners with leading scholars, or finance, or other resources. What is interesting in such type of collaboration is that (1) it may be measured by bibliometrics and (2) it may shed more light on the scholarly level of both collaborating organizations themselves. In this sense measuring institutions in collaboration sometimes may tell more than attempts to assess them as stand-alone organizations. Evaluation of collaborative patterns was explained in detail, for example, by Glänzel (2001; 2003). Here we combine these methods with a new one, made available by separating ‘the best’ journals from ‘others’ on the same platform of Russian Index of Science Citation (RISC). Such sub-universes of journals from ‘different leagues’ provide additional methods to study how collaboration influences the quality of papers published by organizations.
Introduction
This paper is a next stage in deep analysis of a relatively new national citation database RISC (Russian Index of Science Citation, launched in 2005). In the article by Moskaleva et al. (2018) the main characteristics and internal structure of RISC were for the first time presented to international audience. Next, our conference paper (Akoev, Moskaleva & Pislyakov, 2018) discussed the essential findings about publication profiles of Russian organizations obtained from the RISC database. Rather intriguing facts were revealed, but it was an analysis of universities/research institutes as stand-alone units. Here we explore collaboration between higher education institutions, their interdependence and their roles in this interdependence. We briefly summarize here for a reader the results of our previous works which are essential for this paper and present new findings on collaborative performance of universities. What is essential and motivated us in this series of studies: (1) the ‘segregation by databases’ method sometimes is more tangible than citation analysis. The latter, of course, has theoretically greater potential but it needs longer time lag and more citation database accuracy to be reliable; (2) these methods are not focused on a single nation case (Russia) but may be used for world literature as soon as ‘publication sub-universes’ are already established by such sub-databases as other national indexes on the Web of Science (WoS) platform or even WoS Emerging Sources Citation Index as opposed to SCIE+SSCI+AHCI, ‘main journal indexes’. One example of this international technique will be presented later in our paper.
ISC and RISC Core
Russian Index of Science Citation (RISC) was launched in 2005 on the platform eLIBRARY.RU of the Russian company
Scientific Electronic Library . As of January 2019, RISC contains more than 31 mln documents with more than 360 mln references, the total list of journals contains more than 60,000 titles, about 6,000 of them are fully indexed in RISC, other journals are sources of documents with Russian affiliations and papers citing them. Along with journals, RISC indexes conference proceedings, books, patents, dissertations and other research artefacts. As it follows from its name, RISC main feature is citation indexing and analytics. However, in our study of collaborations we will not use citation analysis. What is more important for the present paper is that in 2017 a special subset of journals was defined, RISC Core. It contains those RISC journals which are also indexed either in Web of Science or in Scopus. It is a kind of ‘best journals’ subset of RISC periodicals. It should be mentioned that since 2016 there is a special national citation index on the Web of Science platform, Russian Science Citation Index with about 800 journals (RSCI, not to be confused with RISC which is a much bigger database on the separate eLIBRARY.RU platform, with no connection to WoS). Journals for RSCI were selected through professional expertise by specially summoned Russian academia board and by expert opinion of top Russian scientists, more than 12,500 of them voted for journals. All RSCI journals are also included into RISC Core subset. It is important as soon as makes RISC Core independent from decisions made by foreign commercial companies Clarivate Analytics and Elsevier on inclusion/non-inclusion of Russian titles into their databases. This adds value to RISC Core as an instrument for analysis of Russian organizations’ performance. For assessment of the RISC Core a brief experiment was made. It was found that RISC Core which contained in 2016 only 23% of papers of all eLIBRARY.RU platform has attracted 93% of all citations there. This affirms the quality of the RISC Core sources and their status of, so to say, ‘best journals’. Further details on RISC may be found in (Moskaleva et al., 2018; Akoev, Moskaleva & Pislyakov, 2018).
Data and overview of previous results
We study 16 Russian universities and their output indicators taken from RISC database. These universities were chosen by three criteria: (1) they should be leaders by number of documents in RISC as a whole; (2) they should be leaders by number of documents in RISC Core; (3) they should be involved into collaboration with the most of other chosen universities. In tables and graphs we use abbreviated names of these institutions, but in the text we will usually spell them out. The full list of the universities with their abbreviations may be found in Appendix 1. We omit those collaborations which resulted in less than 20 papers in the whole RISC and regard these data as insufficient in all tables. Output indicators of these organizations for the whole RISC and for the RISC Core are taken for five-year 2013–2017 period from the predefined reports in their profiles at eLIBRARY.RU. The same approach was used by Akoev, Moskaleva and Pislyakov (2018) for 2012–2016 time period. Here we reproduce a graph from there (Figure 4 there, Figure 1 here), most important for the current analysis. The graph is corrected so that only universities studied in this paper are shown.
Figure 1. Most prolific Russian Universities: journal papers inside and outside RISC Core. (adapted from Akoev, Moskaleva and Pislyakov, 2018)
We can see that for different universities the share of documents in the Core is drastically different. The highest percentage of the Core papers is found for Moscow Institute of Physics and Technology (MIPT), Moscow Engineering Physics Institute (MEPhI) and Tomsk State University (TSU). The lowest share of their papers in ‘the best journals’ have Academy of National Economy (RANEPA), Plekhanov University of Economics (REA) and Financial University (FU). This previous analysis of the universities as stand-alone organizations will be useful in our study on collaborations.
Results and discussion
Symmetrical (bi-directional) collaboration indexes
First, we study simple symmetrical indexes measuring the strength of collaboration between Russian universities. Generally, two such indexes may be used, Jaccard index or Salton measure (Glänzel, 2003). We use here Jaccard and calculate it both for collaboration in the whole RISC and only in RISC Core. Jaccard index is a ratio of the cardinality of the intersection of two sets to the cardinality of the union of these sets, expressed as a percentage. In our case, it is the number of papers y = x ritten by both organizations jointly divided by the number of papers written at least by one of these organizations. This index gives an estimate of the relative strength of collaboration between two organizations, normalized in a way by the volume of their outputs. Five joint papers are a weaker level of the collaboration when each of the institutions has 1000 articles than when they have published only 10 papers each. Jaccard aims to correct for this difference. To illustrate, the highest Jaccard index in our data is found for collaboration of two Tomsk universities (State and Polytechnic, TSU and TPU) in the RISC Core — 7.22%. This means that every 14th paper (100/7.22) written by scientists either from TSU or TPU in the RISC Core journals is jointly written by them. The full data on Jaccard indices may be found in Appendix 2. For our study, the Jaccard indices themselves are not so interesting as their ratio for collaboration in the RISC Core and RISC as a whole. It will reveal the change of the importance of collaboration between two organizations when moving from the entire database to ‘the best journals’. Table 1 shows Jaccards for inside-core collaboration of each pair of universities divided by Jaccard for their co-authorship in the whole RISC. As soon as Jaccard is a symmetrical indicator, the matrix in Table 1 is also symmetrical.
Table 1. Ratio of Jaccard indices for collaboration in RISC Core/whole RISC. (light blue cell: <1; bold font: >2; yellow cell: >3; empty cell: insufficient data)
There are only 13 out of 120 collaborative pairs in Table 1 for which ratio of Jaccard indices is less than one (these are shown as light blue cells). This means that for 89% pairs of universities their collaboration inside RISC Core is more important than in all other journals in RISC. Extramural collaboration plays a greater role in high-level research. A similar behaviour of Jaccard indices was found by Pislyakov and Shukshina (2014) for international collaboration of Russian organizations in producing highly cited papers. Most remarkable are four pairs shown in Table 1 as yellow cells. These are Plekhanov University of Economics (REA) in collaboration with Kazan Federal University (KFU) and Moscow Engineering Physics Institute (MEPhI); Peoples’ Friendship University (RUDN) again with Kazan Federal University (KFU); and the highest value in Table 1 is for the pair Financial University (FU)/Moscow Medical University (1st MSMU). In the latter case ollaboration of these institutions in the core is 4.65 times more important for them than in the whole RISC. They do not collaborate actively, but when they do so, most of their joint papers become published in the core (15 out of 20). There are five organizations which become more important for any partner when the observation moves to the core publications. These are St. Petersburg University, Moscow Medical University, Tomsk Polytechnic University, Moscow Engineering Physics Institute and Moscow Institute of Physics and Technology. The latter, MIPT, also has the highest average ratio of Jaccards in Table 1 (2.16). And conversely (as we speak of bi-directional indicator), for these universities each other collaborator is also more important in the RISC Core than in the RISC as a whole database. On the other side one can notice Academy of National Economy (RANEPA) performance. There are five cases (out of 13) of weakened collaboration when moving to the Core. Collaboration is not so important for RANEPA when preparing papers for the leading journals. Or, maybe, this relative negligence of collaboration in the RISC Core is a cause of the most unbalanced position of RANEPA concerning inside/outside Core publications observed in Figure 1.
Asymmetrical indexes: collaborative gain
More profound analysis of the patterns of inter-university collaboration may be obtained by investigating asymmetrical, unidirectional indexes which show how profitable is the collaboration for one partner and for another. Here we use an indicator of ‘collaborative gain’. It is a ratio between percentage of papers found in the RISC Core for joint papers written by two institutions and share of Core papers in total output by one of them. In calculating the collaborative gain the numerator of the index is the same for both partner universities, it is a share of their collaborative papers which have entered the Core. But the denominator is different as soon as it is a share of all Core papers for a given institution. That is why A-B index is different from B-A.
Table 2. Collaborative gain received by university in a row from university in a column. (light blue cell: <1; empty cell: insufficient data)
Collaborative gain indicator for 16 universities we study are summarized in Table 2. Collaborative gain may easily be interpreted in such a way: how more often university A ublishes its research in ‘the best journals’ if it collaborates with the university B? For example, value 3.82 in the row 3, column 2 cell of Table 2 means that collaborative papers by Financial University (FU) and Moscow University (MSU) have a 3.82 times higher share in RISC Core than all papers by FU. Columns in Table 2 represent the ‘gain’, ‘profit’ which an organization brings to its partner. Cells marked by light blue color contain values less than one, this means that the gain is negative. In a sense this means that collaboration with this university (in a column) hinders its partner university (in a row). The probability of publishing such collaborative research in high-quality journals is lower than average for organization in a row. We may notice that six out of 16 universities are always positive partners (see their columns in Table 2). Each other organization receives profit from making collaborative research with them. This is, of course, a natural situation as soon as collaboration should bring success. However, several institutions often bring difficulties with their partnership. These ‘negative leaders’ are Financial University (FU) and Academy of National Economy (RANEPA) with nine ‘negative’ cells in their columns, Plekhanov University of Economics (REA) following them with seven negative cases. Note that all three universities are among weakest in Figure 1. And they all are focused more on social sciences. This phenomenon should be scrutinized in further research. The last row of Table 2 contains averages for all indexes for a given university in a column. Although these averages are somewhat artificial, still they may provide some insight into the overall value of the organization as a partner. The data from this row of Table 2 are sorted and organized into Figure 2.
Figure 2. Average collaborative gain a university brings to its partners.
Finally, Table 2 also shows that there are three organizations which collaborations always bring gain to themselves . They are Financial University (FU), Plekhanov University of Economics (REA) and Tomsk Polytechnical University (TPU), their rows do not contain blue cells. The profitable strategy for these institutions is further strengthening of extramural collaborations.
Reciprocity of collaboration
Generally, collaboration is a win-win process when each of the partners receives some surplus compared to its sole performance. We observe such situation for the majority of collaborative airs in Table 2 when both A-B and B-A cells contain values greater than one and in this sense their partnership is reciprocal , profitable for both sides. However, there are some remarkable exceptions. There are three pairs when collaboration is not very successful for both sides: Bauman University (BMSTU) with Kazan University (KFU); Tomsk State University (TSU) with Academy of National Economy (RANEPA); and the Higher School of Economics (HSE) with Peoples’ Friendship University (RUDN). In each case there is negative collaborative effect for both institutions. More often another situation may be found—when one of the partners gains from collaboration while another side loses. There are 9 cases of such kind of ‘collaborative vampirism’ for Financial University (FU), 8 for Academy of National Economy (RANEPA), 7 for Plekhanov University of Economics (REA), 2 for both Southern Federal University (SFU) and Peoples’ Friendship University (RUDN), one for Bauman University (BMSTU), Kazan Federal University (KFU), Moscow Medical University (1st MSMU) and the Higher School of Economics (HSE). These are the cases of non-reciprocal inter-university collaboration, win-loss situation, when partnership benefits one side while harms another. Still, even in win-win situations the gain one university brings to another is generally not equal to the gain it receives in return. To assess this asymmetry, we created Table 3 where differences of collaborative gains received and brought are calculated for each partnership. Positive values in Table 3 mean that university in a column has brought more gain to university in a row than it received in exchange. By definition, this table is skew-symmetric.
Table 3. Non-reciprocity (difference in collaborative gain). How university in a column is non-reciprocal to university in a row. (blue cell: < –3; light blue cell: <–0.1; yellow cell: between –0.1 and 0.1; empty cell: insuff. data)
There are several pairs with almost perfect reciprocity, for example FU/RANEPA. Next, SPBU, UrFU, HSE and TPU also collaborate almost perfectly reciprocally with each other. We should remind, however, that according to our definitions the reciprocal collaboration may exist only between organizations of equal strength, in terms of share of their papers in the RISC Core (when denominators in the formula for collaborative gain are equal). or each partner MIPT brings more gain than receives from them. The opposite is the case of Financial University (FU). Again, overall relative non-reciprocity of a given institution may be estimated with the column average of Table 3. Results are shown in Figure 3. This graph does not necessarily follow the sorting order from the higher to lower share of university’s documents in RISC (as may be suggested by mathematical definition of ‘gain’), at least because there are empty cells in Table 3, not all collaborations are active.
Figure 3. Average non-reciprocity in collaboration (how much a university brings to collaborations compared to how much it receives).
One may say that here we have nine ‘donors’, six ‘acceptors’ and one university (Tomsk Polytechnical) on the borderline. Of course, it is only an approximation and, additionally, made on the limited number of partner universities. Still it may shed light on the collaborative behavior and collaborative gains of the institutions we study. For example, the most significant change is for Kazan Federal University—from 4th place in Figure 2 to 8th in Figure 3. What does it mean? It means that KFU brings much gain in collaborations, but tends to collaborate mainly with those universities, who also give in return. As a result, it receives a lot too. Perhaps this is the most balanced and prudent though not altruistic approach to scientific collaboration.
Conclusion
We have found that study of collaboration between universities shows them in a more detailed manner, revealing their standing and roles in a complex network of collective scientific activity. We conclude that, in general, organizations strong by themselves are also strong collaborators. On the contrary, the universities with low share of papers published in the high-level journals are not very beneficial partners to the others. However, there are some institutions which break this pattern, for example one of the best collaborators appeared to be Moscow Medical University (1st MSMU), which has quite moderate position in Figure 1 showing performance of universities per se. Our method of quality analysis by distribution of papers across journal subsets of different scientific level may be generalized. For example, those studies which analyze scattering of publications across journal quartiles use the same principle. They may focus only on the highest (first) quartile papers as in (Chowdhury & Philipson, 2016) or on full distribution by quartile (Olmeda-Gómez & Moya-Anegón, 2016). Quartiles may come from impact factor (Bordons & Barrigón, 1992), or SJR (Chinchilla-Rodríguez et al., 2015), or some other rankings. The idea of the method remains the same. Moreover, this approach may be applied to Web of Science databases which have different status—namely Emerging Sources Citation Index as opposed to SCIE+SSCI+AHCI. As a small exercise, we have gathered statistics for different countries on how they publish their collaborative papers in different subsets of Web of Science. It appeared that, for example, collaboration of France and Germany is rather reciprocal and benefits both sides: joint Web of Science papers of these countries enter the ‘main’ journal indexes 2.8% and 2.6% more often than all publications of France and Germany, respectively. But at the same time when China partners with Iran, it publishes 5.1% more papers in ‘emerging’ index of WoS, while Iran gets 7.0% more its papers in the ‘main’ SCIE+SSCI+AHCI indexes, compared to its total WoS output. This is an example of non-reciprocal collaboration of countries measured with the international citation database. As next steps of our research we plan to focus on disciplinary differences in collaboration and investigate cross-collaboration patterns between universities and Russian research institutes as observed from Russian Index of Science Citation and its Core.
References
Akoev, M., Moskaleva, O. & Pislyakov, V. (2018). Confidence and RISC: How Russian papers indexed in the national citation database Russian Index of Science Citation (RISC) characterize universities and research institutes. In
STI 2018 Conference Proceedings (pp. 1328–1338). Leiden: Universiteit Leiden – CWTS. Retrieved January 8, 2019 from: https://openaccess.leidenuniv.nl/handle/1887/65344 Bordons, M. & Barrigón, S. (1992). Bibliometric analysis of publications of Spanish pharmacologists in the SCI (1984-89). Part II. Contribution to subfields other than “pharmacology & pharmacy” (ISI).
Scientometrics , 25(3), 425–446. DOI: 10.1007/BF02016930 Chinchilla-Rodríguez, Z., Arencibia-Jorge, R., Moya-Anegón, F.D. & Corera Álvarez, E. (2015). Some patterns of Cuban scientific publication in Scopus: the current situation and challenges.
Scientometrics , 103(3), 779–794. DOI:10.1007/s11192-015-1568-8 Chowdhury, G., Koya, K. & Philipson, P. (2016). Measuring the impact of research: Lessons from the UK's research excellence framework 2014.
PLoS ONE , 11(6). DOI: 10.1371/journal.pone.0156978 Glänzel, W. (2001). National characteristics in international scientific co-authorship relations.
Scientometrics , 51(1), 69–115. DOI: 10.1023/A:1010512628145 Glänzel, W. (2003).
Bibliometrics as a research field: A course on theory and application of bibliometric indicators . Course Handouts, Leuven. Retrieved January 8, 2019 from: http://nsdl.niscair.res.in/jspui/bitstream/123456789/968/1/Bib_Module_KUL.pdf Moskaleva, O., Pislyakov, V., Sterligov, I., Akoev, M. & Shabanova, S. (2018). Russian Index of Science Citation: Overview and review.
Scientometrics , 116(1), 449–462. DOI: 10.1007/s11192-018-2758-y Olmeda-Gómez, C. & Moya-Anegón, F.D. (2016). Publishing Trends in Library and Information Sciences across European Countries and Institutions.
Journal of Academic Librarianship , 42(1), 27–37. DOI: 10.1016/j.acalib.2015.10.005 Pislyakov, V. & Shukshina, E. (2014). Measuring excellence in Russia: Highly cited papers, leading institutions, patterns of national and international collaboration.
Journal of the Association for Information Science and Technology , 65(11), 2321–2330. DOI: 10.1002/asi.23093
Appendix 1. List of universities’ abbreviations.
Abbreviation Full name of university bbreviation Full name of university
FU Financial University under the Government of the Russian Federation HSE National Research University Higher School of Economics KFU Kazan Volga Region Federal University MEPhI Moscow Engineering Physics Institute MIPT Moscow Institute of Physics and Technology MSU Moscow State University RANEPA Russian Presidential Academy of National Economy and Public Administration REA Plekhanov Russian University of Economics RUDN Peoples' Friendship University of Russia SFU Southern Federal University SPBU St. Petersburg State University TPU Tomsk Polytechnic University TSU Tomsk State University UrFU Ural Federal University