Daniel Torres-Salinas

The Google scholar experiment: How to index false papers and manipulate bibliometric indicators

Google Scholar has been well received by the research community. Its promises of free, universal, and easy access to scientific literature coupled with the perception that it covers the social sciences and the humanities better than other traditional multidisciplinary databases have contributed to the quick expansion of Google Scholar Citations and Google Scholar Metrics: 2 new bibliometric products that offer citation data at the individual level and at journal level. In this article, we show the results of an experiment undertaken to analyze Google Scholars capacity to detect citation‐counting manipulation. For this, we uploaded 6 documents to an institutional web domain that were authored by a fictitious researcher and referenced all the publications of the members of the EC3 research group at the University of Granada. The detection by Google Scholar of these papers caused an outburst in the number of citations included in the Google Scholar Citations profiles of the authors. We discuss the effects of such an outburst and how it could affect the future development of such products, at both the individual level and the journal level, especially if Google Scholar persists with its lack of transparency.

Library Catalog Analysis as a tool in studies of social sciences and humanities: An exploratory study of published book titles in Economics

This paper explores the use of Library Catalog Analysis (LCA), defined as the application of bibliometric or informetric techniques to a set of library online catalogs, to describe quantitatively a scientific-scholarly field on the basis of published book titles. It focuses on its value as a tool in studies of Social Sciences and Humanities, especially its cognitive structures, main book publishers and the research performance of its actors. The paper proposes an analogy model between traditional citation analysis of journal articles and Library Catalog Analysis of book titles. It presents the outcomes of an exploratory study of book titles in Economics included in 42 academic library catalogs from 7 countries. It describes the process of data collection and cleaning, and applies a series of indicators and thematic mapping techniques. It illustrates how LCA can be fruitfully used to assess book production and research performance at the level of an individual researcher, a research department, an entire country and a book publisher. It discusses a number of issues that should be addressed in follow-up studies and concludes that LCA of published book titles can be developed into a powerful and useful tool in studies of Social Sciences and Humanities.

Ranking of departments and researchers within a university using two different databases: Web of Science versus Scopus

In this work, we compare the difference in the number of citations compiled with Scopus as opposed to the Web of Science (WoS) with the aim of analysing the agreement among the citation rankings generated by these databases. For this, we analysed the area of Health Sciences of the University of Navarra (Spain), composed of a total of 50 departments and 864 researchers. The total number of published works reflected in the WoS during the period 1999–2005 was 2299. For each work, the number of citations in both databases was recorded. The results indicate that the works received 14.7% more citations in Scopus than in WoS. In the departments, the difference was greater in the clinical ones than in the basic ones. In the case of the rankings of citations, it was found that both databases generate similar results. The Spearman and Kendall-Tau coefficients were higher than 0.9. It was concluded that the difference in the number of citations found did not correspond to the difference of coverage of WoS and Scopus.

New data, new possibilities: exploring the insides of Altmetric.com

This paper analyzes Altmetric.com , one of the most important altmetric data providers currently used. We have analyzed a set of publications with doi number indexed in the Web of Science during the period 2011-2013 and collected their data with the Altmetric API. 19% of the original set of papers was retrieved from Altmetric.com including some altmetric data. We identified 16 different social media sources from which Altmetric.com retrieves data. However five of them cover 95.5% of the total set. Twitter (87.1%) and Mendeley (64.8%) have the highest coverage. We conclude that Altmetric.com is a transparent, rich and accurate tool for altmetric data. Nevertheless, there are still potential limitations on its exhaustiveness as well as on the selection of social media sources that need further research.

Altmetrics: New Indicators for Scientific Communication in Web 2.0

En el presente trabajo se realiza una revision de las altmetrics o indicadores alternativos. Este concepto se define como la creacion y estudio de nuevos indicadores, basados en la web 2.0, para el analisis de la actividad cientifica y academica. La idea que subyace es que, por ejemplo, las menciones en blogs, el numero de tuits o el de personas que guardan un articulo en su gestor de referencias puede ser una medida valida del uso y repercusion de las publicaciones cientificas. En este sentido, estas medidas se han situado en el centro del debate de los estudios bibliometricos cobrando especial relevancia. En el articulo se ilustran en primer lugar las plataformas e indicadores principales de este tipo de medidas, para posteriormente estudiar un conjunto de trabajos del ambito de la comunicacion, comparando el numero de citas recibidas con sus indicadores 2.0. Los resultados senalan que los articulos mas citados de la disciplina en los ultimos anos tambien presentan indicadores significativamente mas elevados de altmetrics. Seguidamente se realiza un repaso por los principales estudios empiricos realizados, deteniendonos en las correlaciones entre indicadores bibliometricos y alternativos. Se finaliza, a modo de reflexion, senalando las principales limitaciones y el papel que las altmetrics pueden desempenar a la hora de captar la repercusion de la investigacion en las plataformas de la web 2.0.In this paper we review the socalled altmetrics or alternative metrics. This concept raises from the development of new indicators based on Web 2.0, for the evaluation of the research and academic activity. The basic assumption is that variables such as mentions in blogs, number of twits or of researchers bookmarking a research paper for instance, may be legitimate indicators for measuring the use and impact of scientific publications. In this sense, these indicators are currently the focus of the bibliometric community and are being discussed and debated. We describe the main platforms and indicators and we analyze as a sample the Spanish research output in Communication Studies. Comparing traditional indicators such as citations with these new indicators. The results show that the most cited papers are also the ones with a highest impact according to the altmetrics. We conclude pointing out the main shortcomings these metrics present and the role they may play when measuring the research impact through 2.0 platforms.

Analyzing data citation practices using the data citation index

We present an analysis of data citation practices based on the Data Citation Index (DCI) (Thomson Reuters). This database launched in 2012 links data sets and data studies with citations received from the other citation indexes. The DCI harvests citations to research data from papers indexed in the Web of Science. It relies on the information provided by the data repository. The findings of this study show that data citation practices are far from common in most research fields. Some differences have been reported on the way researchers cite data: Although in the areas of science and engineering & technology data sets were the most cited, in the social sciences and arts & humanities data studies play a greater role. A total of 88.1% of the records have received no citation, but some repositories show very low uncitedness rates. Although data citation practices are rare in most fields, they have expanded in disciplines such as crystallography and genomics. We conclude by emphasizing the role that the DCI could play in encouraging the consistent, standardized citation of research data—a role that would enhance their value as a means of following the research process from data collection to publication.

A methodology for Institution-Field ranking based on a bidimensional analysis: the IFQ2A index

The problem of comparing academic institutions in terms of their research production is nowadays a priority issue. This paper proposes a relative bidimensional index that takes into account both the net production and the quality of it, as an attempt to provide a comprehensive and objective way to compare the research output of different institutions in a specific field, using journal contributions and citations. The proposed index is then applied, as a case study, to rank the top Spanish universities in the fields of Chemistry and Computer Science in the period ranging from 2000 until 2009. A comparison with the top 50 universities in the ARWU rankings is also made, showing the proposed ranking is better suited to distinguish among non-elite universities.

Mapping citation patterns of book chapters in the Book Citation Index

In this paper we provide the reader with a visual representation of relationships among the impact of book chapters indexed in the Book Citation Index using information gain values and published by different academic publishers in specific disciplines. The impact of book chapters can be characterized statistically by citations histograms. For instance, we can compute the probability of occurrence of book chapters with a number of citations in different intervals for each academic publisher. We predict the similarity between two citation histograms based on the amount of relative information between such characterizations. We observe that the citation patterns of book chapters follow a Lotkaian distribution. This paper describes the structure of the Book Citation Index using ‘heliocentric clockwise maps’ which allow the reader not only to determine the grade of similarity of a given academic publisher indexed in the Book Citation Index with a specific discipline according to their citation distribution, but also to easily observe the general structure of a discipline, identifying the publishers with higher impact and output.

Mapping academic institutions according to their journal publication profile: Spanish universities as a case study

We introduce a novel methodology for mapping academic institutions based on their journal publication profiles. We believe that journals in which researchers from academic institutions publish their works can be considered as useful identifiers for representing the relationships between these institutions and establishing comparisons. However, when academic journals are used for research output representation, distinctions must be introduced between them, based on their value as institution descriptors. This leads us to the use of journal weights attached to the institution identifiers. Since a journal in which researchers from a large proportion of institutions published their papers may be a bad indicator of similarity between two academic institutions, it seems reasonable to weight it in accordance with how frequently researchers from different institutions published their papers in this journal. Cluster analysis can then be applied to group the academic institutions, and dendrograms can be provided to illustrate groups of institutions following agglomerative hierarchical clustering. In order to test this methodology, we use a sample of Spanish universities as a case study. We first map the study sample according to an institutions overall research output, then we use it for two scientific fields (Information and Communication Technologies, as well as Medicine and Pharmacology) as a means to demonstrate how our methodology can be applied, not only for analyzing institutions as a whole, but also in different disciplinary contexts.

Altmetrics: nuevos indicadores para la comunicación científica en la Web 2.0

En el presente trabajo se realiza una revision de las altmetrics o indicadores alternativos. Este concepto se define como la creacion y estudio de nuevos indicadores, basados en la web 2.0, para el analisis de la actividad cientifica y academica. La idea que subyace es que, por ejemplo, las menciones en blogs, el numero de tuits o el de personas que guardan un articulo en su gestor de referencias puede ser una medida valida del uso y repercusion de las publicaciones cientificas. En este sentido, estas medidas se han situado en el centro del debate de los estudios bibliometricos cobrando especial relevancia. En el articulo se ilustran en primer lugar las plataformas e indicadores principales de este tipo de medidas, para posteriormente estudiar un conjunto de trabajos del ambito de la comunicacion, comparando el numero de citas recibidas con sus indicadores 2.0. Los resultados senalan que los articulos mas citados de la disciplina en los ultimos anos tambien presentan indicadores significativamente mas elevados de altmetrics. Seguidamente se realiza un repaso por los principales estudios empiricos realizados, deteniendonos en las correlaciones entre indicadores bibliometricos y alternativos. Se finaliza, a modo de reflexion, senalando las principales limitaciones y el papel que las altmetrics pueden desempenar a la hora de captar la repercusion de la investigacion en las plataformas de la web 2.0.In this paper we review the socalled altmetrics or alternative metrics. This concept raises from the development of new indicators based on Web 2.0, for the evaluation of the research and academic activity. The basic assumption is that variables such as mentions in blogs, number of twits or of researchers bookmarking a research paper for instance, may be legitimate indicators for measuring the use and impact of scientific publications. In this sense, these indicators are currently the focus of the bibliometric community and are being discussed and debated. We describe the main platforms and indicators and we analyze as a sample the Spanish research output in Communication Studies. Comparing traditional indicators such as citations with these new indicators. The results show that the most cited papers are also the ones with a highest impact according to the altmetrics. We conclude pointing out the main shortcomings these metrics present and the role they may play when measuring the research impact through 2.0 platforms.