Pardeep Sud

Evaluating altmetrics

The rise of the social web and its uptake by scholars has led to the creation of altmetrics, which are social web metrics for academic publications. These new metrics can, in theory, be used in an evaluative role, to give early estimates of the impact of publications or to give estimates of non-traditional types of impact. They can also be used as an information seeking aid: to help draw a digital library user’s attention to papers that have attracted social web mentions. If altmetrics are to be trusted then they must be evaluated to see if the claims made about them are reasonable. Drawing upon previous citation analysis debates and web citation analysis research, this article discusses altmetric evaluation strategies, including correlation tests, content analyses, interviews and pragmatic analyses. It recommends that a range of methods are needed for altmetric evaluations, that the methods should focus on identifying the relative strengths of influences on altmetric creation, and that such evaluations should be prioritised in a logical order.

Commenting on YouTube videos: From guatemalan rock to El Big Bang

YouTube is one of the worlds most popular websites and hosts numerous amateur and professional videos. Comments on these videos might be researched to give insights into audience reactions to important issues or particular videos. Yet, little is known about YouTube discussions in general: how frequent they are, who typically participates, and the role of sentiment. This article fills this gap through an analysis of large samples of text comments on YouTube videos. The results identify patterns and give some benchmarks against which future YouTube research into individual videos can be compared. For instance, the typical YouTube comment was mildly positive, was posted by a 29-year-old male, and contained 58 characters. About 23% of comments in the complete comment sets were replies to previous comments. There was no typical density of discussion on YouTube videos in the sense of the proportion of replies to other comments: videos with both few and many replies were common. The YouTube audience engaged with each other disproportionately when making negative comments, however; positive comments elicited few replies. The biggest trigger of discussion seemed to be religion, whereas the videos attracting the least discussion were predominantly from the Music, Comedy, and How to & Style categories. This suggests different audience uses for YouTube, from passive entertainment to active debating.

A comparison of methods for collecting web citation data for academic organizations

The primary webometric method for estimating the online impact of an organization is to count links to its website. Link counts have been available from commercial search engines for over a decade but this was set to end by early 2012 and so a replacement is needed. This article compares link counts to two alternative methods: URL citations and organization title mentions. New variations of these methods are also introduced. The three methods are compared against each other using Yahoo!. Two of the three methods (URL citations and organization title mentions) are also compared against each other using Bing. Evidence from a case study of 131 UK universities and 49 US Library and Information Science (LIS) departments suggests that Bings Hit Count Estimates (HCEs) for popular title searches are not useful for webometric research but that Yahoo!s HCEs for all three types of search and Bings URL citation HCEs seem to be consistent. For exact URL counts the results of all three methods in Yahoo! and both methods in Bing are also consistent. Four types of accuracy factors are also introduced and defined: search engine coverage, search engine retrieval variation, search engine retrieval anomalies, and query polysemy.

Mendeley readership counts: An investigation of temporal and disciplinary differences

Scientists and managers using citation‐based indicators to help evaluate research cannot evaluate recent articles because of the time needed for citations to accrue. Reading occurs before citing, however, and so it makes sense to count readers rather than citations for recent publications. To assess this, Mendeley readers and citations were obtained for articles from 2004 to late 2014 in five broad categories (agriculture, business, decision science, pharmacy, and the social sciences) and 50 subcategories. In these areas, citation counts tended to increase with every extra year since publication, and readership counts tended to increase faster initially but then stabilize after about 5 years. The correlation between citations and readers was also higher for longer time periods, stabilizing after about 5 years. Although there were substantial differences between broad fields and smaller differences between subfields, the results confirm the value of Mendeley reader counts as early scientific impact indicators.

Not all international collaboration is beneficial: The Mendeley readership and citation impact of biochemical research collaboration

Biochemistry is a highly funded research area that is typified by large research teams and is important for many areas of the life sciences. This article investigates the citation impact and Mendeley readership impact of biochemistry research from 2011 in the Web of Science according to the type of collaboration involved. Negative binomial regression models are used that incorporate, for the first time, the inclusion of specific countries within a team. The results show that, holding other factors constant, larger teams robustly associate with higher impact research, but including additional departments has no effect and adding extra institutions tends to reduce the impact of research. Although international collaboration is apparently not advantageous in general, collaboration with the United States, and perhaps also with some other countries, seems to increase impact. In contrast, collaborations with some other nations seems to decrease impact, although both findings could be due to factors such as differing national proportions of excellent researchers. As a methodological implication, simpler statistical models would find international collaboration to be generally beneficial and so it is important to take into account specific countries when examining collaboration.

Webometric research with the Bing Search API 2.0

In May 2011 the Bing Search API 2.0 had become the only major international web search engine data source available for automatic offline processing for webometric research. This article describes its key features, contrasting them with previous web search data sources, and discussing implications for webometric research. Overall, it seems that large-scale quantitative web research is possible with the Bing Search API 2.0, including query splitting, but that legal issues require the redesign of webometric software to ensure that all results obtained from Bing are displayed directly to the user.

Link and co‐inlink network diagrams with URL citations or title mentions

Webometric network analyses have been used to map the connectivity of groups of websites to identify clusters, important sites or overall structure. Such analyses have mainly been based upon hyperlink counts, the number of hyperlinks between a pair of websites, although some have used title mentions or URL citations instead. The ability to automatically gather hyperlink counts from Yahoo! ceased in April 2011 and the ability to manually gather such counts was due to cease by early 2012, creating a need for alternatives. This article assesses URL citations and title mentions as possible replacements for hyperlinks in both binary and weighted direct link and co-inlink network diagrams. It also assesses three different types of data for the network connections: hit count estimates, counts of matching URLs, and filtered counts of matching URLs. Results from analyses of U.S. library and information science departments and U.K. universities give evidence that metrics based upon URLs or titles can be appropriate replacements for metrics based upon hyperlinks for both binary and weighted networks, although filtered counts of matching URLs are necessary to give the best results for co-title mention and co-URL citation network diagrams.

No citation advantage for monograph-based collaborations?

It is widely believed that collaboration is advantageous in science, for example, with collaboratively written articles tending to attract more citations than solo articles and strong arguments for the value of interdisciplinary collaboration. Nevertheless, it is not known whether the same is true for research that produces books. This article tests whether co-authored scholarly monographs attract more citations than solo monographs using books published before 2011 from 30 categories in the Web of Science. The results show that solo monographs numerically dominate collaborative monographs, but give no evidence of a citation advantage for collaboration on monographs. In contrast, for nearly all these subjects (28 out of 30) there was a citation advantage for collaboratively produced journal articles. As a result, research managers and funders should not incentivise collaborative research in book-based subjects or in research that aims to produce monographs, but should allow the researchers themselves to freely decide whether to collaborate or not.

National, disciplinary and temporal variations in the extent to which articles with more authors have more impact: Evidence from a geometric field normalised citation indicator

The importance of collaboration in research is widely accepted, as is the fact that articles with more authors tend to be more cited. Nevertheless, although previous studies have investigated whether the apparent advantage of collaboration varies by country, discipline, and number of co-authors, this study introduces a more fine-grained method to identify differences: the geometric Mean Normalized Citation Score (gMNCS). Based on comparisons between disciplines, years and countries for two million journal articles, the average citation impact of articles increases with the number of authors, even when international collaboration is excluded. This apparent advantage of collaboration varies substantially by discipline and country and changes a little over time. Against the trend, however, in Russia solo articles have more impact. Across the four broad disciplines examined, collaboration had by far the strongest association with impact in the arts and humanities. Although international comparisons are limited by the availability of systematic data for author country affiliations, the new indicator is the most precise yet and can give statistical evidence rather than estimates.

Linked title mentions: a new automated link search candidate

Many webometric studies have used hyperlinks to investigate links to or between specific collections of websites to estimate their impact or identify connectivity patterns. Whilst major commercial search engines have previously been used to identify hyperlinks for these purposes, their hyperlink search facilities have now been shut down. In response, a range of alternative sources of link data have been suggested, but all have limitations. This article introduces a new type of link that can be identified from commercial search engines, linked title mentions. These can be found by querying title mentions in a search engine and then removing those not associated with a relevant hyperlink. Results of a proof of concept test on 51 U.S. library and information science schools and four other sets of schools suggest that linked title mentions may tend to give better results than title mentions in some cases when used for site inlinks but may not always be an improvement on URL citations. For links between or co-inlinks to specified pairs of academic websites, linked title mentions do not generally provide an improvement over title mentions, but they do over URL citations in some cases. Linked title mentions may also be useful for sets of non-academic websites when the alternatives give too few or misleading results.