Cassidy R. Sugimoto

Bias in peer review

Research on bias in peer review examines scholarly communication and funding processes to assess the epistemic and social legitimacy of the mechanisms by which knowledge communities vet and self-regulate their work. Despite vocal concerns, a closer look at the empirical and methodological limitations of research on bias raises questions about the existence and extent of many hypothesized forms of bias. In addition, the notion of bias is predicated on an implicit ideal that, once articulated, raises questions about the normative implications of research on bias in peer review. This review provides a brief description of the function, history, and scope of peer review; articulates and critiques the conception of bias unifying research on bias in peer review; characterizes and examines the empirical, methodological, and normative claims of bias in peer review research; and assesses possible alternatives to the status quo. We close by identifying ways to expand conceptions and studies of bias to contend with the complexity of social interactions among actors involved directly and indirectly in peer review.

Tweets as impact indicators: Examining the implications of automated bot accounts on Twitter

This brief communication presents preliminary findings on automated Twitter accounts distributing links to scientific articles deposited on the preprint repository arXiv. It discusses the implication of the presence of such bots from the perspective of social media metrics (altmetrics), where mentions of scholarly documents on Twitter have been suggested as a means of measuring impact that is both broader and timelier than citations. Our results show that automated Twitter accounts create a considerable amount of tweets to scientific articles and that they behave differently than common social bots, which has critical implications for the use of raw tweet counts in research evaluation and assessment. We discuss some definitions of Twitter cyborgs and bots in scholarly communication and propose distinguishing between different levels of engagement—that is, differentiating between tweeting only bibliographic information to discussing or commenting on the content of a scientific work.

A systematic review of interactive information retrieval evaluation studies, 1967–2006

With the increasing number and diversity of search tools available, interest in the evaluation of search systems, particularly from a user perspective, has grown among researchers. More researchers are designing and evaluating interactive information retrieval (IIR) systems and beginning to innovate in evaluation methods. Maturation of a research specialty relies on the ability to replicate research, provide standards for measurement and analysis, and understand past endeavors. This article presents a historical overview of 40 years of IIR evaluation studies using the method of systematic review. A total of 2,791 journal and conference units were manually examined and 127 articles were selected for analysis in this study, based on predefined inclusion and exclusion criteria. These articles were systematically coded using features such as author, publication date, sources and references, and properties of the research method used in the articles, such as number of subjects, tasks, corpora, and measures. Results include data describing the growth of IIR studies over time, the most frequently occurring and cited authors and sources, and the most common types of corpora and measures used. An additional product of this research is a bibliography of IIR evaluation research that can be used by students, teachers, and those new to the area. To the authors knowledge, this is the first historical, systematic characterization of the IIR evaluation literature, including the documentation of methods and measures used by researchers in this specialty.

Institutional interactions: Exploring social, cognitive, and geographic relationships between institutions as demonstrated through citation networks

The objective of this research is to examine the interaction of institutions, based on their citation and collaboration networks. The domain of library and information science is examined, using data from 1965–2010. A linear model is formulated to explore the factors that are associated with institutional citation behaviors, using the number of citations as the dependent variable, and the number of collaborations, physical distance, and topical distance as independent variables. It is found that institutional citation behaviors are associated with social, topical, and geographical factors. Dynamically, the number of citations is becoming more associated with collaboration intensity and less dependent on the country boundary and/or physical distance. This research is informative for scientometricians and policy makers.

Community-based topic modeling for social tagging

Exploring community is fundamental for uncovering the connections between structure and function of complex networks and for practical applications in many disciplines such as biology and sociology. In this paper, we propose a TTR-LDA-Community model which combines the Latent Dirichlet Allocation model (LDA) and the Girvan-Newman community detection algorithm with an inference mechanism. The model is then applied to data from Delicious, a popular social tagging system, over the time period of 2005-2008. Our results show that 1) users in the same community tend to be interested in similar set of topics in all time periods; and 2) topics may divide into several sub-topics and scatter into different communities over time. We evaluate the effectiveness of our model and show that the TTR-LDA-Community model is meaningful for understanding communities and outperforms TTR-LDA and LDA models in tag prediction.

Scholarly use of social media and altmetrics: a review of the literature

Social media has become integrated into the fabric of the scholarly communication system in fundamental ways, principally through scholarly use of social media platforms and the promotion of new indicators on the basis of interactions with these platforms. Research and scholarship in this area has accelerated since the coining and subsequent advocacy for altmetrics—that is, research indicators based on social media activity. This review provides an extensive account of the state‐of‐the art in both scholarly use of social media and altmetrics. The review consists of 2 main parts: the first examines the use of social media in academia, reviewing the various functions these platforms have in the scholarly communication process and the factors that affect this use. The second part reviews empirical studies of altmetrics, discussing the various interpretations of altmetrics, data collection and methodological limitations, and differences according to platform. The review ends with a critical discussion of the implications of this transformation in the scholarly communication system.

Contributorship and division of labor in knowledge production

Scientific authorship has been increasingly complemented with contributorship statements. While such statements are said to ensure more equitable credit and responsibility attribution, they also provide an opportunity to examine the roles and functions that authors play in the construction of knowledge and the relationship between these roles and authorship order. Drawing on a comprehensive and multidisciplinary dataset of 87,002 documents in which contributorship statements are found, this article examines the forms that division of labor takes across disciplines, the relationships between various types of contributions, as well as the relationships between the contribution types and various indicators of authors’ seniority. It shows that scientific work is more highly divided in medical disciplines than in mathematics, physics, and disciplines of the social sciences, and that, with the exception of medicine, the writing of the paper is the task most often associated with authorship. The results suggest a clear distinction between contributions that could be labeled as ‘technical’ and those that could be considered ‘conceptual’: While conceptual tasks are typically associated with authors with higher seniority, technical tasks are more often performed by younger scholars. Finally, results provide evidence of a U-shaped relationship between extent of contribution and author order: In all disciplines, first and last authors typically contribute to more tasks than middle authors. The paper concludes with a discussion of the implications of the results for the reward system of science.

Academic genealogy as an indicator of interdisciplinarity: An examination of dissertation networks in Library and Information Science

Interdisciplinarity has been studied using cognitive connections among individuals in corresponding domains, but rarely from the perspective of academic genealogy. This article utilizes academic genealogy network data from 3,038 PhD dissertations in Library and Information Science (LIS) over a span of 80 years (1930–2009) to describe interdisciplinary changes in the discipline. Aspects of academic pedigree of advisors and committee members are analyzed, such as country, school, and discipline of highest degree, to reveal the interdisciplinary features of LIS. The results demonstrate a strong history of mentors from fields such as education and psychology, a decreasing trend of mentors with LIS degrees, and an increasing trend in mentors receiving degrees in computer science, business, and communication, among other disciplines. This work proposes and explores the use of academic genealogy as an indicator of interdisciplinarity and calls for additional research on the role of doctoral committee composition in a students subsequent academic career.

Biobibliometric profiling: An examination of multifaceted approaches to scholarship

We conducted a fine-grained prosopography of six distinguished information scientists to explore commonalities and differences in their approaches to scholarly production at different stages of their careers. Specifically, we gathered data on authors genre preferences, rates and modes of scholarly production, and coauthorship patterns. We also explored the role played by gender and place in determining mentoring and collaboration practices across time. Our biobibliometric profiles of the sextet reveal the different shapes a scholars career can take. We consider the implications of our findings for new entrants into the academic marketplace.

Scientists have most impact when they’re free to move

An analysis of researchers global mobility reveals that limiting the circulation of scholars will damage the scientific system, say Cassidy R. Sugimoto and colleagues.