Cathy C. Sarli

Beyond citation analysis: a model for assessment of research impact

QUESTION Is there a means of assessing research impact beyond citation analysis? SETTING The case study took place at the Washington University School of Medicine Becker Medical Library. METHOD This case study analyzed the research study process to identify indicators beyond citation count that demonstrate research impact. MAIN RESULTS The authors discovered a number of indicators that can be documented for assessment of research impact, as well as resources to locate evidence of impact. As a result of the project, the authors developed a model for assessment of research impact, the Becker Medical Library Model for Assessment of Research. CONCLUSION Assessment of research impact using traditional citation analysis alone is not a sufficient tool for assessing the impact of research findings, and it is not predictive of subsequent clinical applications resulting in meaningful health outcomes. The Becker Model can be used by both researchers and librarians to document research impact to supplement citation analysis.

Using publication metrics to highlight academic productivity and research impact.

This article provides a broad overview of widely available measures of academic productivity and impact using publication data and highlights uses of these metrics for various purposes. Metrics based on publication data include measures such as number of publications, number of citations, the journal impact factor score, and the h-index, as well as emerging metrics based on document-level metrics. Publication metrics can be used for a variety of purposes for tenure and promotion, grant applications and renewal reports, benchmarking, recruiting efforts, and administrative purposes for departmental or university performance reports. The authors also highlight practical applications of measuring and reporting academic productivity and impact to emphasize and promote individual investigators, grant applications, or department output.

Breaking down silos: mapping growth of cross-disciplinary collaboration in a translational science initiative.

The importance of transdisciplinary collaboration is growing, though not much is known about how to measure collaboration patterns. The purpose of this paper is to present multiple ways of mapping and evaluating the growth of cross‐disciplinary partnerships over time. Social network analysis was used to examine the impact of a Clinical and Translational Science Award (CTSA) on collaboration patterns. Grant submissions from 2007 through 2010 and publications from 2007 through 2011 of Institute of Clinical and Translational Sciences (ICTS) members were examined. A Cohort Model examining the first‐year ICTS members demonstrated an overall increase in collaborations on grants and publications, as well as an increase in cross‐discipline collaboration as compared to within‐discipline. A Growth Model that included additional members over time demonstrated the same pattern for grant submissions, but a decrease in cross‐discipline collaboration as compared to within‐discipline collaboration for publications. ICTS members generally became more cross‐disciplinary in their collaborations during the CTSA. The exception of publications for the Growth Model may be due to the time lag between funding and publication, as well as pressure for younger scientists to publish in their own fields. Network analysis serves as a valuable tool for evaluating changes in scientific collaboration.

Library-based clinical and translational research support.

There has been a shift in the workflow at academic biomedical research and clinical care centers to promote more efficient clinical and community implementation of bench discoveries. Strong financial support for this effort is provided by the Clinical and Translational Science Awards (CTSAs) from the National Center for Advancing Translational Sciences (NCATS), National Institutes of Health (NIH), awarded to about sixty biomedical research institutions constituting the CTSA Consortium [1]. CTSAs offer an opportunity to speed the translation of bench discoveries to improved human health by transforming the

The Translational Science Benefits Model: A New Framework for Assessing the Health and Societal Benefits of Clinical and Translational Sciences

We report the development of the Translational Science Benefits Model (TSBM), a framework designed to support institutional assessment of clinical and translational research outcomes to measure clinical and community health impacts beyond bibliometric measures. The TSBM includes 30 specific and potentially measurable indicators that reflect benefits that accrue from clinical and translational science research such as products, system characteristics, or activities. Development of the TSBM was based on literature review, a modified Delphi method, and in‐house expert panel feedback. Three case studies illustrate the feasibility and face validity of the TSBM for identification of clinical and community health impacts that result from translational science activities. Future plans for the TSBM include further pilot testing and a resource library that will be freely available for evaluators, translational scientists, and academic institutions who wish to implement the TSBM framework in their own evaluation efforts.

Research evaluation support services in biomedical libraries

Objective The paper provides a review of current practices related to evaluation support services reported by seven biomedical and research libraries. Methods A group of seven libraries from the United States and Canada described their experiences with establishing evaluation support services at their libraries. A questionnaire was distributed among the libraries to elicit information as to program development, service and staffing models, campus partnerships, training, products such as tools and reports, and resources used for evaluation support services. The libraries also reported interesting projects, lessons learned, and future plans. Results The seven libraries profiled in this paper report a variety of service models in providing evaluation support services to meet the needs of campus stakeholders. The service models range from research center cores, partnerships with research groups, and library programs with staff dedicated to evaluation support services. A variety of products and services were described such as an automated tool to develop rank-based metrics, consultation on appropriate metrics to use for evaluation, customized publication and citation reports, resource guides, classes and training, and others. Implementing these services has allowed the libraries to expand their roles on campus and to contribute more directly to the research missions of their institutions. Conclusions Libraries can leverage a variety of evaluation support services as an opportunity to successfully meet an array of challenges confronting the biomedical research community, including robust efforts to report and demonstrate tangible and meaningful outcomes of biomedical research and clinical care. These services represent a transformative direction that can be emulated by other biomedical and research libraries.