Juan D. Rogers

A churn model of scientific knowledge value: Internet researchers as a knowledge value collective

Abstract Determining the value of scientific and technical knowledge poses a great many problems. One of the most acute and widely recognized is that the value of knowledge shifts dramatically over time as new uses for the knowledge emerge. A related problem is that market-based valuation of knowledge is an inadequate index of certain types of scientific knowledge. We present an alternative framework for the value of scientific and technical knowledge, one based not on market pricing of information, but instead, on the intensity and range of uses of scientific knowledge. Our “churn” model of scientific knowledge value emphasizes the distinctive properties of scientific and technical knowledge and focuses on the social context of its production. In particular, we consider the value of scientific and technical knowledge in enhancing “knowledge value collectives”, our term for the set of individuals who interact in the demand, production, technical evaluation, and application of scientific and technical knowledge. To illustrate the use of the churn model as an interpretive framework, we examine the recent history of the Internet and the churning knowledge use and transformation accompanying its emergence. The development of the knowledge brought together in the Internet shows us how little traditional disciplines and institutions help in explaining today’s epoch-changing knowledge and technology innovations. We urge a focus on the social configurations producing knowledge value. Rather than counting discrete output, we argue that research evaluation is most helpful when its subject is the capacity of social configurations to produce new scientific and technical knowledge uses.

“Knowledge Value Alliances”: An Alternative to the R&D Project Focus in Evaluation

The question of what the relevant entities or units of analysis for studying the dynamics of R&D are is central not only for adequate characterizations of the system of scientific and technological knowledge production but also for determining the correct focus for evaluation of R&D activities. Typically, R&D performance evaluations have focused not only on the wrong thing but have looked in the wrong place. Most evaluations have been project or program based. Often this focus is misleading. This article presents a “knowledge value” framework as an alternative focus for understanding and evaluating scientific and technical work. This framework consists of two core concepts: the Knowledge Value Collective (KVC) and the Knowledge Value Alliance (KVA). On the basis of the analysis of twenty-eight case studies of research activities, the authors present a typology of KVAs and conclude that they are a better object of evaluation than discipline-based projects.

Obstacles and opportunities in the application of network analysis to the evaluation of R&D

A comprehensive review of studies that apply the network approach to investigating the development of ST examine more closely untidy networks; focus on the content of network links rather than their formal aspects; and develop a concept of ‘network effectiveness’ in terms of the networks ability to expand the uses of S&T knowledge. Copyright , Beech Tree Publishing.

Strategic Management of Government-Sponsored R&D Portfolios

Although strategic management of R&D portfolios is common practice in private sector R&D, government R&D management tends to be more discrete and ad hoc, focusing on generating maximum output from individual projects. Often, there is no clear notion of the desired public sector output. Whereas private sector R&D evaluation is generally straightforward, with the function of R&D being measured in terms of a companys internal return on investment, the benefits of public-sponsored R&D tend to be more diffuse with respect to both type and impact. Drawing from evidence developed in twenty-four case studies of R&D projects sponsored by the Department of Energys Office of Basic Energy Sciences, one of the primary sources of funds for basic research in US universities and national laboratories, we examine implications of R&D project impacts for a sort of ‘portfolio management’ of government-sponsored R&D. Although the meaning of ‘portfolio’ is not the same in public sector R&D management, it is nonetheless possible to think strategically about projects and their cumulative impact. Indeed, it is clear that many government R&D managers already do so. In this study, we contrast two types of ‘portfolio’: (1) R&D output portfolios (focusing on one type of scientific output, such as, for example, fundamental knowledge or technology development); and (2) a balanced portfolio that considers both R&D outputs and ‘scientific and technical human capital’, the capacity created by R&D projects. The case-study evidence shows different approaches to achieving each portfolio type. An analysis of the relation of project attributes to output types focuses on aspects of projects potentially under the control of strategic public managers, including magnitude of funding, degree and type of management oversight, and interorganizational and intraorganizational linkages. Each of these variables affects the type of project outputs obtained. From the results, we suggest that a balanced approach to government R&D portfolio management is appropriate for many government agencies. That is, government managers may wish to consider the extent to which their projects both produce traditional outputs such as articles and patents, as well as provide contributions to scientific and technical human capital.

Engaging national organizations for knowledge translation: Comparative case studies in knowledge value mapping

BackgroundGovernment sponsors of research and development, along with their funded investigators, are increasingly tasked with demonstrating evidence of knowledge use by nontraditional audiences. This requires efforts to translate their findings for effective communication. For technology-related knowledge, these audiences include clinicians, consumers, manufacturers, public policy agencies, and knowledge brokers. One potentially efficient approach is to communicate research findings through relevant national organizations. However, this requires an understanding of how such organizations view and treat research knowledge, which can be determined through knowledge-value mapping. Do knowledge values differ between national organizations representing different audiences? Can a deeper understanding of knowledge values help sponsors, investigators, and organizations better communicate research findings to stakeholders?MethodsA series of comparative case studies on knowledge-value mapping were derived through interviews with spokespersons for six national organizations. The semi-structured interviews followed a 10-item questionnaire to characterize different ways in which each organization engages with research-based knowledge. Each participating organization represents a particular stakeholder group, while all share a common interest in the research subject matter.ResultsEach national organization considers the value of the research knowledge in the context of their organizations mission and the interests of their members. All are interested in collaborating with researchers to share relevant findings, while they vary along the following dimensions of knowledge engagement: create, identify, translate, adapt, communicate, use, promote, absorptive capacity, and recommendations for facilitation.ConclusionsThe principles of knowledge translation suggest that investigators can increase use by tailoring the format and context of their findings to the absorptive capacity of nonscholars. Greater absorption should result in higher levels of knowledge awareness, interest, and use, which can then be documented. National organizations and their members, in turn, can strive to optimize their absorptive capacities regarding the state of the sciences. This combination will ensure the highest possible return on public investment in research activities. This knowledge-value mapping study concludes that national organizations are appropriate channels for communicating research findings and for meeting statutory requirements and general expectations for generating and documenting knowledge use.

Knowledge Translation in Disability and Rehabilitation Research Lessons From the Application of Knowledge Value Mapping to the Case of Accessible Currency

Knowledge translation (KT) has emerged recently in the health science community as a means to address perceived gaps in the application of the best research to treatment of disease. Specifically, in the area of disability and rehabilitation research, the National Institute on Disability and Rehabilitation Research (NIDRR) has identified KT as one of the three areas for critical outcome achievement. This article analyzes some of the issues raised by the notion of KT. First, the article puts KT in the broader context of the study of knowledge flow problems. Second, it introduces the knowledge value mapping (KVM) framework as an avenue for addressing some of the fundamental issues that KT raises in the context of disability and rehabilitation research. Third, it illustrates the application of the framework with a KVM case study of accessible currency. Finally, it discusses the implications of the case study in the broader context of health research agencies such as NIDRR.

Basic research and the success of federal lab-industry partnerships

This paper examines the role that basic research plays in the strategies pursued by industry in their interactions with federal labs. It draws on questionnaire-based data of 229 federal laboratory-industry joint R&D projects with 219 companies and 27 laboratories. The study documents the relative importance of basic research in the success of the interactions by comparing the incidence of basic research on several indicators of success. The study shows that, even though projects involving basic research tend to have higher costs, they also have a high percentage of product outputs in the short term. Typical high payoff strategies for partnership were those in which the company performed several technical roles and the federal laboratory was more narrowly focused.

Citation analysis of nanotechnology at the field level: implications of R&D evaluation

This article addresses the question of how much time it takes for contributions to the nanotechnology literature to establish themselves in the field by analyzing the dynamics of the citations to several cohorts of its papers and the consequences this has for the use of citations in evaluation of R&D. It focuses on the first ten years of publications (cohorts 1991 to 2000) in the field of nanotechnology and eighteen years (1991–2008) worth of citations in windows of increasing length for each cohort to establish some of the basic features of the dynamics of citations in this emerging field. It offers a characterization of the citation distributions of these cohorts of papers and analyzes the time it takes for information contained in those papers to be absorbed by the field as reflected in citations. With a measure developed for that purpose and graphical representation of several dynamical characteristics it finds that there are significant delays in the absorption of information from papers in each cohort. Many papers have sustained growth of citations for many years, sometimes a decade or more, at all levels of the absolute number of citations, and the rank of papers by number of citations has many changes over long periods of time. This suggests that more refined tools for analysis of field level characteristics of impact should be developed to pick up not only the early signs of a potential opportunity in the short term but also recognize topics with older antecedents on their way to a deep and sustained influence. Copyright , Beech Tree Publishing.

Evaluation in R&D management and knowledge use: a knowledge value mapping approach to currency accessible to the visually impaired

The dynamics of knowledge flow between R&D activities and a diverse set of actual or potential users is clearly critical to the realization of desired outcomes. However, current R&D evaluation frameworks focus almost exclusively on measurable discrete outputs and outcomes. We show that more general categories of the realization of values from the creation and utilization of knowledge are needed to make R&D evaluation more relevant to R&D program management. We propose a set of procedures that comprise a knowledge value mapping approach to R&D evaluation. This is achieved by considering the explicit and implicit normative dimension of the evaluation situation as an empirical domain that requires its own analytical elucidation to direct and give meaning to other technical aspects of the evaluation agenda. The benefits for research portfolio management are explored based on the experience of deployment of a specific technology in the context of rehabilitation research in the USA, namely, accessible currency. Copyright , Beech Tree Publishing.

Blind matching versus matchmaking: Comparison group selection for highly creative researchers

This research examines approaches for constructing a comparison group relative to highly creative researchers in nanotechnology and human genetics in the US and Europe. Such a comparison group would be useful in identifying factors that contribute to scientific creativity in these emerging fields. Two comparison group development approaches are investigated. The first approach is based on propensity score analysis and the second is based on knowledge from the literature on scientific creativity and early career patterns. In the first approach, the log of citations over the years of activity in the domains under analysis produces a significant result, but the distribution of matches is not adequate at the middle and high ends of the scale. The second approach matches highly creative researchers in nanotechnology and human genetics with a comparison group of researchers that have the same or similar early career characteristics were considered: (1) same first year of publication (2) same subject category of the first publication, (3) similar publication volume for the first six years in the specified emerging domain. High levels of diversity among the highly creative researchers, especially those in human genetics, underscore the difficulties of constructing a comparison group to understand factors that have brought about their level of performance.