Christine L. Borgman

The conundrum of sharing research data

We must all accept that science is data and that data are science, and thus provide for, and justify the need for the support of, much-improved data curation. (Hanson, Sugden, & Alberts) Researchers are producing an unprecedented deluge of data by using new methods and instrumentation. Others may wish to mine these data for new discoveries and innovations. However, research data are not readily available as sharing is common in only a few fields such as astronomy and genomics. Data sharing practices in other fields vary widely. Moreover, research data take many forms, are handled in many ways, using many approaches, and often are difficult to interpret once removed from their initial context. Data sharing is thus a conundrum. Four rationales for sharing data are examined, drawing examples from the sciences, social sciences, and humanities: (1) to reproduce or to verify research, (2) to make results of publicly funded research available to the public, (3) to enable others to ask new questions of extant data, and (4) to advance the state of research and innovation. These rationales differ by the arguments for sharing, by beneficiaries, and by the motivations and incentives of the many stakeholders involved. The challenges are to understand which data might be shared, by whom, with whom, under what conditions, why, and to what effects. Answers will inform data policy and practice.

What are digital libraries? Competing visions

Research and practice in digital libraries (DL) has exploded worldwide in the 1990s. Substantial research funding has become available, libraries are actively involved in DL projects and conferences, journals and online news lists proliferate. This article explores reasons for these developments and the influence of key players, while speculating on future directions. We find that the term ‘digital library’ is used in two distinct senses. In general, researchers view digital libraries as content collected on behalf of user communities, while practicing librarians view digital libraries as institutions or services. Tensions exist between these communities over the scope and concept of the term ‘library’. Research-oriented definitions serve to build a community of researchers and to focus attention on problems to be addressed; these definitions have expanded considerably in scope throughout the 1990s. Library community definitions are more recent and serve to focus attention on practical challenges to be addressed in the transformation of research libraries and universities. Future trends point toward the need for extensive research in digital libraries and for the transformation of libraries as institutions. The present ambiguity of terminology is hindering the advance of research and practice in digital libraries and in our ability to communicate the scope and significance of our work. # 1999 Elsevier Science Ltd. All rights reserved.

Why are online catalogs still hard to use

We return to arguments made 10 years ago (Borgman, 1986a) that online catalogs are difficult to use because their design does not incorporate sufficient understanding of searching behavior. The earlier article examined studies of information retrieval system searching for their implications for online catalog design; this article examines the implications of card catalog design for online catalogs. With this analysis, we hope to contribute to a better understanding of user behavior and to lay to rest the card catalog design model for online catalogs. We discuss the problems with query matching systems, which were designed for skilled search intermediaries rather than end‐users, and the knowledge and skills they require in the information‐seeking process, illustrated with examples of searching card and online catalogs. Searching requires conceptual knowledge of the information retrieval process—translating an information need into a searchable query; semantic knowledge of how to implement a query in a given system—the how and when to use system features; and technical skills in executing the query—basic computing skills and the syntax of entering queries as specific search statements. In the short term, we can help make online catalogs easier to use through improved training and documentation that is based on information‐seeking behavior, with the caveat that good training is not a substitute for good system design. Our long term goal should be to design intuitive systems that require a minimum of instruction. Given the complexity of the information retrieval problem and the limited capabilities of todays systems, we are far from achieving that goal. If libraries are to provide primary information services for the networked world, they need to put research results on the information‐seeking process into practice in designing the next generation of online public access information retrieval systems.

The user's mental model of an information retrieval system: an experiment on a prototype online catalog

Abstract An empirical study was performed to train naive subjects in the use of a prototype Boolean logic-based information retrieval system on a database of bibliographic records. The research was based on the mental models theory which proposes that people can be trained to develop a “mental model” or a qualitative simulation of a system which will aid in generating methods for interacting with the system, debugging errors, and keeping track of ones place in the system. It follows that conceptual training based on a system model will be superior to procedural training based on the mechanics of the system. We performed a laboratory experiment with two training conditions (model and procedural), and with each condition split by sex. Forty-three subjects participated in the experiment, but only 32 were able to reach the minimum competency level required to complete the experiment. The data analysis incorporated time-stamped monitoring data, personal characteristics variables, affective variables, and interview data in which subjects described how they thought the system worked (an articulation of the model). As predicted, the model-based training had no effect on the ability to perform simple, procedural tasks, but subjects trained with a model performed better on complex tasks that required extrapolation from the basic operations of the system. A stochastic process analysis of search-state transitions reinforced this conclusion. Subjects had difficulty articulating a model of the system, and we found no differences in articulation by condition. The high number of subjects (26% ) who were unable to pass the benchmark test indicates that the retrieval tasks were inherently difficult. More interestingly, those who dropped out were significantly more likely to be humanities or social science majors than science or engineering majors, suggesting important individual differences and equity issues. The sex-related differences were slight, although significant, and suggest future research questions.

Why are online catalogs hard to use? Lessons learned from information-retrieval studies

Research in user behavior on online catalogs is in its early stages, but preliminary findings suggest that users encounter many of the same problems identified in behavioral studies of other types of bibliographic retrieval systems. Much can be learned from comparing the results of user behavior studies on these two types of systems. Research on user problems with both the mechanical aspects and the conceptual aspects of system use is reviewed, with the conclusion that more similarity exists across types of systems in conceptual than in mechanical problems. Also discussed are potential sources of the problems, due either to individual characteristics or to system variables. A series of research questions is proposed and a number of potential interim solutions are suggested for alleviating some of the problems encountered by users of information systems.

Children's searching behavior on browsing and keyword online catalogs: the Science Library Catalog project

As we seek both to improve public school education in high technology areas and to link libraries and classrooms on the “information superhighway,” we need to understand more about children’s information searching abilities. We present results of four experiments conducted on four versions of the Science Library Catalog (SLC), a Dewey decimal-based hierarchical browsing system implemented in HyperCard without a keyboard. The experiments were conducted over a 3-year period at three sites, with four databases, and with comparisons to two different keyword online catalogs. Subjects were ethnically and culturally diverse children aged 9 through 12; with 32 to 34 children participating in each experiment. Children were provided explicit instruction and reference materials for the keyword systems but not for the SLC. The number of search topics matched was comparable across all systems and all experiments; search times were comparable, though they varied among the four SLC versions and between the two keyword online public access catalogs (OPACs). The SLC overall was robust to differences in age, sex, and computer experience. One of the keyword OPACs was subject to minor effects of age and computer experience; the other was not. We found relationships between search topic and system structure, such that the most difficult topics on the SLC were those hard to locate in the hierarchy, and those most difficult on the keyword OPACs were hard to spell or required children to generate their own search terms. The SLC approach overcomes problems with several searching features that are difficult for children in typical keyword OPAC systems: typing skills, spelling, vocabulary, and Boolean logic. Results have general implications for the design of information retrieval systems for children.

All users of information retrieval systems are not created equal: an exploration into individual differences

Abstract User performance on information retrieval (IR) systems, as on other computer-based tasks, is highly variable. After briefly reviewing the characteristics on which IR performance varies, we report on a specific study that sought to identify both technical aptitudes and personality characteristics that were related to academic orientation variables previously found to predict IR performance. We found that academic orientation was related to technical aptitudes as measured by standardized achievement tests and coursework; and that academic orientation was related to some personality characteristics, as measured by standardized tests. The relationships were stronger when only subjects who had remained in one discipline were considered. We also found that these technical aptitudes and personality characteristics are largely independent. Since individual differences in information retrieval are assumed not to be random, we argue that individual characteristics should be described so they can be accommodated through design and training.

Little science confronts the data deluge: habitat ecology, embedded sensor networks, and digital libraries

Abstracte-Science promises to increase the pace of science via fast, distributed access to computational resources, analytical tools, and digital libraries. “Big science” fields such as physics and astronomy that collaborate around expensive instrumentation have constructed shared digital libraries to manage their data and documents, while “little science” research areas that gather data through hand-crafted fieldwork continue to manage their data locally. As habitat ecology researchers begin to deploy embedded sensor networks, they are confronting an array of challenges in capturing, organizing, and managing large amounts of data. The scientists and their partners in computer science and engineering make use of common datasets but interpret the data differently. Studies of this field in transition offer insights into the role of digital libraries in e-Science, how data practices evolve as science becomes more instrumented, and how scientists, computer scientists, and engineers collaborate around data. Among the lessons learned are that data on the same variables are gathered by multiple means, that data exist in many states and in many places, and that publication practices often drive data collection practices. Data sharing is embraced in principle but little sharing actually occurs, due to interrelated factors such as lack of demand, lack of standards, and concerns about publication, ownership, data quality, and ethics. We explore the implications of these findings for data policy and digital library architecture. Research reported here is affiliated with the Center for Embedded Networked Sensing.

If we share data, will anyone use them? Data sharing and reuse in the long tail of science and technology.

Research on practices to share and reuse data will inform the design of infrastructure to support data collection, management, and discovery in the long tail of science and technology. These are research domains in which data tend to be local in character, minimally structured, and minimally documented. We report on a ten-year study of the Center for Embedded Network Sensing (CENS), a National Science Foundation Science and Technology Center. We found that CENS researchers are willing to share their data, but few are asked to do so, and in only a few domain areas do their funders or journals require them to deposit data. Few repositories exist to accept data in CENS research areas.. Data sharing tends to occur only through interpersonal exchanges. CENS researchers obtain data from repositories, and occasionally from registries and individuals, to provide context, calibration, or other forms of background for their studies. Neither CENS researchers nor those who request access to CENS data appear to use external data for primary research questions or for replication of studies. CENS researchers are willing to share data if they receive credit and retain first rights to publish their results. Practices of releasing, sharing, and reusing of data in CENS reaffirm the gift culture of scholarship, in which goods are bartered between trusted colleagues rather than treated as commodities.

Science friction: Data, metadata, and collaboration:

When scientists from two or more disciplines work together on related problems, they often face what we call ‘science friction’. As science becomes more data-driven, collaborative, and interdisciplinary, demand increases for interoperability among data, tools, and services. Metadata – usually viewed simply as ‘data about data’, describing objects such as books, journal articles, or datasets – serve key roles in interoperability. Yet we find that metadata may be a source of friction between scientific collaborators, impeding data sharing. We propose an alternative view of metadata, focusing on its role in an ephemeral process of scientific communication, rather than as an enduring outcome or product. We report examples of highly useful, yet ad hoc, incomplete, loosely structured, and mutable, descriptions of data found in our ethnographic studies of several large projects in the environmental sciences. Based on this evidence, we argue that while metadata products can be powerful resources, usually they must be supplemented with metadata processes. Metadata-as-process suggests the very large role of the ad hoc, the incomplete, and the unfinished in everyday scientific work.