Michael D. Cooper

Usage patterns of a Web-based library catalog

This article reports on a model and patterns of use of a library catalog that can be accessed through the Internet. Three categories of users are identified: individuals who perform a search of the catalog, tourists who look only at opening pages of the library catalogs site, and Web spiders that come to the site to obtain pages for indexing the Web. A number of types of use activities are also identified, and can be grouped with the presearch phase (which takes place before any searching begins): the search phase, the display phase (in which users display the results of their search), and phases in which users make errors, ask the system for help or assistance, and take other actions. An empirical investigation of patterns of use of a university Web-based library catalog was conducted for 479 days. During that period, the characteristics of about 2.5 million sessions were recorded and analyzed, and usage trends were identified. Of the total, 62% of the sessions were for users who performed a search, 27% were from spiders, and 11% were for tourists. During the study period, the average search session lasted about 5 minutes when the study began and had increased to about 10 minutes 16 months later. An average search consisted of about 1.5 presearch actions lasting about 25 seconds, about 5.3 display actions, and 2.5 searches per session. The latter two categories are in the range of 35-37 seconds per session each. There were major differences in usage (number of searches, search time, number of display actions, and display time), depending upon the database accessed.

Using Clustering Techniques To Detect Usage Patterns in a Web-based Information System

Different users of a Web-based information system will have different goals and different ways of performing their work. This article explores the possibility that we can automatically detect usage patterns without demographic information about the individuals. First, a set of 47 variables was defined that can be used to characterize a user session. The values of these variables were computed for approximately 257,000 sessions. Second, principal component analysis was employed to reduce the dimensions of the original data set. Third, a two-stage, hybrid clustering method was proposed to categorize sessions into groups. Finally, an external criteria-based test of cluster validity was performed to verify the validity of the resulting usage groups (clusters). The proposed methodology was demonstrated and tested for validity using two independent samples of user sessions drawn from the transaction logs of the University of Californias MELVYL® on-line library catalog system (www.melvyl.ucop.edu). The results indicate that there were six distinct categories of use in the MELVYL system: knowledgeable and sophisticated use, unsophisticated use, highly interactive use with good search performance, known-item searching, help-intensive searching, and relatively unsuccessful use. Their characteristics were interpreted and compared qualitatively. The analysis shows that each group had distinct patterns of use of the system, which justifies the methodology employed in this study.

Stochastic modeling of usage patterns in a Web-based information system

Users move from one state (or task) to another in an information systems labyrinth as they try to accomplish their work, and the amount of time they spend in each state varies. This article uses continuous-time stochastic models, mainly based on semi-Markov chains, to derive user state transition patterns (both in rates and in probabilities) in a Web-based information system. The methodology was demonstrated with 126,925 search sessions drawn from the transaction logs of the University of Californias MELVYL library catalog system (www.melvyl.ucop.edu). First, user sessions were categorized into six groups based on their similar use of the system. Second, by using a three-layer hierarchical taxonomy of the system Web pages, user sessions in each usage group were transformed into a sequence of states. All the usage groups but one have third-order sequential dependency in state transitions. The sole exception has fourth-order sequential dependency. The transition rates as well as transition probabilities of the semi-Markov model provide a background for interpreting user behavior probabilistically, at various levels of detail. Finally, the differences in derived usage patterns between usage groups were tested statistically. The test results showed that different groups have distinct patterns of system use. Knowledge of the extent of sequential dependency is beneficial because it allows one to predict a users next move in a search space based on the past moves that have been made. It can also be used to help customize the design of the user interface to the system to facilitate interaction. The group CL6 labeled knowledgeable and sophisticated usage and the group CL7 labeled unsophisticated usage both had third-order sequential dependency and had the same most-frequently occurring search pattern: screen display, record display, screen display, and record display. The group CL8 called highly interactive use with good search results had fourth-order sequential dependency, and its most frequently occurring pattern was the same as CL6 and CL7 with one more screen display action added. The group CL13, called known-item searching had third-order sequential dependency, and its most frequently occurring pattern was index access, search with retrievals, screen display, and record display. Group CL14 called help intensive searching, and CL18 called relatively unsuccessful both had third-order sequential dependency, and for both groups the most frequently occurring pattern was index access, search without retrievals, index access, and again, search without retrievals.

Design considerations in instrumenting and monitoring Web-based information retrieval systems

The Internet Web environment opens up extraordinary information are changing rapidly. Daily, hundreds of opportunities for user access to information. Techniques thousands of individuals use the Internet and Web for monitoring users and systems and for evaluating sys- browsers like Netscape to obtain a much broader range tem design and performance have not kept pace with of information than has previously been possible. This Web development. This article reviews concepts of Web information consists not only of bibliographic citations operations (including browsers, clients, information retrieval applications, servers, and data communications but of full text, data, images, sound, programs, and the systems) with specific attention given to how monitoring like. The manner of searching has changed as has the should take place and how privacy can be protected. It content of search results, but the design of transaction examines monitoring needs of users, systems designers, logging programs has not caught up with these changes. managers, and customer support staff and outlines mea- Despite the changing nature of the content of informasures of workload, capacity, and performance for hardware, software, and data communications systems. Fi- tion retrieval systems, the user interface, and the data nally, the article proposes a client-server design for communications interface, a basic research question remonitoring, which involves creation of a series of server mains: Is the system effective in helping users find inforand client systems to obtain and process transaction mation? It is growing more difficult to answer this quesand computer performance information. These systems tion because interface and communications systems are include: A Log Server, which captures all levels of transactions and packets on the network; a Monitor Server, more complex. However, there is much useful informawhich synthesizes the log and packet data; an Assis- tion ideally suited for logging ( such as user actions and tance Server, which processes requests for information system performance ) and these logs can be used to supand help from the Web Server in real time; and an Ac- plement qualitative evaluations or confirm user uncercounting Server, which authenticates user access to the tainty regarding actions taken. system. A special System Administrator Client is proposed to control the monitoring system, as is a System It is extremely expensive to conduct personal interInformation Client to receive real-time and on-demand views with users to find out if they are satisfied with reports of system activity. the information retrieved and / or the information system itself. So efficient evaluation must rely on computerized methods to capture this data as unobtrusively and effec

Usage Patterns of an Online Search System

This article examines the usage patterns of the ELHILL retrieval program of the National Library of Medicines MEDLARS system. Based on a sample of 6759 searches, the study analyzes the frequency of various commands and command options, classifies messages issued by the system, and investigates searcher error rates. The article concludes with suggestions for improving and redesigning both the program and query language.

The structure and future of the information economy

Abstract The structure of the U.S. economy is analyzed to determine whether a shift has occurred from a service to an information economy. National Income Accounting concepts are reviewed to facilitate a critical comparison of the works of Machlup and Porat. Evidence about the future of an information economy is reviewed. It is concluded that the growth of the information economy will not continue at the same rate it has in the past and further that the growth will be in information products, not information services. It is also concluded that it is unwarranted to restructure the National Income Accounts to reflect a change to an information economy.

Charging users for library service

Abstract The question of instituting direct charges for library service is examined and is contrasted with the current indirect changing system where services are paid for by taxes of various types. The policy of whether to levy direct charges for on-line bibliographic searching serves as an example for the analysis. In this case it is concluded that due to considerations of information as a merit good, the service should be supplied free of direct charges to certain users in a specified amount. For other users it is reasonable to impose direct charges.

A cost model for evaluating information retrieval systems

This paper develops a mathematical model for use in studying how to minimize the cost of operating a mechanized retrieval system. Through the use of cost analysis, the model provides a method for comparative evaluation between information retrieval systems. The cost model divides the costs of a retrieval system into two components: system costs and user costs. In addition, it suggests that a trade off exists between the performance level of the system and the combination of user and system time that is expended in working with the system. With this approach it is possible to determine the allocation of user and system. This minimizes the total cost of operating the system. This allocation is done for a given performance level and for a given cost per unit of user and system time.

Response Time Variations in an Online Search System.

The response time characteristics of the National Library of Medicines (NLM) ELHILL bibliographic search system are examined in this article. Transactions for a five-week period are analyzed and average response times are calculated for typical search commands, by time of day, and by file being searched. Overall, the response time of the system was found to be 2.1 seconds, a very low value. Based on statistical tests of significance applied to the data, it was concluded that response time differences can be explained in terms of the number of users on the system and not the command issued by the user nor the file the user searched.

A simulation model of an information retrieval system

Abstract This paper describes a simulation model which is used as a tool for designing and evaluating literature searching systems. The simulation program creates a well specified collection of documents and analyzes the effect of changes in query file characteristics on system output performance. First a thesaurus of term relations is generated. Then, employing the thesaurus, routines generate pseudo-documents and pseudo-queries. These pseudo-documents and pseudo-queries are then compared to see the effect of various query file parameter changes on the quantity of material retrieved. Evaluation of the simulation output indicates that there are small differences between the results of the experimental runs. It is concluded that one method for generating pseudo-queries is not clearly better than another. It is believed, however, that the simulation model as an approach to the evaluation of retrieval systems provides a limited but useful framework for the evaluation of information retrieval systems.