Frances C. Johnson

Using the information seeker to elicit construct models for search engine evaluation

With the rise of interactive search engines, direct access to information is available to the end user. Evaluation of these systems is clearly important for their comparison and development. From the perspective of the user engaged with the system in some information-seeking task, the range of relevant evaluation factors could be considerable. This article presents an investigation into the suitability of repertory grid technique eliciting a mental model of search engines in the user. Such a model comprises constructs held important to the user, which can be used to evaluate the system. The design of the repertory grid application is described and the data analyzed in two stages. The important issues in determining the suitability of the method are identified and the constructs are analyzed to determine their association, discriminatory ability, and clustering around a central overall rating. From this, it is concluded that repertory grid technique is appropriate for user-centered determination of evaluative constructs. Further analysis of the characteristics of the resulting construct set is undertaken towards establishing an underlying user model of the search engines, and further research is identified for its use in system evaluation.

User satisfaction as a measure of system performance

It is evident from previous research that user satisfaction is a multidimensional, subjective variable which can be affected by many factors other than performance of the system or searcher. This article draws on information retrieval and information systems literature in an attempt to understand what user satisfaction is, how it is measured, what factors affect it, and why findings on user satisfaction have been so varied and contradictory. It concludes with recommendations for future investigation of the use of user satisfaction as a measure of system performance.

Understanding trust formation in digital information sources: The case of Wikipedia

This article contributes to knowledge on how users establish the trustworthiness of digital information. An exploratory two-stage study was conducted with Master’s and undergraduate students in information studies. In the first phase of the study respondents commented on the factors and processes associated with trust formation. Participants commented on authorship and references, quality of writing and editing, and verification via links to external reference sources. Findings from the second phase, based on a checklist, suggested that participants relied on a range of factors when assessing the trustworthiness of articles, including content factors such as authorship, currency and usefulness together with context factors such as references, expert recommendation and triangulation with their own knowledge. These findings are discussed in the light of previous related research and recommendations for further research are offered.

Automatic abstracting research

The prospect of automatically generating abstracts has attracted researchers for some time, but the promise of superseding the human effort has yet to be realized. Surveys the approaches and techniques developed with the view to showing why this is so. Particular emphasis is placed on the requirements for the production of abstracts, which effectively serve their intended function, to show the ways in which this has hampered research in the past. Suggests that progress of automatic abstracting research may come about via the integration of some of the techniques into computerized information retrieval systems. This will allow researchers to shift the aim from reproducing the conventional benefits of abstracts to accentuating the advantages to users of computerized representation of information in large textual databases.

Students’ trust judgements in online health information seeking:

As one of the most active groups of Internet users, students and other young people are active users of digital health information. Yet, research into young people’s evaluation of health information is limited, and no previous studies have focused on trust formation. In addition, prior studies on adults’ use of digital information do not reach a consensus regarding the key factors in trust formation. This study seeks to address this gap. A questionnaire-based survey was used to collect data from undergraduate students studying a variety of disciplines in one UK university. The Trust in Online Health Information Scale is proposed, and it includes the following dimensions: authority, style, content, usefulness, brand, ease of use, recommendation, credibility, and verification. In addition, inspection of responses to specific items/questions provides further insights into aspects of the information that were of specific importance in influencing trust judgements.

Gender as an influencer of online health information-seeking and evaluation behavior

This article contributes to the growing body of research that explores the significance of context in health information behavior. Specifically, through the lens of trust judgments, it demonstrates that gender is a determinant of the information evaluation process. A questionnaire‐based survey collected data from adults regarding the factors that influence their judgment of the trustworthiness of online health information. Both men and women identified credibility, recommendation, ease of use, and brand as being of importance in their trust judgments. However, women also take into account style, while men eschew this for familiarity. In addition, men appear to be more concerned with the comprehensiveness and accuracy of the information, the ease with which they can access it, and its familiarity, whereas women demonstrate greater interest in cognition, such as the ease with which they can read and understand the information. These gender differences are consistent with the demographic data, which suggest that: women consult more types of sources than men; men are more likely to be searching with respect to a long‐standing health complaint; and, women are more likely than men to use tablets in their health information seeking. Recommendations for further research to better inform practice are offered.

Modelling trust formation in health information contexts

This study explores trust formation in the context of health information. Trust as an interpersonal notion, when formed in a vulnerable state, is a response or belief about how the trusted will behave towards the trustor. This study focuses on the process of assessing the trustworthiness of information, in a dependency state of information need, through the identification of the many factors influencing this assessment. A set of propositions are developed to suggest the criteria by which trustworthiness is assessed as well as the factors that influence these judgements. The proposed model is tested in a large-scale survey using a trust inventory with factor analysis to explore the constructs of trust formation. Structural equation modelling is used to explore the relationship among the identified criteria and their influencing factors. The resulting framework contributes to the understanding of trust formation in digital information contexts on the criteria of usefulness and credibility and further research into the influencing factors is recommended.

BEYOND USABILITY: THE STUDY OF FUNCTIONALITY OF THE 2.0 ONLINE CATALOGUE (OPAC)

A usability test of Copac was carried out to inform developments in both its usability and functionality. Participants were assigned tasks, using specific system features, and their actions and thoughts were collected through think aloud to reveal potential issues with the system. Retrospective thinking and subsequent reflection were gathered in a focus group to provide greater insight into perceived functionality (current and potential) of the system. The study demonstrated the importance of the application of usability principles in the interface design. Many of the issues associated with search, records display, and use of specific features related to the attributes of ease of use, appearance, navigation, and terminology and provided the evidence base for re-design. The study also highlighted the importance of the users’ perceptions and expectations of functionality in the specific usage contexts. Continued user testing is recommended to inform further development of functionality in the broader contexts of information use to ensure the value and use of the system features are evident in the system design and meet the expectations of its users.

Exploring trust in online health information: a study of user experiences of patients.co.uk

This feature has been co-authored by Anna Cunningham and her supervisor Frances Johnson. It is based on the research Anna conducted for her dissertation, which she completed as part of her MA in Library and Information Management at Manchester Metropolitan University. The study explored how people assess the trustworthiness of online health information, and the participants were asked to talk aloud whilst viewing information on the consumer health information website patients.co.uk. The study confirmed that their assessment was based on the information usefulness and credibility as well as identifying the factors relating to information quality and website design that helped to form these judgements. A. M.

Students' approaches to the evaluation of digital information: Insights from their trust judgments

© 2015 British Educational Research Association. This study contributes to an understanding of the role of experience in the evaluation phase of the information search process. A questionnaire-based survey collected data from 1st and 3rd-year undergraduate students regarding the factors that influence their judgment of the trustworthiness of online health information. Exploratory and confirmatory factor analyses were conducted. First-year students identified four factors: ease of use, content, recommendation and brand. Third-year students identified seven factors, in order of importance: content, credibility, recommendation, ease of use, usefulness, style and brand. They also were much clearer about their evaluation processes than 1st-year students; for 3rd-year students, the factor structure was clearer, and items generally loaded onto the expected factors. The significance of these findings is discussed and recommendations for practice and further research are offered.