David Li Tang

Development and Content Validation of the Information Assessment Method for Patients and Consumers

Background Online consumer health information addresses health problems, self-care, disease prevention, and health care services and is intended for the general public. Using this information, people can improve their knowledge, participation in health decision-making, and health. However, there are no comprehensive instruments to evaluate the value of health information from a consumer perspective. Objective We collaborated with information providers to develop and validate the Information Assessment Method for all (IAM4all) that can be used to collect feedback from information consumers (including patients), and to enable a two-way knowledge translation between information providers and consumers. Methods Content validation steps were followed to develop the IAM4all questionnaire. The first version was based on a theoretical framework from information science, a critical literature review and prior work. Then, 16 laypersons were interviewed on their experience with online health information and specifically their impression of the IAM4all questionnaire. Based on the summaries and interpretations of interviews, questionnaire items were revised, added, and excluded, thus creating the second version of the questionnaire. Subsequently, a panel of 12 information specialists and 8 health researchers participated in an online survey to rate each questionnaire item for relevance, clarity, representativeness, and specificity. The result of this expert panel contributed to the third, current, version of the questionnaire. Results The current version of the IAM4all questionnaire is structured by four levels of outcomes of information seeking/receiving: situational relevance, cognitive impact, information use, and health benefits. Following the interviews and the expert panel survey, 9 questionnaire items were confirmed as relevant, clear, representative, and specific. To improve readability and accessibility for users with a lower level of literacy, 19 items were reworded and all inconsistencies in using a passive or active voice have been solved. One item was removed due to redundancy. The current version of the IAM4all questionnaire contains 28 items. Conclusions We developed and content validated the IAM4all in partnership with information providers, information specialists, researchers and representatives of information consumers. This questionnaire can be integrated within electronic knowledge resources to stimulate users’ reflection (eg, their intention to use information). We claim that any organization (eg, publishers, community organizations, or patient associations), can evaluate and improve their online consumer health information from a consumers’ perspective using this method.

Patient-Oriented Evidence that Matters (POEMs)™ Suggest Potential Clinical Topics for the Choosing Wisely™ Campaign

Objective: We propose a method of identifying clinical topics for campaigns like Choosing Wisely. Methods: In the context of an ongoing continuing medication education program, we analyzed ratings on every patient-oriented evidence that matters (POEM) synopsis delivered in 2012 and 2013. Given the objective of the Choosing Wisely campaign, we focused this analysis on 1 specific item in the validated questionnaire used by physicians to rate POEMs. This questionnaire item is about “avoiding an unnecessary diagnostic test or treatment.” For each POEM, we calculated frequencies and proportions for this item, then we identified the 20 POEMs that were most commonly associated with this item in 2012 and 2013. Finally, we determined whether the clinical topic of each of these POEMs was mentioned in the Choosing Wisely master list. Results: In 2012 and 2013 we received 506,809 completed questionnaires (or ratings) linked to 530 POEMs, for an average of 956 ratings per POEM. In 59% of these POEMs (n = 312), the most commonly expected type of health benefit was “avoiding an unnecessary diagnostic test or treatment.” We then identified the top 20 POEMs most commonly associated with this item in each year by ranking all 312 POEMs from the top down. The clinical topic addressed by 29 of these 40 POEMs was not addressed in the Choosing Wisely master list. These topics fell into 3 categories: diagnostic tests, medical interventions, and surgical interventions. Conclusion: “Big data” can identify clinical topics relevant to campaigns such as Choosing Wisely. This process represents a new way to inform the expert panel approach.

The value of user feedback: Healthcare professionals' comments to the health information provider

The construct of value is highly relevant to information. For research on the value of information, Saracevic and Kantor (1997) proposed a framework from a value perspective in philosophy. In this report, we substantiate the framework with an updated review of the literature and demonstrate its applicability to understanding the value of user feedback as one type of information. Our field study, in the setting of a health information provider whose information products serve thousands of Canadian healthcare professionals, provides an example of how this value‐of‐information framework can be operationalized for an organization. In addition to the theoretical and methodological contributions, this research adds to the literature by documenting the way that textual feedback data were used to optimize the content of an information resource. This contrasts with published studies that only dealt with the use of quantitative feedback by information providers not involved in content production.

Toward an information management system for handling parenting information users' comments

Little is known about how their qualitative feedback can be used by information providers. In this study, researchers worked with information providers, ‘Naitre et grandir’ (N&G), to implement the Information Assessment Method (IAM) for assessing and improving parenting information. Qualitative feedback was collected from participants who visited the N&G website during the study period and who completed an IAM questionnaire. Using an Organizational Participatory Research approach, a coding manual for the identification of participants’ comments was created, and developed by the researchers in partnership with information providers. This manual was used by two coders for classifying participants’ comments. A 4‐step process was followed. For each step, a sample of comments were codes, coding was compared, and codes were further refined. At step‐4, the inter‐coder reliability was tested. This led to a reliable coding manual that will be used in the creation of an online system to facilitate the coding of comments, and provide selected comments to N&G editors on a weekly basis. This system can be adapted by other website editors.

Expected health benefits of e-Therapeutics Highlights according to pharmacists and physicians.

The purpose of this research brief is to describe the expected health benefits of e-Therapeutics Highlights,* as evaluated by Canadian pharmacists and family physicians. The Canadian Pharmacists Association (CPhA), the College of Family Physicians of Canada (CFPC) and the Information Technology Primary Care Research Group (McGill University) collaborated to create an innovative continuing education program, called e-Therapeutics Highlights. Highlights are key treatment recommendations from e-Therapeutics+.** CPhA produces and maintains e-Therapeutics+, a web portal comprising clinical topic summaries that are authored and peer reviewed by subject matter experts (www.e-therapeutics.ca). Once a week, CPhA and CFPC members receive an e-Therapeutics Highlight delivered by e-mail, which they have the option of evaluating by means of a reflective learning activity based on the Information Assessment Method (IAM).1 For each rated Highlight, pharmacists receive continuing education units, and family physicians receive Mainpro credits. We present the aggregated results of IAM ratings as a way of summarizing the wisdom of the crowd of Highlight raters. Crowdsourcing is a force multiplier, defined as the release of online material to a crowd of users who may be interested in contributing ideas or performing a task such as rating a Highlight (voting) and submitting their work to a platform or an organization such as the CPhA, for the profit of the entire community.2,3 Large groups can be collectively wise in identifying relevant information.4 This principle of “crowdsourcing” has been applied to the development of innovative learning network approaches such as those seen in Wikipedia and various voting systems.5,6 Crowdsourcing allows the traditional “ask-the-user” approach to reach a wider audience.7,8

Decisional needs assessment of patients with complex care needs in primary care: a participatory systematic mixed studies review protocol

Introduction Patients with complex care needs (PCCNs) often suffer from combinations of multiple chronic conditions, mental health problems, drug interactions and social vulnerability, which can lead to healthcare services overuse, underuse or misuse. Typically, PCCNs face interactional issues and unmet decisional needs regarding possible options in a cascade of interrelated decisions involving different stakeholders (themselves, their families, their caregivers, their healthcare practitioners). Gaps in knowledge, values clarification and social support in situations where options need to be deliberated hamper effective decision support interventions. This review aims to (1) assess decisional needs of PCCNs from the perspective of stakeholders, (2) build a taxonomy of these decisional needs and (3) prioritise decisional needs with knowledge users (clinicians, patients and managers). Methods and analysis This review will be based on the interprofessional shared decision making (IP-SDM) model and the Ottawa Decision Support Framework. Applying a participatory research approach, we will identify potentially relevant studies through a comprehensive literature search; select relevant ones using eligibility criteria inspired from our previous scoping review on PCCNs; appraise quality using the Mixed Methods Appraisal Tool; conduct a three-step synthesis (sequential exploratory mixed methods design) to build taxonomy of key decisional needs; and integrate these results with those of a parallel PCCNs’ qualitative decisional need assessment (semistructured interviews and focus group with stakeholders). Ethics and dissemination This systematic review, together with the qualitative study (approved by the Centre Intégré Universitaire de Santé et Service Sociaux du Saguenay-Lac-Saint-Jean ethical committee), will produce a working taxonomy of key decisional needs (ontological contribution), to inform the subsequent user-centred design of a support tool for addressing PCCNs’ decisional needs (practical contribution). We will adapt the IP-SDM model, normally dealing with a single decision, for PCCNs who experience cascade of decisions involving different stakeholders (theoretical contribution). Knowledge users will facilitate dissemination of the results in the Canadian primary care network. PROSPERO registration number CRD42015020558.

Customer feedback management: Developing an organizational process of information use

This best-practice poster presents an organization process for making efficient use of customer feedback comments. This process was developed to address a need of the Canadian Pharmacists Association (CPhA) in feedback management. The challenge lies with identifying issues related to feedback (information) use and developing a technological solution to address the issues. As a result of this project, the CPhA can better cope with volumes of feedback comments. This poster reports on the methodology, outcome and experience from a collaborative project between the CPhA and McGill University in Canada, and contributes to the topic of information use in two ways. From the practitioners perspective, our experience is valuable for undertaking similar initiatives of process innovation in organizational settings. From the researchers perspective, this study contributes to scientific knowledge by (1) demonstrating the applicability of Saracevic and Kantors (1997) Acquisition-Cognition- Application model to study information use at the organizational level, as opposed to information use by individuals, as well as (2) identifying three factors uniquely related to information use at that level.