Adam G. Dunn

Systematic review automation technologies.

Systematic reviews, a cornerstone of evidence-based medicine, are not produced quickly enough to support clinical practice. The cost of production, availability of the requisite expertise and timeliness are often quoted as major contributors for the delay. This detailed survey of the state of the art of information systems designed to support or automate individual tasks in the systematic review, and in particular systematic reviews of randomized controlled clinical trials, reveals trends that see the convergence of several parallel research projects.We surveyed literature describing informatics systems that support or automate the processes of systematic review or each of the tasks of the systematic review. Several projects focus on automating, simplifying and/or streamlining specific tasks of the systematic review. Some tasks are already fully automated while others are still largely manual. In this review, we describe each task and the effect that its automation would have on the entire systematic review process, summarize the existing information system support for each task, and highlight where further research is needed for realizing automation for the task. Integration of the systems that automate systematic review tasks may lead to a revised systematic review workflow. We envisage the optimized workflow will lead to system in which each systematic review is described as a computer program that automatically retrieves relevant trials, appraises them, extracts and synthesizes data, evaluates the risk of bias, performs meta-analysis calculations, and produces a report in real time.

The automation of systematic reviews

Would lead to best currently available evidence at the push of a button

Associations between exposure to and expression of negative opinions about Human Papillomavirus vaccines on social media: an observational study

Background Groups and individuals that seek to negatively influence public opinion about the safety and value of vaccination are active in online and social media and may influence decision making within some communities. Objective We sought to measure whether exposure to negative opinions about human papillomavirus (HPV) vaccines in Twitter communities is associated with the subsequent expression of negative opinions by explicitly measuring potential information exposure over the social structure of Twitter communities. Methods We hypothesized that prior exposure to opinions rejecting the safety or value of HPV vaccines would be associated with an increased risk of posting similar opinions and tested this hypothesis by analyzing temporal sequences of messages posted on Twitter (tweets). The study design was a retrospective analysis of tweets related to HPV vaccines and the social connections between users. Between October 2013 and April 2014, we collected 83,551 English-language tweets that included terms related to HPV vaccines and the 957,865 social connections among 30,621 users posting or reposting the tweets. Tweets were classified as expressing negative or neutral/positive opinions using a machine learning classifier previously trained on a manually labeled sample. Results During the 6-month period, 25.13% (20,994/83,551) of tweets were classified as negative; among the 30,621 users that tweeted about HPV vaccines, 9046 (29.54%) were exposed to a majority of negative tweets. The likelihood of a user posting a negative tweet after exposure to a majority of negative opinions was 37.78% (2780/7361) compared to 10.92% (1234/11,296) for users who were exposed to a majority of positive and neutral tweets corresponding to a relative risk of 3.46 (95% CI 3.25-3.67, P<.001). Conclusions The heterogeneous community structure on Twitter appears to skew the information to which users are exposed in relation to HPV vaccines. We found that among users that tweeted about HPV vaccines, those who were more often exposed to negative opinions were more likely to subsequently post negative opinions. Although this research may be useful for identifying individuals and groups currently at risk of disproportionate exposure to misinformation about HPV vaccines, there is a clear need for studies capable of determining the factors that affect the formation and adoption of beliefs about public health interventions.

Conflict of interest disclosure in biomedical research: A review of current practices, biases, and the role of public registries in improving transparency

Conflicts of interest held by researchers remain a focus of attention in clinical research. Biases related to these relationships have the potential to directly impact the quality of healthcare by influencing decision-making, yet conflicts of interest remain underreported, inconsistently described, and difficult to access. Initiatives aimed at improving the disclosure of researcher conflicts of interest are still in their infancy but represent a vital reform that must be addressed before potential biases associated with conflicts of interest can be mitigated and trust in the impartiality of clinical evidence restored. In this review, we examine the prevalence of conflicts of interest, evidence of the effects that disclosed and undisclosed conflicts of interest have had on the reporting of clinical evidence, and the emerging approaches for improving the completeness and consistency of disclosures. Through this review of emerging technologies, we recognize a growing interest in publicly accessible registries for researcher conflicts of interest and propose five desiderata aimed at maximizing the value of such registries: mandates for ensuring that researchers keep their records up to date; transparent records that are made available to the public; interoperability to allow researchers, bibliographic databases, and institutions to interact with the registry; a consistent taxonomy for describing different classes of conflicts of interest; and the ability to automatically generate conflicts of interest statements for use in published articles.

Interpreting social network metrics in healthcare organisations: A review and guide to validating small networks

Social network analysis is an increasingly popular sociological method used to describe and understand the social aspects of communication patterns in the health care sector. The networks studied in this area are special because they are small, and for these sizes, the metrics calculated during analysis are sensitive to the number of people in the network and the density of observed communication. Validation is of particular value in controlling for these factors and in assisting in the accurate interpretation of network findings, yet such approaches are rarely applied. Our aim in this paper was to bring together published case studies to demonstrate how a proposed validation technique provides a basis for standardised comparison of networks within and across studies. A validation is performed for three network studies comprising ten networks, where the results are compared within and across the studies in relation to a standard baseline. The results confirm that hierarchy, centralisation and clustering metrics are highly sensitive to changes in size or density. Amongst the three case studies, we found support for some conclusions and contrary evidence for others. This validation approach is a tool for identifying additional features and verifying the conclusions reached in observational studies of small networks. We provide a methodological basis from which to perform intra-study and inter-study comparisons, for the purpose of introducing greater rigour to the use of social network analysis in health care applications.

Characterizing Twitter Discussions About HPV Vaccines Using Topic Modeling and Community Detection

Background In public health surveillance, measuring how information enters and spreads through online communities may help us understand geographical variation in decision making associated with poor health outcomes. Objective Our aim was to evaluate the use of community structure and topic modeling methods as a process for characterizing the clustering of opinions about human papillomavirus (HPV) vaccines on Twitter. Methods The study examined Twitter posts (tweets) collected between October 2013 and October 2015 about HPV vaccines. We tested Latent Dirichlet Allocation and Dirichlet Multinomial Mixture (DMM) models for inferring topics associated with tweets, and community agglomeration (Louvain) and the encoding of random walks (Infomap) methods to detect community structure of the users from their social connections. We examined the alignment between community structure and topics using several common clustering alignment measures and introduced a statistical measure of alignment based on the concentration of specific topics within a small number of communities. Visualizations of the topics and the alignment between topics and communities are presented to support the interpretation of the results in context of public health communication and identification of communities at risk of rejecting the safety and efficacy of HPV vaccines. Results We analyzed 285,417 Twitter posts (tweets) about HPV vaccines from 101,519 users connected by 4,387,524 social connections. Examining the alignment between the community structure and the topics of tweets, the results indicated that the Louvain community detection algorithm together with DMM produced consistently higher alignment values and that alignments were generally higher when the number of topics was lower. After applying the Louvain method and DMM with 30 topics and grouping semantically similar topics in a hierarchy, we characterized 163,148 (57.16%) tweets as evidence and advocacy, and 6244 (2.19%) tweets describing personal experiences. Among the 4548 users who posted experiential tweets, 3449 users (75.84%) were found in communities where the majority of tweets were about evidence and advocacy. Conclusions The use of community detection in concert with topic modeling appears to be a useful way to characterize Twitter communities for the purpose of opinion surveillance in public health applications. Our approach may help identify online communities at risk of being influenced by negative opinions about public health interventions such as HPV vaccines.

Citation networks of related trials are often disconnected: Implications for bidirectional citation searches

BACKGROUND AND OBJECTIVES Reports of randomized controlled trials (RCTs) should set findings within the context of previous research. The resulting network of citations would also provide an alternative search method for clinicians, researchers, and systematic reviewers seeking to base decisions on all available evidence. We sought to determine the connectedness of citation networks of RCTs by examining direct (referenced trials) and indirect (through references of referenced trials, etc) citation of trials to one another. METHODS Meta-analyses were used to create citation networks of RCTs addressing the same clinical questions. The primary measure was the proportion of networks where following citation links between RCTs identifies the complete set of RCTs, forming a single connected citation group. Other measures included the number of disconnected groups (islands) within each network, the number of citations in the network relative to the maximum possible, and the maximum number of links in the path between two connected trials (a measure of indirectness of citations). RESULTS We included 259 meta-analyses with a total of 2,413 and a median of seven RCTs each. For 46% (118 of 259) of networks, the RCTs formed a single connected citation group-one island. For the other 54% of networks, where at least one RCT group was not cited by others, 39% had two citation islands and 4% (10 of 257) had 10 or more islands. On average, the citation networks had 38% of the possible citations to other trials (if each trial had cited all earlier trials). The number of citation islands and the maximum number of citation links increased with increasing numbers of trials in the network. CONCLUSION Available evidence to answer a clinical question may be identified by using network citations created with a small initial corpus of eligible trials. However, the number of islands means that citation networks cannot be relied on for evidence retrieval.

Social and self-reflective use of a Web-based personally controlled health management system.

Background Personally controlled health management systems (PCHMSs) contain a bundle of features to help patients and consumers manage their health. However, it is unclear how consumers actually use a PCHMS in their everyday settings. Objective To conduct an empirical analysis of how consumers used the social (forum and poll) and self-reflective (diary and personal health record [PHR]) features of a Web-based PCHMS designed to support their physical and emotional well-being. Methods A single-group pre/post-test online prospective study was conducted to measure use of a Web-based PCHMS for physical and emotional well-being needs during a university academic semester. The PCHMS integrated an untethered PHR with social forums, polls, a diary, and online messaging links with a health service provider. Well-being journeys additionally provided information to encourage engagement with clinicians and health services. A total of 1985 students and staff aged 18 and above with access to the Internet were recruited online, of which 709 were eligible for analysis. Participants’ self-reported well-being, health status, health service utilization, and help-seeking behaviors were compared using chi-square, McNemar’s test, and Student’s t test. Social networks were constructed to examine the online forum communication patterns among consumers and clinicians. Results The two PCHMS features that were used most frequently and considered most useful and engaging were the social features (ie, the poll and forum). More than 30% (213/709) of participants who sought well-being assistance during the study indicated that other people had influenced their decision to seek help (54.4%, 386/709 sought assistance for physical well-being; 31.7%, 225/709 for emotional well-being). Although the prevalence of using a self-reflective feature (diary or PHR) was not as high (diary: 8.6%, 61/709; PHR: 15.0%, 106/709), the proportion of participants who visited a health care professional during the study was more than 20% greater in the group that did use a self-reflective feature (diary: P=.03; PHR: P<.001). Conclusions There was variation in the degree to which consumers used social and self-reflective PCHMS features but both were significantly associated with increased help-seeking behaviors and health service utilization. A PCHMS should combine both self-reflective as well as socially driven components to most effectively influence consumers’ help-seeking behaviors.

Role of electronic health records in comparative effectiveness research

The gold standard in evaluating treatment effects are randomized controlled trials (RCTs). Their design minimizes bias and maximizes our ability to identify causality. By contrast, observational data, which is routinely collected for other purposes, has many well-known limitations, including selection bias. Why then are we seeing such enthusiasm for ‘big data’ in healthcare, driven in part by the relentless growth in adoption of electronic health records (EHRs)? The first part of the answer is that, despite their many strengths, RCTs also have limitations. First, they do not represent real-world populations or settings. Their stringent exclusion criteria mean that the evidence they produce will not directly replicate the circumstances of many at-risk patients with common comorbidities, who might also benefit from the interventions under trial [1]. Most RCTs are also geographically localized, both in terms of the demographics of the participants as well as in clinical setting. RCTs also often lack the size required to detect the small effect sizes and significant variance encountered in many comparative effectiveness studies, and tend to be too short to detect long-term effects of interventions [2,3]. These limitations are imposed by the need to create controlled conditions, as well as by funding and ethical constraints common to experimental studies. The second motivation for the renewed interest in observational data is the enormous amount of digital data now being collected by clinical institutions, industry and government, and our recent technical capacity to warehouse, link and analyze data in volumes unprecedented a decade ago. Not only are clinical data being accumulated rapidly, they are providing an increasingly detailed record of individual behaviors and journeys. Together, these new attributes of observational data may become a ‘game changer’. The reason we randomize is to deal with the effects of unmodeled variation, and current strategies to deal with such variation in observational studies remain controversial [4]. However, if we had access to health records that included deep phenotypic, genotypic and environmental data, then at some stage we should reach a crossover point where observational data and RCTs are of equivalent value. At that moment we should be able to pull together, through case-matching, a personalized ‘virtual cohort’ of individuals whose collective recorded clinical destiny is at least as predictive of treatment outcomes as any RCT for a given patient. There is ongoing discussion of the relative merits of observational studies and RCTs, and the complementary roles that different forms of evidence play in contributing to the evidence base [3,5–8]. Historically, RCTs were designed to overcome the problems encountered in observational analyses and have therefore been seen as superior to observational studies rather than complementary. However, the rapid growth in EHR data has generated an unprecedented source of information, making it essential that we reassess this artificial wedge separating RCTs and observational studies, and recognize the important complementary roles both must play. “...the recent availability of large electronic health records data sets is challenging us to reconsider the role that observational studies can play in evidence-based medicine.”

Is Biblioleaks Inevitable

In 2014, the vast majority of published biomedical research is still hidden behind paywalls rather than open access. For more than a decade, similar restrictions over other digitally available content have engendered illegal activity. Music file sharing became rampant in the late 1990s as communities formed around new ways to share. The frequency and scale of cyber-attacks against commercial and government interests has increased dramatically. Massive troves of classified government documents have become public through the actions of a few. Yet we have not seen significant growth in the illegal sharing of peer-reviewed academic articles. Should we truly expect that biomedical publishing is somehow at less risk than other content-generating industries? What of the larger threat—a “Biblioleaks” event—a database breach and public leak of the substantial archives of biomedical literature? As the expectation that all research should be available to everyone becomes the norm for a younger generation of researchers and the broader community, the motivations for such a leak are likely to grow. We explore the feasibility and consequences of a Biblioleaks event for researchers, journals, publishers, and the broader communities of doctors and the patients they serve.