Pirkko Nykänen

STARE-HI - Statement on reporting of evaluation studies in Health Informatics

OBJECTIVE Development of guidelines for publication of evaluation studies of Health Informatics applications. METHODS An initial list of issues to be addressed in reports on evaluation studies was drafted based on experiences as editors and reviewers of journals in Health Informatics and as authors of systematic reviews of Health Informatics studies, taking into account guidelines for reporting of medical research. This list has been discussed in several rounds by an increasing number of experts in Health Informatics evaluation during conferences and by using e-mail and has been put up for comments on the web. RESULTS A set of STARE-HI principles to be addressed in papers describing evaluations of Health Informatics interventions is presented. These principles include formulation of title and abstract, of introduction (e.g. scientific background, study objectives), study context (e.g. organizational setting, system details), methods (e.g. study design, outcome measures), results (e.g. study findings, unexpected observations) and discussion and conclusion of an IT evaluation paper. CONCLUSION A comprehensive list of principles relevant for properly describing Health Informatics evaluations has been developed. When manuscripts submitted to Health Informatics journals and general medical journals adhere to these aspects, readers will be better positioned to place the studies in a proper context and judge their validity and generalisability. It will also be possible to judge better whether papers will fit in the scope of meta-analyses of Health Informatics interventions. STARE-HI may also be used for study planning and hence positively influence the quality of evaluation studies in Health Informatics. We believe that better publication of both quantitative and qualitative evaluation studies is an important step toward the vision of evidence-based Health Informatics. LIMITATIONS This study is based on experiences from editors, reviewers, authors of systematic reviews and readers of the scientific literature. The applicability of the principles has not been evaluated in real practice. Only when authors start to use these principles for reporting, shortcomings in the principles will emerge.

A Conceptual Framework and Principles for Trusted Pervasive Health

Background Ubiquitous computing technology, sensor networks, wireless communication and the latest developments of the Internet have enabled the rise of a new concept—pervasive health—which takes place in an open, unsecure, and highly dynamic environment (ie, in the information space). To be successful, pervasive health requires implementable principles for privacy and trustworthiness. Objective This research has two interconnected objectives. The first is to define pervasive health as a system and to understand its trust and privacy challenges. The second goal is to build a conceptual model for pervasive health and use it to develop principles and polices which can make pervasive health trustworthy. Methods In this study, a five-step system analysis method is used. Pervasive health is defined using a metaphor of digital bubbles. A conceptual framework model focused on trustworthiness and privacy is then developed for pervasive health. On that model, principles and rules for trusted information management in pervasive health are defined. Results In the first phase of this study, a new definition of pervasive health was created. Using this model, differences between pervasive health and health care are stated. Reviewed publications demonstrate that the widely used principles of predefined and static trust cannot guarantee trustworthiness and privacy in pervasive health. Instead, such an environment requires personal dynamic and context-aware policies, awareness, and transparency. A conceptual framework model focused on information processing in pervasive health is developed. Using features of pervasive health and relations from the framework model, new principles for trusted pervasive health have been developed. The principles propose that personal health data should be under control of the data subject. The person shall have the right to verify the level of trust of any system which collects or processes his or her health information. Principles require that any stakeholder or system collecting or processing health data must support transparency and shall publish its trust and privacy attributes and even its domain specific policies. Conclusions The developed principles enable trustworthiness and guarantee privacy in pervasive health. The implementation of principles requires new infrastructural services such as trust verification and policy conflict resolution. After implementation, the accuracy and usability of principles should be analyzed.

The VATAM guidelines

Evaluation and assessment of the impact of information and communication technology in medicine is gaining interest. Unfortunately, till now there were no agreed upon approaches. The objective of the VATAM project is to develop guidelines that will assist assessors to set-up and execute studies. This paper describes the background of the VATAM project and provides an account of the current state of the guidelines. It concludes with an indication of the developments that will take place in the short term to further elaborate the guidelines and some considerations for consolidation of VATAMs results.

STARE-HI – Statement on Reporting of Evaluation Studies in Health Informatics: Explanation and Elaboration

BACKGROUND Improving the quality of reporting of evaluation studies in health informatics is an important requirement towards the vision of evidence-based health informatics. The STARE-HI - Statement on Reporting of Evaluation Studies in health informatics, published in 2009, provides guidelines on the elements to be contained in an evaluation study report. OBJECTIVES To elaborate on and provide a rationale for the principles of STARE-HI and to guide authors and readers of evaluation studies in health informatics by providing explanatory examples of reporting. METHODS A group of methodologists, researchers and editors prepared the present elaboration of the STARE-HI statement and selected examples from the literature. RESULTS The 35 STARE-HI items to be addressed in evaluation papers describing health informatics interventions are discussed one by one and each is extended with examples and elaborations. CONCLUSION The STARE-HI statement and this elaboration document should be helpful resources to improve reporting of both quantitative and qualitative evaluation studies. Evaluation manuscripts adhering to the principles will enable readers of such papers to better place the studies in a proper context and judge their validity and generalizability, and thus in turn optimize the exploitation of the evidence contained therein. LIMITATIONS This paper is based on experiences of a group of editors, reviewers, authors of systematic reviews and readers of the scientific literature. The applicability of the details of these principles has to evolve as a function of their use in practice.

A Methodology for Evaluation of Knowledge-Based Systems

Knowledge based systems (KBS) are of great potential use in medicine. However, the progress in development and actual day to day use of KBSs in medicine has been disappointing. Shortliffe (1989) believes that this is because medical decision aids are not user friendly, they do not explain their reasoning adequately, they are too time consuming to use in the actual work environment, and particularly because of physician fears of e.g. legal liability and intrusion of the systems into their practice.

Trust information-based privacy architecture for ubiquitous health.

Background Ubiquitous health is defined as a dynamic network of interconnected systems that offers health services independent of time and location to a data subject (DS). The network takes place in open and unsecure information space. It is created and managed by the DS who sets rules that regulate the way personal health information is collected and used. Compared to health care, it is impossible in ubiquitous health to assume the existence of a priori trust between the DS and service providers and to produce privacy using static security services. In ubiquitous health features, business goals and regulations systems followed often remain unknown. Furthermore, health care-specific regulations do not rule the ways health data is processed and shared. To be successful, ubiquitous health requires novel privacy architecture. Objective The goal of this study was to develop a privacy management architecture that helps the DS to create and dynamically manage the network and to maintain information privacy. The architecture should enable the DS to dynamically define service and system-specific rules that regulate the way subject data is processed. The architecture should provide to the DS reliable trust information about systems and assist in the formulation of privacy policies. Furthermore, the architecture should give feedback upon how systems follow the policies of DS and offer protection against privacy and trust threats existing in ubiquitous environments. Methods A sequential method that combines methodologies used in system theory, systems engineering, requirement analysis, and system design was used in the study. In the first phase, principles, trust and privacy models, and viewpoints were selected. Thereafter, functional requirements and services were developed on the basis of a careful analysis of existing research published in journals and conference proceedings. Based on principles, models, and requirements, architectural components and their interconnections were developed using system analysis. Results The architecture mimics the way humans use trust information in decision making, and enables the DS to design system-specific privacy policies using computational trust information that is based on systems’ measured features. The trust attributes that were developed describe the level systems for support awareness and transparency, and how they follow general and domain-specific regulations and laws. The monitoring component of the architecture offers dynamic feedback concerning how the system enforces the polices of DS. Conclusions The privacy management architecture developed in this study enables the DS to dynamically manage information privacy in ubiquitous health and to define individual policies for all systems considering their trust value and corresponding attributes. The DS can also set policies for secondary use and reuse of health information. The architecture offers protection against privacy threats existing in ubiquitous environments. Although the architecture is targeted to ubiquitous health, it can easily be modified to other ubiquitous applications.

Development of personal wellness information model for pervasive healthcare

Copyright

Interpretative reporting and alarming based on laboratory data

The utilisation of laboratory services for patient diagnosis and management involves many steps with both clinical and laboratory components. The clinical components include the decision to order a test, interpretation of the test results and actions taken on the basis of the results. The laboratory components on the other hand include receipt of the request, specimen collection, preparation and analysis, result entry, test result validation and verification and reporting of the results. In this paper, which is part of the OpenLabs project, we concentrate on the post-analytical applications which include interpretation and reporting of the laboratory results to the users in primary care and in high dependency care units. The final objective of the work described is to develop generic modules which can be integrated both with an Open laboratory information system architecture and existing laboratory information processing environment.

Quality of health IT evaluations

Health IT evaluation studies have often been found to be of limited quality. To address this problem, several guidelines and frameworks have been developed as tools to support improvement of the quality of evaluation studies. In this contribution, we review available guidelines and then present the Good Evaluation Practice Guideline in Health Informatics (GEP-HI) in more detail. GEP-HI is a comprehensive guideline which supports especially planning and execution of a health IT evaluation study. The GEP-HI guideline helps to overcome the quality problems related to weak study planning and methodological study design. We also discuss application of GEP-HI on an evaluation project and discuss the need to publish systematically following the recognised publication guidelines. Finally we discuss the future trend on multi-method evaluation approaches.

Clinical decision support systems: need for evidence, need for evaluation.

Modern health care is unthinkable without the progress in heory and practice of health information systems [1]. Health inforatics (alias Health IT, or HIT) in general has been shown to have he potential for positive impact on quality and efficiency of patient are [2–4]. On the other side, experience shows that these benefits re not self-evident; they can only be reached when Health IT is arefully designed, implemented, and managed [5]. Failures in this espect can lead to ill-functioning or user-unfriendly technology hat does not understand and is not well integrated into the clinial workflow and is therefore not accepted by the users [6,7]. This, n turn, can negatively affect clinical processes and may even lead o patient harm [8–10]. In any aspect of healthcare, policies and practice should be rmly based on evidence, and informatics should be no exception 11]. Consequently, any decision regarding design, implementaion, and management of Health IT needs to consider carefully all pportunities and risks for quality and efficiency of patient care, ased on the available scientific evidence. Evaluation is a robust ource of such evidence, provided the evaluation is scientific. Thus, vidence should come from well-designed qualitative and quantiative evaluation studies [12].