Bernhard Breil

Qualitative and quantitative evaluation of EHR-integrated mobile patient questionnaires regarding usability and cost-efficiency

OBJECTIVES The objective of this evaluation study is to assess a web-based application, currently available on iPad, to document questionnaires regarding patient reported outcomes such as quality of life. Based on the single source approach, the results of these questionnaires are available in the electronic health record to be used for treatment and research purposes. The assessment focuses on the usability and efficiency of the system. METHODS The system usability scale questionnaire with seven additional items was used to rate the usability by the patients. It was formally validated by a Cronbach Alpha test. In addition, semi-structured interviews were conducted with patients and medical staff. Time and cost measures, based on official tables of costs, were taken through workflow observations. This study was conducted in the department of dermatology at the University Hospital of Münster, Germany from April to June 2011. RESULTS Using the web-based application questionnaire, results about patient reported outcomes like quality of life are immediately available in the electronic health record and can be used for treatment or research purposes. 118 patients and four staff members participated in the study. The usability score reached 80 from 100 points and patients as well as medical staff stated in the interviews that the usability of the web-based system was high, and they preferred it to the previously used paper-based questionnaires. In the setting of our pilot department the mobile devices amortized their costs after 6.7 months. In general, depending on the professional group who are going to post process the paper-based forms, the earliest break-even point to use mobile questionnaires is at 1737 paper sheets per year. CONCLUSION The mobile patient questionnaires, integrated into the electronic health record, were well accepted in our pilot setting with high usability scores from patients and medical staff alike. The system has also proved to be cost-efficient compared to the paper-based workflow, given that a certain number of questionnaires is used per year.

Mapping Turnaround Times (TAT) to a Generic Timeline: A Systematic Review of TAT Definitions in Clinical Domains

BackgroundAssessing turnaround times can help to analyse workflows in hospital information systems. This paper presents a systematic review of literature concerning different turnaround time definitions. Our objectives were to collect relevant literature with respect to this kind of process times in hospitals and their respective domains. We then analysed the existing definitions and summarised them in an appropriate format.MethodsOur search strategy was based on Pubmed queries and manual reviews of the bibliographies of retrieved articles. Studies were included if precise definitions of turnaround times were available. A generic timeline was designed through a consensus process to provide an overview of these definitions.ResultsMore than 1000 articles were analysed and resulted in 122 papers. Of those, 162 turnaround time definitions in different clinical domains were identified. Starting and end points vary between these domains. To illustrate those turnaround time definitions, a generic timeline was constructed using preferred terms derived from the identified definitions. The consensus process resulted in the following 15 terms: admission, order, biopsy/examination, receipt of specimen in laboratory, procedure completion, interpretation, dictation, transcription, verification, report available, delivery, physician views report, treatment, discharge and discharge letter sent. Based on this analysis, several standard terms for turnaround time definitions are proposed.ConclusionUsing turnaround times to benchmark clinical workflows is still difficult, because even within the same clinical domain many different definitions exist. Mapping of turnaround time definitions to a generic timeline is feasible.

CIS-based registration of quality of life in a single source approach.

BackgroundDocumenting quality of life (QoL) in routine medical care and using it both for treatment and for clinical research is not common, although such information is absolutely valuable for physicians and patients alike. We therefore aimed at developing an efficient method to integrate quality of life information into the clinical information system (CIS) and thus make it available for clinical care and secondary use.MethodsWe piloted our method in three different medical departments, using five different QoL questionnaires. In this setting we used structured interviews and onsite observations to perform workflow and form analyses. The forms and pertinent data reports were implemented using the integrated tools of the local CIS. A web-based application for mobile devices was developed based on XML schemata to facilitate data import into the CIS. Data exports of the CIS were analysed with statistical software to perform an analysis of data quality.ResultsThe quality of life questionnaires are now regularly documented by patients and physicians. The resulting data is available in the Electronic Health Record (EHR) and can be used for treatment purposes and communication as well as research functionalities. The completion of questionnaires by the patients themselves using a mobile device (iPad) and the import of the respective data into the CIS forms were successfully tested in a pilot installation. The quality of data is rendered high by the use of automatic score calculations as well as the automatic creation of forms for follow-up documentation. The QoL data was exported to research databases for use in scientific analysis.ConclusionThe CIS-based QoL is technically feasible, clinically accepted and provides an excellent quality of data for medical treatment and clinical research. Our approach with a commercial CIS and the web-based application is transferable to other sites.

Automated UMLS-Based Comparison of Medical Forms

Medical forms are very heterogeneous: on a European scale there are thousands of data items in several hundred different systems. To enable data exchange for clinical care and research purposes there is a need to develop interoperable documentation systems with harmonized forms for data capture. A prerequisite in this harmonization process is comparison of forms. So far – to our knowledge – an automated method for comparison of medical forms is not available. A form contains a list of data items with corresponding medical concepts. An automatic comparison needs data types, item names and especially item with these unique concept codes from medical terminologies. The scope of the proposed method is a comparison of these items by comparing their concept codes (coded in UMLS). Each data item is represented by item name, concept code and value domain. Two items are called identical, if item name, concept code and value domain are the same. Two items are called matching, if only concept code and value domain are the same. Two items are called similar, if their concept codes are the same, but the value domains are different. Based on these definitions an open-source implementation for automated comparison of medical forms in ODM format with UMLS-based semantic annotations was developed. It is available as package compareODM from http://cran.r-project.org. To evaluate this method, it was applied to a set of 7 real medical forms with 285 data items from a large public ODM repository with forms for different medical purposes (research, quality management, routine care). Comparison results were visualized with grid images and dendrograms. Automated comparison of semantically annotated medical forms is feasible. Dendrograms allow a view on clustered similar forms. The approach is scalable for a large set of real medical forms.

Does single-source create an added value? Evaluating the impact of introducing x4T into the clinical routine on workflow modifications, data quality and cost–benefit

OBJECTIVES The first objective of this study is to evaluate the impact of integrating a single-source system into the routine patient care documentation workflow with respect to process modifications, data quality and execution times in patient care as well as research documentation. The second one is to evaluate whether it is cost-efficient using a single-source system in terms of achieved savings in documentation expenditures. METHODS We analyzed the documentation workflow of routine patient care and research documentation in the medical field of pruritus to identify redundant and error-prone process steps. Based on this, we established a novel documentation workflow including the x4T (exchange for Trials) system to connect hospital information systems with electronic data capture systems for the exchange of study data. To evaluate the workflow modifications, we performed a before/after analysis as well as a time-motion study. Data quality was assessed by measuring completeness, correctness and concordance of previously and newly collected data. A cost-benefit analysis was conducted to estimate the savings using x4T per collected data element and the additional costs for introducing x4T. RESULTS The documentation workflow of patient care as well as clinical research was modified due to the introduction of the x4T system. After x4T implementation and workflow modifications, half of the redundant and error-prone process steps were eliminated. The generic x4T system allows direct transfer of routinely collected health care data into the x4T research database and avoids manual transcription steps. Since x4T has been introduced in March 2012, the number of included patients has increased by about 1000 per year. The average entire documentation time per patient visit has been significantly decreased by 70.1% (from 1116±185 to 334±83 s). After the introduction of the x4T system and associated workflow changes, the completeness of mandatory data elements raised from 82.2% to 100%. In case of the pruritus research study, the additional costs for introducing the x4T system are €434.01 and the savings are 0.48ct per collected data element. So, with the assumption of a 5-year runtime and 82 collected data elements per patient, the amount of documented patients has to be higher than 1102 to create a benefit. CONCLUSION Introduction of the x4T system into the clinical and research documentation workflow can optimize the data collection workflow in both areas. Redundant and cumbersome process steps can be eliminated in the research documentation, with the result of reduced documentation times as well as increased data quality. The usage of the x4T system is especially worthwhile in a study with a large amount of collected data or a high number of included patients.

HIS-based Kaplan-Meier plots - a single source approach for documenting and reusing routine survival information

BackgroundSurvival or outcome information is important for clinical routine as well as for clinical research and should be collected completely, timely and precisely. This information is relevant for multiple usages including quality control, clinical trials, observational studies and epidemiological registries. However, the local hospital information system (HIS) does not support this documentation and therefore this data has to generated by paper based or spreadsheet methods which can result in redundantly documented data. Therefore we investigated, whether integrating the follow-up documentation of different departments in the HIS and reusing it for survival analysis can enable the physician to obtain survival curves in a timely manner and to avoid redundant documentation.MethodsWe analysed the current follow-up process of oncological patients in two departments (urology, haematology) with respect to different documentation forms. We developed a concept for comprehensive survival documentation based on a generic data model and implemented a follow-up form within the HIS of the University Hospital Muenster which is suitable for a secondary use of these data. We designed a query to extract the relevant data from the HIS and implemented Kaplan-Meier plots based on these data. To re-use this data sufficient data quality is needed. We measured completeness of forms with respect to all tumour cases in the clinic and completeness of documented items per form as incomplete information can bias results of the survival analysis.ResultsBased on the form analysis we discovered differences and concordances between both departments. We identified 52 attributes from which 13 were common (e.g. procedures and diagnosis dates) and were used for the generic data model. The electronic follow-up form was integrated in the clinical workflow. Survival data was also retrospectively entered in order to perform survival and quality analyses on a comprehensive data set. Physicians are now able to generate timely Kaplan-Meier plots on current data. We analysed 1029 follow-up forms of 965 patients with survival information between 1992 and 2010. Completeness of forms was 60.2%, completeness of items ranges between 94.3% and 98.5%. Median overall survival time was 16.4 years; median event-free survival time was 7.7 years.ConclusionIt is feasible to integrate survival information into routine HIS documentation such that Kaplan-Meier plots can be generated directly and in a timely manner.

The need for harmonized structured documentation and chances of secondary use - Results of a systematic analysis with automated form comparison for prostate and breast cancer

INTRODUCTION Medical documentation is a time-consuming task and there is a growing number of documentation requirements. In order to improve documentation, harmonization and standardization based on existing forms and medical concepts are needed. Systematic analysis of forms can contribute to standardization building upon new methods for automated comparison of forms. Objectives of this research are quantification and comparison of data elements for breast and prostate cancer to discover similarities, differences and reuse potential between documentation sets. In addition, common data elements for each entity should be identified by automated comparison of forms. MATERIALS AND METHODS A collection of 57 forms regarding prostate and breast cancer from quality management, registries, clinical documentation of two university hospitals (Erlangen, Münster), research datasets, certification requirements and trial documentation were transformed into the Operational Data Model (ODM). These ODM-files were semantically enriched with concept codes and analyzed with the compareODM algorithm. Comparison results were aggregated and lists of common concepts were generated. Grid images, dendrograms and spider charts were used for illustration. RESULTS Overall, 1008 data elements for prostate cancer and 1232 data elements for breast cancer were analyzed. Average routine documentation consists of 390 data elements per disease entity and site. Comparisons of forms identified up to 20 comparable data elements in cancer conference forms from both hospitals. Urology forms contain up to 53 comparable data elements with quality management and up to 21 with registry forms. Urology documentation of both hospitals contains up to 34 comparable items with international common data elements. Clinical documentation sets share up to 24 comparable data elements with trial documentation. Within clinical documentation administrative items are most common comparable items. Selected common medical concepts are contained in up to 16 forms. DISCUSSION The amount of documentation for cancer patients is enormous. There is an urgent need for standardized structured single source documentation. Semantic annotation is time-consuming, but enables automated comparison between different form types, hospital sites and even languages. This approach can help to identify common data elements in medical documentation. Standardization of forms and building up forms on the basis of coding systems is desirable. Several comparable data elements within the analyzed forms demonstrate the harmonization potential, which would enable better data reuse. CONCLUSION Identifying common data elements in medical forms from different settings with systematic and automated form comparison is feasible.

HIS-based electronic documentation can significantly reduce the time from biopsy to final report for prostate tumours and supports quality management as well as clinical research

BackgroundTimely and accurate information is important to guide the medical treatment process. We developed, implemented and assessed an order-entry system to support documentation of prostate histologies involving urologists, pathologists and physicians in private practice.MethodsWe designed electronic forms for histological prostate biopsy reports in our hospital information system (HIS). These forms are created by urologists and sent electronically to pathologists. Pathological findings are entered into the system and sent back to the urologists. We assessed time from biopsy to final report (TBF) and compared pre-implementation phase (paper-based forms) and post-implementation phase. In addition we analysed completeness of the electronic data.ResultsWe compared 87 paper-based with 86 electronic cases. Using electronic forms within the HIS decreases time span from biopsy to final report by more than one day per patient (p < 0.0001). Beyond the optimized workflow we observed a good acceptance because physicians were already familiar with the HIS. The possibility to use these routine data for quality management and research purposes is an additional important advantage of the electronic system.ConclusionElectronic documentation can significantly reduce the time from biopsy to final report of prostate biopsy results and generates a reliable basis for quality management and research purposes.

Preoperative prostate-specific antigen isoform p2PSA≤22.5 pg/ml predicts advanced prostate cancer in patients undergoing radical prostatectomy

BACKGROUND The prediction value of prostate-specific antigen (PSA) isoform [-2]proPSA (p2PSA) for detecting advanced prostate cancer (PCa) remains unclear. Our objective was to evaluate the additional clinical utility of p2PSA compared with total PSA (tPSA), free PSA (fPSA), and preoperative Gleason score (Gls) in predicting locally advanced PCa (pT3/T4) with high-accuracy discrimination. The aim was to develop a novel classification based on p2PSA and preoperative Gls for predicting advanced PCa. MATERIALS AND METHODS In 208 consecutive men diagnosed with clinically localized PCa who underwent radical prostatectomy, we determined the predictive and discriminatory accuracy of serum tPSA, fPSA, percentage of fPSA to tPSA, p2PSA, p2PSA density, percentage of p2PSA to fPSA, and the Prostate Health Index. The cutoff level of p2PSA with best accuracy was estimated. The novel classification was developed by analyzing the interaction between p2PSA and Gls in predicting pathologic outcomes using a chi-square automatic interaction detection analysis. Decision curve analysis was applied to test the clinical consequences of using the novel classification. RESULTS On univariate analyses, p2PSA, p2PSA density, percentage of p2PSA to fPSA, and Prostate Health Index were accurate but were not independent predictors by multivariate analysis. The p2PSA cutoff level of 22.5 pg/ml showed the best accuracy level for predicting and discriminating advanced diseases (area under the curve [AUC] = 0.725, sensitivity = 51.4%, specificity = 81.8%). By chi-square automatic interaction detection, univariate and multivariate analysis, a p2PSA level > 22.5 pg/ml was significantly associated with an increased frequency and risk of advanced disease. In patients with a p2PSA level ≤ 22.5 pg/ml, 91.8% of Gleason sum 6 PCa was organ confined. The combination of p2PSA and Gls enhanced slightly but significantly the predictive and discriminatory accuracy for advanced disease (0.6%-3.6%). CONCLUSIONS The p2PSA cutoff level of 22.5 pg/ml can accurately discriminate between organ-confined and advanced PCa. The additional use of p2PSA enhanced slightly the predictive accuracy for advanced PCa (pT3/pT4) and has limited additional predictive value in identifying aggressive PCa (Gls > 7a).

Diagnostic Classification of Cystoscopic Images Using Deep Convolutional Neural Networks

PURPOSE The recognition of cystoscopic findings remains challenging for young colleagues and depends on the examiners skills. Computer-aided diagnosis tools using feature extraction and deep learning show promise as instruments to perform diagnostic classification. MATERIALS AND METHODS Our study considered 479 patient cases that represented 44 urologic findings. Image color was linearly normalized and was equalized by applying contrast-limited adaptive histogram equalization. Because these findings can be viewed via cystoscopy from every possible angle and side, we ultimately generated images rotated in 10-degree grades and flipped them vertically or horizontally, which resulted in 18,681 images. After image preprocessing, we developed deep convolutional neural network (CNN) models (ResNet50, VGG-19, VGG-16, InceptionV3, and Xception) and evaluated these models using F1 scores. Furthermore, we proposed two CNN concepts: 90%-previous-layer filter size and harmonic-series filter size. A training set (60%), a validation set (10%), and a test set (30%) were randomly generated from the study data set. All models were trained on the training set, validated on the validation set, and evaluated on the test set. RESULTS The Xception-based model achieved the highest F1 score (99.52%), followed by models that were based on ResNet50 (99.48%) and the harmonic-series concept (99.45%). All images with cancer lesions were correctly determined by these models. When the focus was on the images misclassified by the model with the best performance, 7.86% of images that showed bladder stones with indwelling catheter and 1.43% of images that showed bladder diverticulum were falsely classified. CONCLUSION The results of this study show the potential of deep learning for the diagnostic classification of cystoscopic images. Future work will focus on integration of artificial intelligence-aided cystoscopy into clinical routines and possibly expansion to other clinical endoscopy applications.