Leigh A. Baumgart

The Effect of Information Analysis Automation Display Content on Human Judgment Performance in Noisy Environments

Displaying both the strategy that information analysis automation employs to makes its judgments and variability in the task environment may improve human judgment performance, especially in cases where this variability impacts the judgment performance of the information analysis automation. This work investigated the contribution of providing either information analysis automation strategy information, task environment information, or both, on human judgment performance in a domain where noisy sensor data are used by both the human and the information analysis automation to make judgments. In a simplified air traffic conflict prediction experiment, 32 participants made probability of horizontal conflict judgments under different display content conditions. After being exposed to the information analysis automation, judgment achievement significantly improved for all participants as compared to judgments without any of the automation’s information. Participants provided with additional display content pertaining to cue variability in the task environment had significantly higher aided judgment achievement compared to those provided with only the automation’s judgment of a probability of conflict. When designing information analysis automation for environments where the automation’s judgment achievement is impacted by noisy environmental data, it may be beneficial to show additional task environment information to the human judge in order to improve judgment performance.

Characterizing the range of simulated prostate abnormalities palpable by digital rectal examination

BACKGROUND Although the digital rectal exam (DRE) is a common method of screening for prostate cancer and other abnormalities, the limits of ability to perform this hands-on exam are unknown. Perceptible limits are a function of the size, depth, and hardness of abnormalities within a given prostate stiffness. METHODS To better understand the perceptible limits of the DRE, we conducted a psychophysical study with 18 participants using a custom-built apparatus to simulate prostate tissue and abnormalities of varying size, depth, and hardness. Utilizing a modified version of the psychophysical method of constant stimuli, we uncovered thresholds of absolute detection and variance in ability between examiners. RESULTS Within silicone-elastomers that mimic normal prostate tissue (21kPa), abnormalities of 4mm diameter (20mm(3) volume) and greater were consistently detectable (above 75% of the time) but only at a depth of 5mm. Abnormalities located in simulated tissue of greater stiffness (82kPa) had to be twice that volume (5mm diameter, 40mm(3) volume) to be detectable at the same rate. CONCLUSIONS This study finds that the size and depth of abnormalities most influence detectability, while the relative stiffness between abnormalities and substrate also affects detectability for some size/depth combinations. While limits identified here are obtained for idealized substrates, this work is useful for informing the development of training and allowing clinicians to set expectations on performance.

SUPPORTING PHYSICIANS' PRACTICE-BASED LEARNING AND IMPROVEMENT (PBLI) AND QUALITY IMPROVEMENT THROUGH EXPLORATION OF POPULATION-BASED MEDICAL DATA.

Participating in self-assessment activities may stimulate improvement in practice behaviors. However, it is unclear how best to support the development of self-assessment skills, particularly in the health care domain. Exploration of population-based data is one method to enable health care providers to identify deficiencies in overall practice behavior that can motivate quality improvement initiatives. At the University of Virginia, we are developing a decision support tool to integrate and present population-based patient data to health care providers related to both clinical outcomes and non-clinical measures (e.g., demographic information). By enabling users to separate their direct impact on clinical outcomes from other factors out of their control, we may enhance the self-assessment process.

Psychophysical detection of inclusions with the bare finger amidst softness differentials

Softness discrimination and the detection of inclusions are important in surgery and other medical tasks. To better understand how the characteristics of an inclusion (size, depth, hardness) and substrate (stiffness) affect their tactile detection and discrimination with the bare finger, we conducted a psychophysics experiment with eighteen participants. The results indicate that within a more pliant substrate (21 kPa), inclusions of 4 mm diameter (20 mm3 volume) and greater were consistently detectable (above 75% of the time) but only at a depth of 5 mm. Inclusions embedded in stiffer substrates (82 kPa) had to be twice that volume (5 mm diameter, 40 mm3 volume) to be detectable at the same rate. To analyze which tactile cues most impact stimulus detectability, we utilized logistic regression and generalized estimating equations. The results indicate that substrate stiffness most contributes to inclusion detectability, while the size, depth, and hardness of the stimulus follow in individual importance, respectively. The results seek to aid in the development of clinical tools and information displays and more accurate virtual haptic environments in discrimination of soft tissue.

Presenting Information in Simulated Real-Time to Support Part-Task Weather Scenarios

Weather-related judgment and decision-making are critical in many domains. Weather assessments typically involve integrating information from multiple sources with varying content, format, and update rate. One way to study weather-related decision-making is to have participants engage in job related tasks using simulated real-time weather scenarios. This paper discusses a part-task scenario support tool that has been developed to facilitate the process of presenting weather information in simulated real-time. The paper also presents an illustrative example where the tool supported a study of weather decision-making by emergency managers.

Effect of Pooled Comparative Information on Judgments of Quality

Quality assessment is the focus of many healthcare initiatives. Yet, it is not well understood how the type of information used in decision support tools to enable judgments of quality based on data impacts the accuracy, consistency, and reliability of judgments made by physicians. Comparative pooled information could allow physicians to judge the quality of their practice by making comparisons with other practices or other specific populations of patients. In this study, resident physicians were provided with varying types of information derived from pooled patient datasets: quality component measures at the individual and group level, a qualitative interpretation of the quality measures using percentile rank, and an aggregate composite quality score. Thirty-two participants viewed 30 quality profiles consisting of information applicable to the practice of 30 deidentified resident physicians. Those provided with quality component measures and a qualitative interpretation of the quality measures (rankings) judged quality of care more similarly to experts and were more internally consistent compared with participants who were provided with quality component measures alone. Reliability between participants was significantly less for those who were provided with a composite quality score compared with those who were not.

Identifying patients at increased cancer risk: Validation of the Health Heritage risk assessment and decision support tool.

277 Background: Family history is critical to assess risk for cancer and inherited cancer syndromes. National Comprehensive Cancer Network (NCCN) guidelines are available to aid in identification and management of at-risk patients, but the guidelines are complex and ever-changing, hindering optimal use. This leads to under-recognition and mismanagement of high-risk patients. To address this need, we have developed and have fully implemented Health Heritage (HH), a web-based risk assessment and decision support tool. HH combines clinical data automatically extracted from a patients EHR with information entered by patients and shared between family members to provide personalized risk assessment and recommendations related to common cancers. This study aims to validate HHs ability to properly identify patients who meet NCCN guidelines for a genetics evaluation and to stratify cancer risk. METHODS We performed a retrospective chart review of 100 patients seen at an adult genetics clinic in 2012. Relevant personal and family medical history and genetic test results were entered into HH. Independent of the HH assessment, each patients information was reviewed by a genetic counselor to assess fulfillment of NCCN guidelines for a genetics evaluation. A subset of records was reviewed to compare cancer risk stratification between HH and a medical geneticist. RESULTS 87 patients met NCCN guidelines for a genetics evaluation for breast/ovarian cancer, 2 for colorectal cancer, and 3 for multiple cancers; 8 patients did not meet guidelines. HH had a sensitivity of 98% and specificity of 88% in identifying patients who met these guidelines. For patients at increased risk for breast, ovarian, or colorectal cancer as determined by a medical geneticist, HH agreed 83%, 86%, and 75% of the time, respectively. Discordances were the result of both complex clinical situations better handled by the geneticist and also HHs strict adherence to incorporating all information. CONCLUSIONS Health Heritage is a highly sensitive tool for identifying at-risk patients appropriate for a genetics evaluation and its risk stratification is complementary in high risk settings requiring management recommendations.

An Analysis of University Rank and the Inclusion of Sexual Orientation and Gender Identity Nondiscrimination Policy Statements

This analysis investigates the association between nondiscrimination policy statements related to sexual orientation and gender identity and university rank. The nondiscriminatory statements included in the analysis are sexual orientation, gender identity and/or expression, marital status, parental or familial status, and pregnancy. All but one university included sexual orientation in the policy. Only 26% included gender identity/expression. In all cases, the percentage of top 25 universities including each individual statement exceeded the percentage of those ranked below top 25. In addition, the top ranked universities included more statements in their policies than the lower ranked universities