Alex Davis

Communicating scientific uncertainty

All science has uncertainty. Unless that uncertainty is communicated effectively, decision makers may put too much or too little faith in it. The information that needs to be communicated depends on the decisions that people face. Are they (i) looking for a signal (e.g., whether to evacuate before a hurricane), (ii) choosing among fixed options (e.g., which medical treatment is best), or (iii) learning to create options (e.g., how to regulate nanotechnology)? We examine these three classes of decisions in terms of how to characterize, assess, and convey the uncertainties relevant to each. We then offer a protocol for summarizing the many possible sources of uncertainty in standard terms, designed to impose a minimal burden on scientists, while gradually educating those whose decisions depend on their work. Its goals are better decisions, better science, and better support for science.

Quantifying Phishing Susceptibility for Detection and Behavior Decisions

Objective: We use signal detection theory to measure vulnerability to phishing attacks, including variation in performance across task conditions. Background: Phishing attacks are difficult to prevent with technology alone, as long as technology is operated by people. Those responsible for managing security risks must understand user decision making in order to create and evaluate potential solutions. Method: Using a scenario-based online task, we performed two experiments comparing performance on two tasks: detection, deciding whether an e-mail is phishing, and behavior, deciding what to do with an e-mail. In Experiment 1, we manipulated the order of the tasks and notification of the phishing base rate. In Experiment 2, we varied which task participants performed. Results: In both experiments, despite exhibiting cautious behavior, participants’ limited detection ability left them vulnerable to phishing attacks. Greater sensitivity was positively correlated with confidence. Greater willingness to treat e-mails as legitimate was negatively correlated with perceived consequences from their actions and positively correlated with confidence. These patterns were robust across experimental conditions. Conclusion: Phishing-related decisions are sensitive to individuals’ detection ability, response bias, confidence, and perception of consequences. Performance differs when people evaluate messages or respond to them but not when their task varies in other ways. Application: Based on these results, potential interventions include providing users with feedback on their abilities and information about the consequences of phishing, perhaps targeting those with the worst performance. Signal detection methods offer system operators quantitative assessments of the impacts of interventions and their residual vulnerability.

Generosity Across Contexts

Extensive research in economics explores generosity in monetary allocations. However, generosity often involves the allocation of non-monetary goods or experiences. Existing evidence suggests that generosity may be higher in such contexts, though no direct comparison exists. Here, we compare generosity in decisions that vary whether allocations are monetary or non-monetary. In two experiments, generosity is significantly higher in non-monetary contexts. Thus, the typical monetary laboratory dictator game may underestimate generosity in many non-laboratory contexts where allocations are non-monetary. We find weaker relationships between individuals’ allocation decisions across monetary and non-monetary contexts than for allocations that hold constant the monetary nature of the context.

Communicating Uncertain Experimental Evidence.

Four experiments examined when laypeople attribute unexpected experimental outcomes to error, in foresight and in hindsight, along with their judgments of whether the data should be published. Participants read vignettes describing hypothetical experiments, along with the result of the initial observation, considered as either a possibility (foresight) or a reality (hindsight). Experiment 1 found that the initial observation seemed more likely to be replicated when viewed in hindsight than in foresight. The remaining experiments contrasted responses to an initial observation from 1 of the 4 studies that was either expected or unexpected (based on the predictions of participants in Experiment 1). Experiments 2A-C and Experiment 3 found that unexpected results were more likely to be attributed to methodological problems than were expected ones-but to the same degree in foresight and in hindsight. Participants in Experiment 4 had more confidence in an explanation for an unexpected outcome when it was mentioned before that outcome was revealed than when it was suggested only after the surprise was known. In all the experiments, most participants recommended collecting more data before publishing the results, especially when they attributed the results to multiple causes. The results suggest that considering the causes of unexpected experimental results in foresight may improve the evaluation and communication of those results in hindsight.

Vendor of choice and the effectiveness of policies to promote health information exchange

BackgroundAs more hospitals adopt Electronic Health Records (EHR), focus has shifted to how these records can be used to improve patient care. One barrier to this improvement is limited information exchange between providers. In this work we examine the role of EHR vendors, hypothesizing that vendors strategically control the exchange of clinical care summaries. Their strategy may involve the creation of networks that easily exchange information between providers with the same vendor but frustrate exchange between providers with different vendors, even as both Federal and State policies attempt to incentivize exchange through a common format.MethodsUsing data from the 2013 American Hospital Association’s Information Technology Supplement, we examine the relationship between a hospital’s decision to share clinical care summaries outside of their network and EHR vendor market share, measured by the percentage of hospitals that have the same vendor in a Hospital Referral Region.ResultsOur findings show that the likelihood of a hospital exchanging clinical summaries with hospitals outside its health system increases as the percentage of hospitals with the same EHR vendor in the region increases. The estimated odds of a hospital sharing clinical care summaries outside their system is 5.4 (95% CI, 3.29–8.80) times greater if all hospitals in the Hospital Referral Region use the same EHR Vendor than the corresponding odds for a hospital in an area with no hospitals using the same EHR Vendor. When reviewing the relationship of vendor market concentration at the state level we find a positive significant relationship with the percentage of hospitals that share clinical care summaries within a state. We find no significant impact from state policies designed to incentivize information exchange through the State Health Information Exchange Cooperative Program.ConclusionThere are benefits to exchanging using proprietary methods that are strengthened when the vendors are more concentrated. In order to avoid closed networks that foreclose some hospitals, it is important that future regulation attempt to be more inclusive of hospitals that do not use large vendors and are therefore unable to use proprietary methods for exchange.

An Approach to Reconciling Competing Ethical Principles in Aggregating Heterogeneous Health Preferences

Background. Health-related quality of life (HRQL) scores are used extensively to quantify the effectiveness of medical interventions. Societal preference-based HRQL scores aim to produce societal valuations of health by aggregating valuations from individuals in the general population, where each aggregation procedure embodies different ethical principles, as explained in social choice theory. Methods. Using the Health Utilities Index as an exemplar, we evaluate societal preference-based HRQL measures in the social choice theory framework. Results. We find that current preference aggregation procedures are typically justified in terms of social choice theory. However, by convention, they use only one of many possible aggregation procedures (the mean). Central to the choice of aggregation procedure is how to treat preference heterogeneity, which can affect analyses that rely on HRQL scores, such as cost-effectiveness analyses. We propose an analytical-deliberative framework for choosing one (or a set of) aggregation procedure(s) in a socially credible way, which we believe to be analytically sound and empirically tractable, but leave open the institutional mechanism needed to implement it. Conclusions. Socially acceptable decisions about aggregating heterogeneous preferences require eliciting stakeholders’ preferences among the set of analytically sound procedures, representing different ethical principles. We describe a framework for eliciting such preferences for the creation of HRQL scores, informed by social choice theory and behavioral decision research.

Are Preferences for Allocating Harm Rational

The allocation of nonmonetary harm is an important—yet understudied—domain of choice. Using a modified Dictator Game, we asked 27 participants to allocate a harmful event (time of putting their hand in ice water) between themselves and an anonymous stranger. We found substantially less coherent, and more egalitarian, preferences compared to other studies that ask participants to allocate monetary endowments. Specifically, 26% of participants made choices inconsistent with utility maximization, and 78% of participants behaved in an egalitarian manner. In comparable studies of monetary gains, only 2% were inconsistent and 30% egalitarian. The results suggest that the focus on monetary gains likely overestimates the rationality of other-regarding preferences and underestimates egalitarianism.

Expert-guided optimization for 3D printing of soft and liquid materials

Additive manufacturing (AM) has rapidly emerged as a disruptive technology to build mechanical parts, enabling increased design complexity, low-cost customization and an ever-increasing range of materials. Yet these capabilities have also created an immense challenge in optimizing the large number of process parameters in order achieve a high-performance part. This is especially true for AM of soft, deformable materials and for liquid-like resins that require experimental printing methods. Here, we developed an expert-guided optimization (EGO) strategy to provide structure in exploring and improving the 3D printing of liquid polydimethylsiloxane (PDMS) elastomer resin. EGO uses three steps, starting first with expert screening to select the parameter space, factors, and factor levels. Second is a hill-climbing algorithm to search the parameter space defined by the expert for the best set of parameters. Third is expert decision making to try new factors or a new parameter space to improve on the best current solution. We applied the algorithm to two calibration objects, a hollow cylinder and a five-sided hollow cube that were evaluated based on a multi-factor scoring system. The optimum print settings were then used to print complex PDMS and epoxy 3D objects, including a twisted vase, water drop, toe, and ear, at a level of detail and fidelity previously not obtained.