Paul E. Lehner

Cognitive biases and time stress in team decision making

This experiment investigates the impart of time stress on the decision making performance of command and control teams. Two person teams were trained to execute a set of simple decision procedures. Some of these procedures required subjects to make judgments that were inconsistent with normal heuristic decision processing. The principal hypothesis was that these decision procedures would be vulnerable-to-bias, and would therefore be more vulnerable to the effects of time stress than other decision procedures. The results support this hypothesis. In addition, the results suggest that the subjects adapted inappropriately to time stress. As time stress increased, they began to use a decision processing strategy that was less effective than the strategy they were trained to use.

Confirmation Bias in Complex Analyses

Most research works investigating the confirmation bias has used abstract experimental tasks where participants drew inferences from just a few items of evidence. The experiment reported in this paper investigated the confirmation bias in a complex analysis task that is more characteristic of law enforcement investigations, financial analysis, and intelligence analysis. Participants were professionals, half of whom had intelligence analysis experience. The effectiveness of a procedure designed to mitigate the confirmation bias, called analysis of competing hypotheses (ACH), was tested. Results showed a confirmation bias for both experience groups, but ACH significantly reduced bias only for participants without intelligence analysis experience. Confirmation bias manifested as a weighting bias, not as an interpretation bias. Participants tended to agree on the interpretation of evidence (i.e., whose hypothesis was supported by the evidence) but tended to disagree on the importance of the evidence-giving more weight to the evidence that supported their preferred hypothesis and less weight to evidence that disconfirmed it.

Decision-Centric Information Monitoring

We are interested in information management for decision support applications, especially those that monitor distributed, heterogeneous databases to assess time-critical decisions. Users of such applications can easily be overwhelmed with data that may change rapidly, may conflict, and may be redundant. Developers are faced with a dilemma: either filter out most information and risk excluding critical items, or gather possibly irrelevant or redundant information, and overwhelm the decision maker. This paper describes a solution to this dilemma called decision-centric information monitoring (DCIM). First, we observe that decision support systems should monitor only information that can potentially change some decision. We present an architecture for DCIM that meets the requirements implied by this observation. We describe techniques for identifying the highest value information to monitor and techniques for monitoring that information despite autonomy, distribution, and heterogeneity of data sources. Finally, we present lessons learned from building LOOKOUT, which is to our knowledge the first implementation of a top-to-bottom system performing decision-centric information monitoring.

Using Hypergames to Increase Planned Payoff and Reduce Risk

Hypergames are an emerging tool for combining decision theoretic and game theoretic information. We explain a new approach to the standard two-player zero-sum single-stage normal game that maximizes expected gain while quantifying possible loss. A DTGT agent can use this formulation to select a plan based on its assessment of an opponents intent, its assessment of an opponents unpredictability and its utility model of the situation.

Confirmation Bias in the Analysis of Remote Sensing Data

Analysis of remote sensing data requires a mix of technical data analysis and expert judgment. Although there is considerable empirical evidence that expert judgments reflect substantial biases, the impact of judgment biases in remote sensing and similar types of technical data analyses has not been investigated. In particular, judgment research suggests that experts are prone to a confirmation bias-where focus on a proposed hypothesis leads the expert to seek and overweigh confirming versus disconfirming evidence. In technical data analysis, this predicts a tendency toward false positives in interpretation-concluding that sensor data support a hypothesis when they do not. In this paper, we empirically examine confirmation bias in technical data analysis of remote sensing data, along with an approach to mitigating this bias that systematically promotes consideration of alternative causes in the analysis. Results suggest that analysts do exhibit confirmation bias in their technical data analysis of remote sensing data, and furthermore, that structured consideration of alternative causes mitigates this bias.

Multipurpose strategic planning in the game of Go

A heuristic method for adversarial planning is developed to address the problem of multipurpose planning in the game of Go. Static analysis and dynamic look ahead on both strategic and tactical levels are used to generate possible goals and to identify interactions among the achievability of various goals. Strategic, multipurpose goals are composed of sets of interacting goals.

An application of document filtering in an operational system

This paper describes an applied document filtering system embedded in an operational watch center that monitors disease outbreaks worldwide. At the initial time of this writing, the system effectively supported monitoring of 23 geographic regions by filtering documents in several thousand daily news sources in 11 different languages. This paper describes the filtering algorithm, statistical procedures for estimating Precision and Recall in an operational environment, summarizes operational performance data and suggests lessons learned for other applications of document filtering technology. Overall, these results are interpreted as supporting the general utility of document filtering and information retrieval technology and offers recommendations for future applications of this technology.