David A. Schum

Analysis of Evidence: Table of legislation and rules

1. Evidence and inference: some food for thought 2. Fact investigation and the nature of evidence 3. Principles of proof 4. Methods of analysis 5. The chart method 6. Outlines, chronologies and narrative 7. Analysing the decided case: anatomy of a cause celebre 8. Evaluating evidence 9. Probabilities, weight and probative force 10. Necessary but dangerous: generalizations and stories in argumentation about facts 11. The principles of proof and the law of evidence 12. The trial lawyers standpoint.

Analysis of Evidence: Frontmatter

1. Evidence and inference: some food for thought 2. Fact investigation and the nature of evidence 3. Principles of proof 4. Methods of analysis 5. The chart method 6. Outlines, chronologies and narrative 7. Analysing the decided case: anatomy of a cause celebre 8. Evaluating evidence 9. Probabilities, weight and probative force 10. Necessary but dangerous: generalizations and stories in argumentation about facts 11. The principles of proof and the law of evidence 12. The trial lawyers standpoint.

Evidence marshaling for imaginative fact investigation

For the past ten years Peter Tillers and I have been investigating the process of discovery and the imaginative reasoning it involves. Though our work has been performed within the context of law, we believe it applies in other contexts as well. One basic premise upon which our work rests is: How well we marshal or organize our existing thoughts and evidence influences how well we are able to generate new ideas in the form of hypotheses, new evidential tests of all hypotheses being considered, and defensible arguments linking our evidence and hypotheses. Existing thoughts and evidence can be marshaled, combined, or juxtaposed in various ways to meet different requirements that arise as the process of discovery unfolds over time. The work being described rests on other discovery-related studies in a variety of disciplines. As expected, we obtain different perspectives on discovery from different disciplines. One reason is that elements of the process of discovery are situation-dependent. In some contexts we may already have in existence an extensive base of information and wish to see what this information reveals. In other contexts, such as in fact investigation in law, we must often begin an episode of discovery with little or no base of relevant information. Further, in some situations we may have existing collections of data that may be analyzed statistically in various ways. But in other situations, such as in law, we usually encounter singular, unique, or one-of-a kind events for which no meaningful statistical analyses are possible. What does seem to be common in discovery in different situations is the need to examine different combinations of existing information. Different combinations or juxtapositions of existing information may, in different ways, suggest new ideas and new avenues of inquiry. In this paper I describe a prototype system that allows a person to juxtapose thoughts and evidence in different ways, each of which is helpful in suggesting new ideas, new evidence to gather, and new questions to ask.

Species of Abductive Reasoning in Fact Investigation in Law

Imaginative reasoning is as vital in law as it is in any other discipline. During fact investigation, hypotheses, in the form of possible charges or complaints, must be generated or discovered, as well as evidence bearing on these hypotheses. During the later process of proof, arguments in defense of the relevance, credibility, and probative force of offered evidence on hypotheses must also be generated. In no context known to me are hypotheses, evidence, and arguments linking them supplied at the outset for investigators and attorneys. These ingredients must be generated by imaginative or creative thinking. How we are able to generate new ideas has been an object of study for millennia. In spite of this, our imaginative and creative reasoning abilities are not well understood. There is considerable debate about the forms of reasoning that take place as we generate new ideas and evidential tests of them. This Article concerns a form of reasoning called “abduction,” which was suggested over a century ago by the American philosopher Charles S. Peirce as a reasoning mechanism underlying imaginative and creative thought. Most of us hear about two forms of reasoning: (1) deduction, showing that something is necessarily true, and (2) induction,showing that something is probably true. There is reason to believe that new ideas, in the form of hypotheses, may not be generated by induction or deduction. In this Article I will suggest that there are several species of abductive reasoning by which we show that something is possibly or plausibly true. I relate these species of abduction to intellectual tasks performed by investigators during fact investigation and to those performed by advocates during the process of proof. Consideration of these species of abductive reasoning exposes the richness of this important discovery-related activity.

Analyzing Evidence and Its Chain of Custody: A Mixed-Initiative Computational Approach

Intelligence analysts encounter a wide variety of items of evidence provided by an array of different sources. Some of these sources are human assets or informants; other sources are sensing devices of various kinds. Of great concern is the extent to which the events revealed in these evidence items can be believed. There is always the possibility that information we receive has been deliberately contrived to mislead us. A human informant may have any number of reasons for fabricating evidence in order to deceive us. It is also possible that our human sources or sensing devices are simply erroneous in their observations. Whether by deliberate fabrications or observational mistakes there is always the possibility of our being misled in the conclusions we draw from intelligence evidence. ------------------------------------

Intelligence Analysis as Agent-Assisted Discovery of Evidence, Hypotheses and Arguments

This paper presents a computational approach to intelligence analysis which is viewed as mixed-initiative discovery of evidence, hypotheses and arguments by an intelligence analyst and a cognitive assistant. The approach is illustrated with the analysis of wide area motion imagery of fixed geographic locations where the goal is to discover threat events such as an ambush or a rocket launch. This example is used to show how the Disciple cognitive assistants developed in the Learning Agents Center can help the analysts in coping with the astonishing complexity of intelligence analysis.

Computational Approach and Cognitive Assistant for Evidence-Based Reasoning in Intelligence Analysis

This paper presents a computational approach to intelligence analysis and its current implementation into a cognitive assistant called Disciple-CD. Intelligence analysis is viewed as ceaseless discovery of evidence, hypotheses, and arguments in a non-stationary world, involving cooperative processes of evidence in search of hypotheses, hypotheses in search of evidence, and evidentiary tests of hypotheses. Disciple-CD helps intelligence analysts formulate hypotheses, develop arguments that reduce complex hypotheses to simpler and simpler ones, collect evidence to evaluate the simplest hypotheses, assess the relevance, believability, and inferential force of evidence, and finally the likeliness of the hypotheses.

How Learning Enables Intelligence Analysts to Rapidly Develop Practical Cognitive Assistants

This paper overviews an end-to-end learning-based approach to the rapid development of practical cognitive assistants for intelligence analysis. A learning agent shell has been trained by a knowledge engineer with general evidence-based reasoning knowledge for intelligence analysis. This agent is further trained by an expert analyst how to analyze complex hypotheses from a given intelligence analysis domain. The resulting cognitive assistant is used by a typical analyst to rapidly analyze hypotheses from agents area of expertise. During its use, the agent continues to learn reasoning patterns from its user. This approach has been implemented and practical agents have been developed and used. This is a significant application of machine learning to agents development in intelligence analysis that can be generalized to many other domains involving evidence-based reasoning, including medicine, law, and science.

Toward cognitive assistants for complex decision making under uncertainty

Discussed in this paper is a quite unique and novel intelligence decision technology resting upon three systems we have called Disciple-LTA Learning, Teaching and Assistance, TIACRITIS Training Intelligence Analysts Critical Reasoning Skills, and Disciple-CD Connecting the Dots. We have so far applied these systems to complex intelligence inferences based on masses of evidence of many different kinds and coming from many different sources. This paper discusses the extension of these systems to be valuable decision support assistants that are capable of helping analysts to answer the two fundamental questions regarding decisions made in the face of uncertainty: whats at stake?, and what are the odds? The stakes question concerns the value or utility of decision consequences and the odds question concerns the probability of these possible consequences. We discuss the requisite ingredients of defensible and persuasive decisions and problems associated with the discovery of these ingredients in a world that keeps changing all the time. But we also consider the constraints facing intelligence analysts who so often have limited time for decisions and who also have deficiencies regarding the availability of information supporting requisite value and probability judgments. Conventional approaches to decision analysis are usually not helpful in the face of these constraints. We offer simplified methods for assessing both value and probability judgments and a simplified method for combining these judgments in the selection of a course of action that does take account of the requisites for defensible and persuasive decision and analysis. In the process, we illustrate our methods with a very complex analysis involving the possible proliferation of nuclear weapons.

On the Marshalling of Evidence and the Structuring of Argument

This paper concerns issues that arise when arguments are constructed from a mass of evidence and when the evidence is weighed or assessed in the process of drawing conclusions from it. Of major concern in this paper are problems associated with the productive integration of structural and weight-related analyses. In addition, the author addresses some common misconceptions about theories of evidential reasoning and attempts to set the record straight by mentioning the legacy of research on the evidential foundations of probabilistic inference that has been available for many years. All of these matters should be of concern in attempts to provide a productive integration of research in Operations Research and Artificial intelligence in situations in which people must draw conclusions from a mass of evidence that is incomplete, inconclusive, and that comes from sources with every gradation of credibility.