Andrey Povyakalo

How to Discriminate between Computer-Aided and Computer-Hindered Decisions: A Case Study in Mammography

Background. Computer aids can affect decisions in complex ways, potentially even making them worse; common assessment methods may miss these effects. We developed a method for estimating the quality of decisions, as well as how computer aids affect it, and applied it to computer-aided detection (CAD) of cancer, reanalyzing data from a published study where 50 professionals (“readers”) interpreted 180 mammograms, both with and without computer support. Method. We used stepwise regression to estimate how CAD affected the probability of a reader making a correct screening decision on a patient with cancer (sensitivity), thereby taking into account the effects of the difficulty of the cancer (proportion of readers who missed it) and the reader’s discriminating ability (Youden’s determinant). Using regression estimates, we obtained thresholds for classifying a posteriori the cases (by difficulty) and the readers (by discriminating ability). Results. Use of CAD was associated with a 0.016 increase in sensitivity (95% confidence interval [CI], 0.003–0.028) for the 44 least discriminating radiologists for 45 relatively easy, mostly CAD-detected cancers. However, for the 6 most discriminating radiologists, with CAD, sensitivity decreased by 0.145 (95% CI, 0.034–0.257) for the 15 relatively difficult cancers. Conclusions. Our exploratory analysis method reveals unexpected effects. It indicates that, despite the original study detecting no significant average effect, CAD helped the less discriminating readers but hindered the more discriminating readers. Such differential effects, although subtle, may be clinically significant and important for improving both computer algorithms and protocols for their use. They should be assessed when evaluating CAD and similar warning systems.

Human-machine diversity in the use of computerised advisory systems: a case study

Computer-based advisory systems form with their users composite, human-machine systems. Redundancy and diversity between the human and the machine are often important for the dependability of such systems. We discuss the modelling approach we applied in a case study. The goal is to assess failure probabilities for the analysis of X-ray films for detecting cancer, performed by a person assisted by a computer-based tool. Differently from most approaches to human reliability assessment, we focus on the effects of failure diversity — or correlation — between humans and machines. We illustrate some of the modelling and prediction problems, especially those caused by the presence of the human component. We show two alternative models, with their pros and cons, and illustrate, via numerical examples and analytically, some interesting and non-intuitive answers to questions about reliability assessment and design choices for human-computer systems.

CAD in mammography: lesion-level versus case-level analysis of the effects of prompts on human decisions

ObjectTo understand decision processes in CAD-supported breast screening by analysing how prompts affect readers’ judgements of individual mammographic features (lesions). To this end we analysed hitherto unexamined details of reports completed by mammogram readers in an earlier evaluation of a CAD tool.Material and methodsAssessments of lesions were extracted from 5,839 reports for 59 cancer cases. Statistical analyses of these data focused on what features readers considered when recalling a cancer case and how readers reacted to CAD prompts.ResultsAbout 13.5% of recall decisions were found to be caused by responses to features other than those indicating actual cancer. Effects of CAD: lesions were more likely to be examined if prompted; the presence of a prompt on a cancer increased the probability of both detection and recall especially for less accurate readers in subtler cases; lack of prompts made cancer features less likely to be detected; false prompts made non-cancer features more likely to be classified as cancer.ConclusionThe apparent lack of impact reported for CAD in some studies is plausibly due to CAD systematically affecting readers’ identification of individual features, in a beneficial way for certain combinations of readers and features and a damaging way for others. Mammogram readers do not ignore prompts. Methodologically, assessing CAD by numbers of recalled cancer cases may be misleading.

Conservative Reasoning about the Probability of Failure on Demand of a 1-out-of-2 Software-Based System in Which One Channel Is "Possibly Perfect"

In earlier work, [11] (henceforth LR), an analysis was presented of a 1-out-of-2 software-based system in which one channel was “possibly perfect”. It was shown that, at the aleatory level, the system pfd (probability of failure on demand) could be bounded above by the product of the pfd of channel A and the pnp (probability of nonperfection) of channel B. This result was presented as a way of avoiding the well-known difficulty that for two certainly-fallible channels, failures of the two will be dependent, i.e., the system pfd cannot be expressed simply as a product of the channel pfds. A price paid in this new approach for avoiding the issue of failure dependence is that the result is conservative. Furthermore, a complete analysis requires that account be taken of epistemic uncertainty-here concerning the numeric values of the two parameters pfdA and pnpB. Unfortunately this introduces a different difficult problem of dependence: estimating the dependence between an assessors beliefs about the parameters. The work reported here avoids this problem by obtaining results that require only an assessors marginal beliefs about the individual channels, i.e., they do not require knowledge of the dependence between these beliefs. The price paid is further conservatism in the results.

Modeling the probability of failure on demand (pfd) of a 1-out-of-2 system in which one channel is “quasi-perfect”

Our earlier work proposed ways of overcoming some of the difficulties of lack of independence in reliability modeling of 1-out-of-2 software-based systems. Firstly, it is well known that aleatory independence between the failures of two channels A and B cannot be assumed, so system pfd is not a simple product of channel pfds. However, it has been shown that the probability of system failure can be bounded conservatively by a simple product of pfdA and pnpB (probability not perfect) in those special cases where channel B is sufficiently simple to be possibly perfect. Whilst this “solves” the problem of aleatory dependence, the issue of epistemic dependence remains: An assessor’s beliefs about unknown pfdA and pnpB will not have them independent. Recent work has partially overcome this problem by requiring only marginal beliefs – at the price of further conservatism. Here we generalize these results. Instead of “perfection” we introduce the notion of “quasi-perfection”: a small pfd practically equivalent to perfection (e.g. yielding very small chance of failure in the entire life of a fleet of systems). We present a conservative argument supporting claims about system pfd. We propose further work, e.g. to conduct “what if?” calculations to understand exactly how conservative our approach might be in practice, and suggest further simplifications.

Comparison of Empirical Data from Two Honeynets and a Distributed Honeypot Network

In this paper we present empirical results and speculative analysis based on observations collected over a two month period from studies with two high interaction honeynets, deployed in a corporate and an SME (small to medium enterprise) environment, and a distributed honeypots deployment. All three networks contain a mixture of Windows and Linux hosts. We detail the architecture of the deployment and results of comparing the observations from the three environments. We analyze in detail the times between attacks on different hosts, operating systems, networks or geographical location. Even though results from honeynet deployments are reported often in the literature, this paper provides novel results analyzing traffic from three different types of networks and some initial exploratory models. This research aims to contribute to endeavours in the wider security research community to build methods, grounded on strong empirical work, for assessment of the robustness of computer-based systems in hostile environments.

Does incorrect computer prompting affect human decision making? A case study in mammography

Abstract The goal of the data collection and analyses described in this paper was to investigate the effects of incorrect output from a CAD tool on the reliability of the decisions of its human users. Our work follows on a clinical trial that evaluated the impact of introducing a computerised prompting tool (R2 ImageChecker) as part of the breast screening programme in the UK. Our goal was to use data obtained in this trial to feed into probabilistic models (similar to those used in “reliability engineering”) which would allow us to find and assess possible ways of improving the interaction between the automated tool and its human user. A crucial requirement for this modelling approach is estimating the probability that a human will fail in his/her task when the output of the automated tool is incorrect. The data obtained from the clinical trial was not sufficient to inform this aspect of our probabilistic model. Therefore, we conducted a follow-up study to elucidate further the effects of computer failure on human performance. Preliminary analyses of the data resulting from the follow-up study are reported and discussed.

Diversity, Safety and Security in Embedded Systems: Modelling Adversary Effort and Supply Chain Risks

We present quantitative considerations for the design of redundancy and diversity in embedded systems with security requirements. The potential for malicious activity against these systems have complicated requirements and design choices. New design trade-offs have arisen besides those already familiar in this area: for instance, adding redundancy may increase the attack surface of a system and thus increase overall risk. Our case study concerns protecting redundant communications between a control system and its controlled physical system. We study the effects of using: (i) different encryption keys on replicated channels, and (ii) diverse encryption schemes and implementations. We consider two attack scenarios, with adversaries having access to (i) ways of reducing the search space in attacks using random searches for keys; or (ii) hidden major flaws in some crypto algorithm or implementation. Trade-offs between the requirements of integrity and confidentiality are found, but not in all cases. Simple models give useful design insights. In this system, we find that key diversity improves integrity without impairing confidentiality - no trade-offs arise between the two - and it can substantially increase adversary effort, but it will not remedy substantial weaknesses of the crypto system. Implementation diversity does involve design trade-offs between integrity and confidentiality, which we analyse, but turns out to be generally desirable for highly critical applications of the control system considered.

A conservative bound for the probability of failure of a 1-out-of-2 protection system with one hardware-only and one software-based protection train

Redundancy and diversity have long been used as means to obtain high reliability in critical systems. While it is easy to show that, say, a 1-out-of-2 diverse system will be more reliable than each of its two individual “trains”, assessing the actual reliability of such systems can be difficult because the trains cannot be assumed to fail independently. If we cannot claim independence of train failures, the computation of system reliability is difficult, because we would need to know the probability of failure on demand (pfd) for every possible demand. These are unlikely to be known in the case of software. Claims for software often concern its marginalpfd, i.e. average across all possible demands. In this paper we consider the case of a 1-out-of-2 safety protection system in which one train contains software (and hardware), and the other train contains only hardware equipment. We show that a useful upper (i.e. conservative) bound can be obtained for the system pfd using only the unconditional pfd for software together with information about the variation of hardware failure probability across demands, which is likely to be known or estimatable. The worst-case result is obtained by “allocating” software failure probability among demand “classes” so as to maximize system pfd.

Deriving a frequentist conservative confidence bound for probability of failure per demand for systems with different operational and test profiles

Reliability testing is typically used in demand-based systems (such as protection systems) to derive a confidence bound for a specific operational profile. To be realistic, the number of tests for each class of demand should be proportional to the demand frequency of the class. In practice, however, the actual operational profile may differ from that used during testing. This paper provides a means for estimating the confidence bound when the test profile differs from the profile used in actual operation. Based on this analysis the paper examines what bound can be claimed for different types of profile uncertainty and options for dealing with this uncertainty. We also show that the same conservative bound estimation equations can be applied to cases where different measures of software test coverage and operational profile are used.