Clare Harries

Predictors of diagnostic accuracy and safe management in difficult diagnostic problems in family medicine.

Objective. To investigate the role of information gathering and clinical experience on the diagnosis and management of difficult diagnostic problems in family medicine. Method. Seven diagnostic scenarios including 1 to 4 predetermined features of difficulty were constructed and presented on a computer to 84 physicians: 21 residents in family medicine, 21 family physicians with 1 to 3 y in practice, and 42 family physicians with ≥10 y in practice. Following the Active Information Search process tracing approach, participants were initially presented with a patient description and presenting complaint and were subsequently able to request further information to diagnose and manage the patient. Evidence-based scoring criteria for information gathering, diagnosis, and management were derived from the literature and a separate study of expert opinion. Results. Rates of misdiagnosis were in accordance with the number of features of difficulty. Seventy-eight percent of incorrect diagnoses were followed by inappropriate management and 92% of correct diagnoses by appropriate management. Number of critical cues requested (cues diagnostic of any relevant differential diagnoses in a scenario) was a significant predictor of accuracy in 6 scenarios: 1 additional critical cue increased the odds of obtaining the correct diagnosis by between 1.3 (95% confidence interval [CI], 1.0—1.8) and 7.5 (95% CI, 3.2— 17.7), depending on the scenario. No effect of experience was detected on either diagnostic accuracy or management. Residents requested significantly more cues than experienced family physicians did. Conclusions. Supporting the gathering of critical information has the potential to improve the diagnosis and management of difficult problems in family medicine.

Participant recruitment methods and statistical reasoning performance

Optimal Bayesian reasoning performance has reportedly been elusive, and a variety of explanations have been suggested for this situation. In a series of experiments, it is demonstrated that these difficulties with replication can be accounted for by differences in participant-sampling methodologies. Specifically, the best performances are obtained with students from top-tier, national universities who were paid for their participation. Performance drops significantly as these conditions are altered regarding inducements (e.g., using unpaid participants) or participant source (e.g., using participants from a second-tier, regional university). Honours-programme undergraduates do better than regular undergraduates within the same university, paid participation creates superior performance, and top-tier university students do better than students from lower ranked universities. Pictorial representations (supplementing problem text) usually have a slight facilitative effect across these participant manipulations. These results indicate that studies should take account of these methodological details and focus more on relative levels of performance rather than absolute performance.

Making decisions about benefits and harms of medicines

Even when good scientific data are available, peoples interpretation of risks and benefits will differ

Managing the risk of suicide in acute psychiatric inpatients: A clinical judgement analysis of staff predictions of imminent suicide risk

Background: Predicting suicide risk in psychiatric in-patients in order to inform risk management decisions is compromised by the poor predictive validity of the available models. Aims: This study explored the factors influencing judgements regarding suicide risk in psychiatrists and nurses working in acute psychiatric in-patient units in Scotland. Method: Clinical judgement analysis. Information used by 12 psychiatrists and 52 nurses to make judgements about suicide risk were analysed over 130 hypothetical cases. Correlations and linear regression analysis were used to examine judgement consistency and information use. Results: There was agreement between clinicians on the relative but not absolute degree of risk of each patient case. Consistency of judgments was low, particularly amongst nurses. All clinicians rated those with more previous suicide attempts, men, those with shorter admission times, and those who were less compliant and not improving clinically as at greater risk of suicide. Conclusions: Clinicians use cues that have been associated with suicide in traditional predictive models based on epidemiological studies and short term factors that may be particularly relevant to acute psychiatric settings. The inconsistencies observed can be interpreted to cast doubt on the validity of predictions of risk for imminent suicide and the role of such predictions in the assessment process. Declaration of interest: None.

Are absolute frequencies, relative frequencies, or both effective in reducing cognitive biases?

Biases in probabilistic reasoning are affected by alterations in the presentation of judgment tasks. In our experiments, students made likelihood judgments that an event was produced by various causes. These judgments were made in terms of probability, relative frequency or absolute frequency on a full or a pruned list of causes. When they had little personal experience of the event (causes of death), the pruning bias was smaller with relative frequencies than with absolute frequencies or probabilities. When they had more personal experience of the event (missing a lecture), the bias was less with both types of frequency than with probability but still lowest with relative frequency. We suggest that likelihood information is usually stored as relative frequencies when it has been obtained from public sources but that it is based on event counts when it is derived from personal experience. Copyright (C) 2000 John Wiley & Sons, Ltd.

Taking advice, using information and knowing what you are doing.

Subjective descriptions of judgment policies have been found to be imperfect. This could be because subjective weights are obtained on just a single occasion after all judgments have been completed. It could also be because people have tended to state their perception of an ideal way of responding rather than their perception of how they actually responded. Finally, it could be because they experience difficulty in relating variation in stimulus dimensions to variation in quite different response dimensions. In our task, people made sales forecasts on the basis of four pieces of information. They also stated the weight they placed on each one and the weight they should have placed on it. The means of weights stated on each trial were more appropriate than those stated at the end of all trials. Stated actual weights were very similar to stated ideal weights. Weights were more appropriate when forecasts and cues varied along the same dimension than when they did not. Thus, our results are consistent with the view that all three factors affect peoples ability to provide subjective descriptions of their judgment policies.

On the descriptive validity and prescriptive utility of fast and frugal models

Simple heuristics and regression models make different assumptions about behaviour. Both the environment and judgment can be described as fast and frugal. We do not know whether humans are successful when being fast and frugal. We must assess both global accuracy and the costs of Type I and II errors. These may be “smart heuristics that make researchers look simple.”

Using advice and assessing its usefulness

Advisors vary in quality. People should make more use of better advisors: they should weight their advice more heavily. They should also assess them as providing more useful advice: they should express greater confidence in their advice by estimating that it has a higher probability of being correct. We discuss whether someone who is good at using advice will be good at assessing it (or vice versa). Performance in these tasks may be dissociated because they depend on different underlying cognitive processes. This issue of whether there is a dissociation between use of advice and assessment of its usefulness has implications for the development of automated systems designed to provide users with expertise and decision support. We review three areas of research relevant to the relation between use of advice and assessment of its usefulness. Then we summarize findings of Harvey, Harries and Fischer (1998) indicating that people are better at assessing the usefulness of advice than at using it. Implications for systems development are discussed.