Robert T. Caldwell

Long Radiology Workdays Reduce Detection and Accommodation Accuracy

PURPOSE The aim of this study was to measure the diagnostic accuracy of fracture detection, visual accommodation, reading time, and subjective ratings of fatigue and visual strain before and after a day of clinical reading. METHODS Forty attending radiologists and radiology residents viewed 60 deidentified, HIPAA-compliant bone examinations, half with fractures, once before any clinical reading (early) and once after a day of clinical reading (late). Reading time was recorded. Visual accommodation (the ability to maintain focus) was measured before and after each reading session. Subjective ratings of symptoms of fatigue and oculomotor strain were collected. The study was approved by local institutional review boards. RESULTS Diagnostic accuracy was reduced significantly after a day of clinical reading, with average areas under the receiver operating characteristic curves of 0.885 for early reading and 0.852 for late reading (P < .05). After a day of image interpretation, visual accommodation was no more variable, though error in visual accommodation was greater (P < .01), and subjective ratings of fatigue were higher. CONCLUSIONS After a day of clinical reading, radiologists have reduced ability to focus, increased symptoms of fatigue and oculomotor strain, and reduced ability to detect fractures. Radiologists need to be aware of the effects of fatigue on diagnostic accuracy and take steps to mitigate these effects.

Gaze Dwell Times on Acute Trauma Injuries Missed Because of Satisfaction of Search

RATIONALE AND OBJECTIVES The authors performed this study to determine whether satisfaction of search (SOS) errors in patients with multiple traumas are caused by faulty visual scanning, faulty recognition, or faulty decision making. MATERIALS AND METHODS A series of radiographs were obtained in patients with multiple traumas. Radiologists interpreted each series under two experimental conditions: when the first radiograph in the series included a fracture, and when it did not. In the first experiment, the initial radiographs showed nondisplaced fractures of the extremities (minor fractures); in the second experiment, the initial radiographs showed abnormalities of greater clinical importance (major fractures). Each series also included a radiograph with a subtle (test) fracture and a normal radiograph on which detection accuracy was measured. In each experiment, gaze dwell time was recorded as 10 radiologists reviewed images from 10 simulated cases of multiple trauma. RESULTS An SOS effect could be demonstrated only in the second experiment. Analysis of dwell times showed that search on subsequent radiographs was shortened when the initial radiograph contained a fracture; however, the errors were not based on faulty scanning. CONCLUSION The SOS effect in musculoskeletal trauma is not caused by faulty scanning. Demonstration of an SOS effect on test fractures with major but not minor additional fractures suggests that detection of other fractures is inversely related to the severity of the detected fracture.

Do Long Radiology Workdays Affect Nodule Detection in Dynamic CT Interpretation

PURPOSE A previous study demonstrated decreased diagnostic accuracy for finding fractures and decreased ability to focus on skeletal radiographs after a long working day. Skeletal radiographic examinations commonly have images that are displayed statically. The aim of this study was to investigate whether diagnostic accuracy for detecting pulmonary nodules on CT of the chest displayed dynamically would be similarly affected by fatigue. METHODS Twenty-two radiologists and 22 residents were given 2 tests searching CT chest sequences for a solitary pulmonary nodule before and after a day of clinical reading. To measure search time, 10 lung CT sequences, each containing 20 consecutive sections and a single nodule, were inspected using free search and navigation. To measure diagnostic accuracy, 100 CT sequences, each with 20 sections and half with nodules, were displayed at preset scrolling speed and duration. Accuracy was measured using receiver operating characteristic curve analysis. Visual strain was measured via dark vergence, an indicator of the ability to keep the eyes focused on the display. RESULTS Diagnostic accuracy was reduced after a day of clinical reading (P = .0246), but search time was not affected (P > .05). After a day of reading, dark vergence was significantly larger and more variable (P = .0098), reflecting higher levels of visual strain, and subjective ratings of fatigue were also higher. CONCLUSIONS After their usual workday, radiologists experience increased fatigue and decreased diagnostic accuracy for detecting pulmonary nodules on CT. Effects of fatigue may be mitigated by active interaction with the display.

Role of faulty decision making in the satisfaction of search effect in chest radiography.

RATIONALE AND OBJECTIVES The authors performed this study to determine whether defective pattern recognition or defective decision making is more to blame for satisfaction of search (SOS) errors in chest radiography. MATERIALS AND METHODS Fifty-eight chest radiographs-half of which demonstrated diverse, native abnormalities-were read by 20 observers. The radiographs were read twice, once with and once without the addition of a simulated pulmonary nodule. Observers provided a verbal account of their focus of attention, indicating suspicious features and regions considered during their inspection of the radiograph. Observers also provided a separate account of the abnormalities they would include in a radiologic report. RESULTS When the authors considered only those reports that did not refer to the simulated nodules, they found no reduction in the area under the proper receiver operating characteristic (ROC) curves in cases that contained nodules. A smaller SOS effect, however, was demonstrated with analysis of events in which the native abnormality was missed in one condition but not the other. Verbal protocols suggested that the SOS errors were mainly caused by recognition failure rather than faulty decision making. CONCLUSION Describing their focus of attention may have prompted observers to inspect the radiographs in a more deliberate, systematic way, thus reducing the SOS effect. More residual SOS errors were caused by defective pattern recognition than by faulty decision making.

Cause of satisfaction of search effects in contrast studies of the abdomen

RATIONALE AND OBJECTIVES Extraintestinal abnormalities visible without contrast material on abdominal radiographs are reported less frequently when contrast examinations are performed. Gaze dwell time was used to determine whether this difference is due to failure by observers to scan plain-film regions of contrast studies or discounting of plain-film abnormalities that were actually scanned. METHODS Patients were included whose contrast studies had elicited the largest reductions in positive responses compared with their plain-film studies in a previous detection experiment. Gaze of 10 radiologists was studied. RESULTS Significantly less time was spent gazing at non-contrast regions of contrast examinations than at the corresponding regions of radiographs. Errors with radiographs were based primarily on failures of recognition and decision making, whereas errors with contrast studies were based primarily on faulty scanning. CONCLUSION Satisfaction of search errors on contrast examinations are caused by reduction in scanning of noncontrast regions.

Proper ROC analysis and joint ROC analysis of the satisfaction of search effect in chest radiology

RATIONALE AND OBJECTIVES The purpose of this study was to test whether the satisfaction of search (SOS) effect in chest radiology could be demonstrated with proper receiver operating characteristic (ROC) curves and with joint detection and localization ROC curves. MATERIAL AND METHODS Data from an earlier ROC study of SOS in chest radiology were analyzed with three proper ROC models and one ROC model for joint detection and localization. Fits of the models were compared on the basis of likelihood-ratio chi-squared statistics (G2). To examine further the validity of the SOS effect in chest radiology, the authors also replicated the earlier study with a new sample of readers, analyzing the new data with the same methods. RESULTS The proper contaminated binormal model fit the data better than the other two proper ROC models. Contaminated binormal analysis of the earlier and the replication experiment demonstrated an SOS effect: a reduction in area under the ROC curve for detection of the native abnormalities with the addition of nodules. Similarly, joint ROC analysis producing curves that appropriately cross the chance line gave similar results. CONCLUSION Preventing inappropriate chance line crossing reduces measurement error and provides more powerful statistical tests. Results of both experiments showed that the SOS effect in chest radiology can be demonstrated with ROC methods that avoid inappropriate crossing of the chance line.

Developmental stage-specific antigens in the nervous system of the cockroach

A specific effort was made to obtain monoclonal antibodies that bind to macromolecules that play a role in the development of the nervous system. It was considered that good candidates for such molecules were those that were only transiently present in the embryonic nervous system. Hybridomas were prepared from spleen cells taken from mice that had been immunized with nerve cords from cockroach embryos at the 43-50% stage of development. The hybridoma supernatants were screened for antibody binding to frozen sections of both embryonic and adult thoracic ganglia. Cell lines that produced monoclonal antibodies that transiently bound to the embryonic nervous system were saved and cloned. These developmental stage-specific monoclonal antibodies either did not bind to the adult nervous system or bound to it with a pattern very different from that in the embryonic nervous system. The developmental stage-specific antigens detected by these monoclonal antibodies were organized into four categories based on the part of the embryonic nervous system in which they were transiently localized. These include binding to the cell bodies of all neurons, cell bodies of subsets of neurons or neuroblasts, subsets of axons, and the neuropile. Preliminary biochemical characterization of the antigens showed that many of these antibodies were recognizing carbohydrate epitopes. Functions for these antigens, most of which are components of the cell surface, are tentatively proposed.

Satisfaction of Search from Detection of Pulmonary Nodules in Computed Tomography of the Chest

RATIONALE AND OBJECTIVES We tested whether satisfaction of search (SOS) effects that occur in computed tomography (CT) examination of the chest on detection of native abnormalities are produced by the addition of simulated pulmonary nodules. MATERIALS AND METHODS Two experiments were conducted. In the first experiment, 70 CT examinations, half that demonstrated diverse, subtle abnormalities and half that demonstrated no native lesions, were read by 18 radiology residents and fellows under two experimental conditions: presented with and without pulmonary nodules. In a second experiment, many of the examinations were replaced to include more salient native abnormalities. This set was read by 14 additional radiology residents and fellows. In both experiments, detection of the natural abnormalities was studied. Receiver operating characteristic (ROC) curve areas for each reader-treatment combination were estimated using empirical and proper ROC models. Additional analyses focused on decision thresholds and visual search time on abnormality-free CT slice ranges. Institutional review board approval and informed consent from 32 participants were obtained. RESULTS Observers more often missed diverse native abnormalities when pulmonary nodules were added, but also made fewer false-positive responses. There was no change in ROC area, but decision criteria grew more conservative. The SOS effect on decision thresholds was accompanied by a reduction in search time on abnormality-free CT slice ranges. CONCLUSION The SOS effect in CT examination of the chest is similar to that found in contrast examination of the abdomen, involving induced visual neglect.

WorkstationJ: workstation emulation software for medical image perception and technology evaluation research

We developed image presentation software that mimics the functionality available in the clinic, but also records time-stamped, observer-display interactions and is readily deployable on diverse workstations making it possible to collect comparable observer data at multiple sites. Commercial image presentation software for clinical use has limited application for research on image perception, ergonomics, computer-aids and informatics because it does not collect observer responses, or other information on observer-display interactions, in real time. It is also very difficult to collect observer data from multiple institutions unless the same commercial software is available at different sites. Our software not only records observer reports of abnormalities and their locations, but also inspection time until report, inspection time for each computed radiograph and for each slice of tomographic studies, window/level, and magnification settings used by the observer. The software is a modified version of the open source ImageJ software available from the National Institutes of Health. Our software involves changes to the base code and extensive new plugin code. Our free software is currently capable of displaying computed tomography and computed radiography images. The software is packaged as Java class files and can be used on Windows, Linux, or Mac systems. By deploying our software together with experiment-specific script files that administer experimental procedures and image file handling, multi-institutional studies can be conducted that increase reader and/or case sample sizes or add experimental conditions.

Lectin receptors in the cockroach neuromuscular system. I. Distribution among the set of coxal depressor muscles

Fluorescent derivatives of plant lectins have been used to determine if there are biochemical differences among the cell surfaces of 6 muscles in the leg of the cockroach innervated in a fixed pattern by two identified motor neurons, Df or Ds. Histochemical analysis of the binding of fluorescein isothiocyanate conjugates of the lectins to frozen sections of muscles has demonstrated that the surfaces of muscles innervated by motor neuron Ds have more alpha-N-acetylgalactosamine and/or D-galactose than do those of muscles innervated by motor neuron Df. Biochemical analysis of the glycoprotein lectin receptors by sodium dodecylsulphate polyacrylamide gel electrophoresis has shown that approximately 10% of them are distributed among the various muscles in a manner that correlates with innervation by Df or Ds. Some of these macromolecules may be responsible for the biochemical differences in the muscle cell surfaces that could be specifically recognized by motor neurons. This intercellular recognition could mediate the reformation of the original innervation pattern during axonal regeneration.