Lawrence A. Cooperstein

Effect of observer instruction on ROC study of chest images

Receiver-operating characteristic (ROC) analysis has been used in many medical imaging applications during the past decade. In order to ensure that reader-confidence ratings are analyzable (well distributed to meet convergence requirements of curve-fitting algorithms) and meaningful (limit extrapolation of the data), many investigators train readers specifically for this purpose. No experimental data are available concerning the possible effects of such training on the results of ROC studies. We performed a multi-observer, multi-disease study in which 300 chest images were rated by four radiologists before and after they were trained to provide well-distributed confidence ratings. The results indicate that for our data set, reader and disease-specific accuracy was not significantly affected by the training process for interstitial disease and pneumothoraces. However, the accuracy of two readers was significantly affected for the detection of nodules (P less than 0.05), and the overall accuracy of one reader was significantly affected for the classification of normal versus abnormal images (P less than 0.01). Thus, in spite of the difficulties associated with the performance of ROC studies in a free-reading environment, one should carefully consider the possible effects of any intervention on the results prior to conducting ROC studies.

The effect of clinical history on chest radiograph interpretations in a PACS environment.

The effect that accompanying patient information has on diagnostic accuracy in radiologic interpretation has been studied by many researchers but remains a matter of contention. Experiments in the past emphasized the chest film because this procedure is the one done most frequently in radiology. However, with the increasing importance of digital imaging, the role of patient history in these procedures should be assessed. The use of a model computerized patient history in the interpretation of digitized chest images that were displayed on a high-resolution workstation was studied. Two hundred forty-seven selected posteroanterior chest images that indicated disease and that indicated no disease were interpreted in random sequence by five board-certified radiologists, with and without accompanying patient histories. Readers were prompted by the response forms to evaluate images for the possible occurrence of interstitial diseases, nodules, or pneumothorax. No significant differences (P = .05) in the detection of these abnormalities were noted between case readings with and without history for any of the radiologists or for the group as a whole. However, this methodology reflects a direct interpretation approach. The results of this study may not necessarily be applicable to individual prompts, different imaging procedures, or other abnormalities.

Radiologists' Perceptions Of And Preferences For A Concise Computerized History Form During Radiologic Interpretations

The purpose of this study was to develop and design a model computerized patient history form that could be used by practicing radiologists during the x-ray reading and interpretation process. We conducted a multi-observer study to determine 1) whether such a form is warranted and can be a feasible part of the reading process; and 2) what the component parts of the form and their order of appearance in the form should be. Twenty board-certified radiologists were tested in their areas of subspecialty; each radiologist read a series of 10 cases, half with and half without clinical histories. Results indicated that all those tested would use such a history if it were provided to them along with the radiographs to be read; that they prefer such a history to be brief but detailed enough to provide more information than the typical requisition does; that they prefer a rapidly accessible history; and that there are preferences about what information appears and the order in which it appears.