Colleen M. Vrbin

Errors in Thyroid Gland Fine-Needle Aspiration

Scant published data exist on redesigning pathology practice based on error data. In this first step of an Agency for Healthcare Research and Quality patient safety project, we measured the performance metrics of thyroid gland fine-needle aspiration, performed root cause analysis to determine the causes of error, and proposed error-reduction initiatives to address specific errors. Eleven cytologists signed out 1,543 thyroid gland aspirates in 2 years, and surgical pathology follow-up was obtained in 364 patients. Of the 364 patients, 91 (25.0%) had a false-negative diagnosis and 36 (9.9%) a false-positive diagnosis. Root cause analysis showed that major sources of error were pre-analytic (poor specimen quality) and analytic (interpretation of unsatisfactory specimens as nonneoplastic and lack of diagnostic category standardization). We currently are evaluating the effectiveness of error reduction initiatives that target pre-analytic and analytic portions of the diagnostic pathway.

Urine cytology discrepancies : Frequency, causes, and outcomes

Although urine cytology is used for the early detection and surveillance of urothelial carcinoma, there has been little study of the frequency, causes, and outcomes of cytologic-histologic discrepancies. We obtained histologic follow-up in 361 (6.2%) of 5,785 voided, 124 (19.5%) of 636 lower tract instrumented, and 23 (33%) of 69 upper tract urinary cytologic specimens from 1 institution during a 2-year timeframe to determine diagnostic discrepancy frequency and outcomes. Cytologic-histologic discrepancies were observed in 208 (40.9%) cases with histologic followup, and the cause of discrepancy was interpretation and sampling in 35.1% and 63.0%, respectively. Of all discrepancies, 101 (48.6%) resulted in minimal or mild harm, consisting mainly of repeated testing and/or diagnostic delays. Severe harm never was observed. We conclude that current screening and surveillance methods that incorporate urine cytology are accurate in diagnosing urothelial cancer. However, the current protocols result in potentially reducible errors that lead to unnecessary testing and diagnostic delays.

Effectiveness of Toyota Process Redesign in Reducing Thyroid Gland Fine-Needle Aspiration Error

Our objective was to determine whether the Toyota Production System process redesign resulted in diagnostic error reduction for patients who underwent cytologic evaluation of thyroid nodules. In this longitudinal, nonconcurrent cohort study, we compared the diagnostic error frequency of a thyroid aspiration service before and after implementation of error reduction initiatives consisting of adoption of a standardized diagnostic terminology scheme and an immediate interpretation service. A total of 2,424 patients underwent aspiration. Following terminology standardization, the false-negative rate decreased from 41.8% to 19.1% (P = .006), the specimen nondiagnostic rate increased from 5.8% to 19.8% (P < .001), and the sensitivity increased from 70.2% to 90.6% (P < .001). Cases with an immediate interpretation had a lower noninterpretable specimen rate than those without immediate interpretation (P < .001). Toyota process change led to significantly fewer diagnostic errors for patients who underwent thyroid fine-needle aspiration.

Amended reports: development and validation of a taxonomy of defects.

Amended pathology reports produce rework, confusion, and distrust. To develop a reproducible amendment taxonomy we derived a classification from 141 amended reports, then validated it with 130 new cases before 4 observers independently reviewed 430 cases measuring agreement (k). Next, agreement in classifying 30 other amended reports in 7 institutions was measured. We further tracked amendment rates, defect categories, defect discoverers, and discovery mechanisms. In the 430-case validation set agreement was excellent (k = 0.8780 [range, 0.8416-0.9144]). Among the 7 institutions, agreement was good (k = 0.6235 [range, 0.3105-0.8975]). Amendment rates ranged from 2.6 to 4.8 per 1,000 reports. Misinterpretation fractions varied least (23%-29%). Misidentification fractions ranged more widely (20%-38%). Specimen defects were least frequent (4%-10%) and report defects most frequent (29%-48%). Misidentifications and report defects inversely correlated. Pathologists discovered most misinterpretations, and clinicians found most misidentifications. Conference review revealed 40% to 80% of misinterpretations. This taxonomy produced excellent reproducibility and good agreement across institutions.

Frequency and outcome of cervical cancer prevention failures in the United States.

We measured the frequency and outcome of cervical cancer prevention failures that occurred in the Papanicolaou (Pap) and colposcopy testing phases involving 1,646,580 Pap tests in 4 American hospital systems between January 1, 1998, and December 31, 2004. We defined a screening failure as a 2-step or greater discordant Pap test result and follow-up biopsy diagnosis. A total of 5,278 failures were detected (0.321% of all Pap tests); 48% and 52% of failures occurred in the Pap test and colposcopy phases, respectively. Missed squamous cancers (1 in 187,786 Pap tests), glandular cancers (1 in 19,426 Pap tests), and high-grade lesions (1 in 6,870 Pap tests) constituted 4.1% of all failures. Unnecessary repeated tests or diagnostic delays occurred in 70.8% and 63.9% of failures involving high- and low-grade lesions, respectively. We conclude that cervical cancer prevention practices are remarkably successful in preventing squamous cancers, although a high frequency of failures results in low-impact negative outcomes.

Double slide viewing as a cytology quality improvement initiative.

Few studies have measured the effect of pre-sign out double viewing of cytology cases as a means to decrease error. Three Agency for Healthcare Research and Quality-funded project sites performed pre-sign out double viewing of 431 pulmonary cytology cases. Two-step or more differences in diagnosis were arbitrated as interpretive errors, and the effect of double viewing was measured by comparing the frequency of cytologic-histologic correlation-detected errors in the previous 2 years with the double-viewing period. The number of interpretive errors detected by double viewing for the 3 institutions was 2.7%, 0% and 1.9%, respectively. Double viewing did not lower the frequency of cytologic-histologic correlation false-negative errors. We conclude that double viewing detects errors in up to 1 of every 37 cases and that biases in the double-viewing process limit error detection.

Interobserver variability in human papillomavirus test results in cervicovaginal cytologic specimens interpreted as atypical squamous cells.

We studied interobserver variability in the proportions of human papillomavirus (HPV)-positive results for atypical squamous cells of undetermined significance (ASCUS) and atypical squamous cells, cannot exclude high-grade squamous intraepithelial lesion (ASC-H) diagnoses among 5 pathologists from the same laboratory during a 2-year period. These proportions were compared with individual pathologist’s ASCUS/squamous intraepithelial lesion (SIL) ratios. Of 1,299 ASCUS diagnoses, 32.3% had HPV testing; 49.4% were HPV+. Positive findings by individual pathologists ranged from 38% to 67% ( P = .057). There was a difference in the proportions of high-risk HPV results for individual pathologists ( P < .001). For the pathologist who diagnosed 38% (23/61) of samples as HPV+, the ASCUS/SIL was 0.58; the pathologist who diagnosed 67% (28/42) as HPV+ had a ratio of 1.02. Of the ASC-H diagnoses, 32.9% were tested for HPV; 63% (46/73) were positive. Although the HPV+ proportion by pathologist ranged from 54% to 83%, no significant differences were identified. Within the same laboratory, interobserver variability exists in the proportions of HPV positivity for ASCUS and ASC-H interpretations.

The Use of Standardized Patients in the Training and Evaluation of Physician Assistant Students

Purpose: Standardized patients (SPs) are used extensively in physician and nursing education, where they have proven to be valuable educational tools. Practice with SPs may provide durable benefits regarding communication skills. However, the extent to which SPs are used in physician assistant (PA) training is unknown. The primary aim of this study was to determine the extent to which programs use SPs and whether SP utilization varied by the programs affiliation with an academic center or medical school. Methods: Self‐administered surveys were sent to one academic coordinator at each of the then 134 accredited PA programs. Results: Ninety‐six programs returned surveys. At programs represented by survey participants, SPs play various roles, including providing medical histories and participating in complaint‐specific and organ‐specific “sensitive” examinations (rectal, pelvic, etc). Programs housed in medical schools were more likely to use professional patients. Respondents from programs not using SPs provided reasons why they were not used. Most common among these were issues related to cost, access, training, and recruitment. To a lesser extent, liability issues were also cited, as was a perceived lack of evidence that SPs add value to PA education. Details about SP compensation were provided by only 48 (50%) respondents, but reflected a broad range of remuneration. Conclusions: The majority of accredited PA training programs use SPs. Although they play a range of educational roles, they are most commonly used for the most sensitive components of physical examination. Use of SP patients was particularly common in programs affiliated with medical schools and community colleges.

Endoscopic ultrasound-guided FNA and ProCore biopsy in sampling pancreatic and intra-abdominal masses.

ProCore fine‐needle biopsy (FNB) was introduced to improve the diagnostic yield of endoscopic ultrasound (EUS)–guided fine‐needle aspiration (FNA) sampling. The aim of this study was to evaluate EUS‐guided sampling of intra‐abdominal masses and compare the diagnostic utility of conventional EUS‐FNA and ProCore FNB.

Dynamic simulation of medical diagnosis: learning in the medical decision making and learning environment MEDIC

MEDIC is a dynamic decision making simulation incorporating time constraints, multiple and delayed feedback and repeated decisions. This tool was developed to study cognition and dynamic decision making in medical diagnosis. MEDIC allows one to study several crucial facets of complex medical decision making while also being well controlled for experimental purposes. Using MEDIC, there is a correct diagnosis for the patient, which provides both outcome and process measures of good performance. MEDIC also allows us to calculate cue diagnosticity and probability functions over the set of hypotheses that participants are explicitly considering, based on assumptions of local (bounded) rationality. MEDIC has served in a series of studies aimed at understanding learning in dynamic and real-time medical diagnotic situations. In this paper, we outline the tool and highlight results from these preliminary studies which set out to measure learning.