John W. Williamson

Overutilization of Acute-Care Beds in Veterans Affairs Hospitals

The authors tested the hypothesis that the Department of Veterans Affairs (VA) hospitals would have substantial overutilization of acute care beds and services because of policies that emphasize inpatient care over ambulatory care. Reviewers from 24 randomly selected VA hospitals applied the InterQual ISD* (Intensity, Severity, Discharge) criteria for appropriateness concurrently to a random sample of 2,432 admissions to acute medical, surgical, and psychiatry services. Reliability of hospital reviewers in applying the ISD* criteria was tested by comparing their reviews with those of a small group of expert reviewers. Validity of the ISD* criteria was tested by comparing the assessments of master reviewers with the implicit judgments of panels of nine physicians. The physician panels validated the ISD* admission criteria for medicine and surgery (74% agreement with master reviewers, kappa > 0.4), whereas the psychiatry criteria were not validated (66% agreement, kappa 0.29). Hospital reviewers reliably used all three criteria sets (> 83% agreement with master reviewers, kappa > 0.6). Rates of nonacute admissions to acute medical and surgical services were > 38% as determined by the hospital and master reviewers and by the physician panels. Nonacute rates of continued stay were > 32% for both medicine and surgery services. Similar rates of nonacute admissions and continued stay were found for all 24 hospitals. Reasons for nonacute admissions and continued stay included lack of an ambulatory care alternative, conservative physician practices, delays in discharge planning, and social factors such as homelessness and long travel distances to the hospital. Using criteria that the authors showed to be reliable and valid, substantial overutilization of acute medicine and surgical beds was found in a representative sample of VA hospitals. Correcting this situation will require changes in physician practice patterns, development of ambulatory care alternatives to inpatient care, and modification of current VA policies determining eligibility for care.

Physician Performance and Its Effects on Patients: A Classification Based on Reports by Internists, Surgeons, Pediatricians, and Obstetricians

&NA; Nine thousand one hundred fifteen summary reports of patient care episodes were obtained from 2,342 practicing specialists in internal medicine, surgery, pediatrics, and obstetrics‐gynecology. Each report described an episode of care in which directly observed or verifiable physician actions had clearly beneficial or detrimental effects on a particular patient. These reported “critical” physician actions and the patient end results attributed to these actions were inductively classified. These classifications contribute to the development of more systematic and objective criteria of effective physician performance. Certain findings in this study suggest that in specialty practice effective and ineffective physician performance may be evenly distributed; neither is more frequent in any particular age or sex grouping of patients, nor in disorders affecting any particular body systems.

Statistical methods for reliability and validity testing: an application to nominal group judgments in health care.

: We discuss statistical methods to analyze data for reliability (agreement among or between raters) and validity. Such analysis are frequently required for medical care data. We also discuss the application of these statistical methods to a study of the reliability and validity of nominal group estimation procedures to identify cost-effective topics for quality assurance purposes.We discuss statistical methods to analyze data for reliability (agreement among or between raters) and validity. Such analyses are frequently required for medical care data. We also discuss the application of these statistical methods to a study of the reliability and validity of nominal group estimation procedures to identify cost-effective topics for quality assurance purposes.

Priority setting in quality assurance: reliability of staff judgments in medical institutions.

A structured procedure using the judgments of a representative group of local providers for establishing priorities for quality assurance activity in diverse medical institutions was tested for reliability. Two independent matched teams of physicians, nurses, administrators, and other staff in eight separate medical facilities generated 320 topics which encompassed areas where quality assurance efforts would have either considerable or little impact in terms of improving health outcomes within reasonable costs. Concordance of judgment between teams in each facility was determined by analyzing the similarity of topics content, the agreement in scaling the health impact of similar topics generated by both teams independently, and the agreement by one team in scaling the health impact of topics generated by the other team. The findings revealed 44 per cent content agreement on topics independently generated, 93 per cent agreement on dichotomous scaling of similar topics, and 87 per cent agreement on five-point scaling of similar topics. Concordance of judgment by one team in scaling the other teams topics was highly significant (p<.001). Preliminary analysis of topic content and scaling agreement among different facilities indicated low agreement both on the content areas and on the health impact of similar topics. It is concluded that the judgments of local providers in identifying cost-effective quality assurance priorities is highly reliable in the medical institutions studied.

Validity of medical staff judgments in establishing quality assurance priorities.

The validation of a structured group judgment procedure to establish priorities for quality assurance activities was undertaken in six medical institutions in the United States. Validation focused on the extent to which health improvement could be documented by outcome-based projects focusing on priority topics. Predictive (criterion-related) validity was sought by analyzing five successively more stringent levels of evidence: 1) feasibility of implementing a quality assessment project within the topic areas selected by the priority procedure; 2) accuracy of identifying health deficiencies or strengths; 3) verification of establishing correctable causes of health deficiencies; 4) capability of effecting significant improvement of health deficiencies; and 5) credibility of evidence that improvement achieved is directly attributable to corrective actions taken. Within the limits of generalizability, predictive validity was documented at every level of analysis. It is recommended that the structured group judgment process used in this study be more thoroughly evaluated and considered for quality assurance planning purposes.

Primary Care, Quality Improvement, and Health Systems Change:

In this article, we have examined evidence that supports inferences regarding the critical role of pri mary care in determining the ultimate quality and success of our health care system as a whole. It seems clear that this nation has a critical need to correct the rapidly eroding structure and impact of our primary care system. We have suggested several strategies necessary to improve the United States primary care system. We need to correct the numeric imbalance of generalists to specialists by enhancing motivation to enter this career field. To enhance primary care as a career choice, we must increase the status, contribu tion, and rewards of primary practitioners. Finally, to facilitate future growth and success, we must en hance the scholarly base of knowledge in this field, particularly as related to quality improvement tech nology and managed health care.