Can we trust the standardized mortality ratio? A formal analysis and evaluation based on axiomatic requirements
CCan we trust the standardized mortality ratio? A formalanalysis and evaluation based on axiomatic requirements
Martin Roessler Jochen Schmitt Olaf SchofferSeptember 9, 2020
AbstractBackground : The standardized mortality ratio (SMR) is often used to assess and comparehospital performance. While it has been recognized that hospitals may differ in their SMRsdue to differences in patient composition, there is a lack of rigorous analysis of this and other -largely unrecognized - properties of the SMR.
Methods : This paper proposes five axiomatic requirements for adequate standardized mor-tality measures: strict monotonicity (monotone relation to actual mortality rates), case-mixinsensitivity (independence of patient composition), scale insensitivity (independence of hospi-tal size), equivalence principle (equal rating of hospitals with equal actual mortality rates in allpatient groups), and dominance principle (better rating of unambiguously better performinghospitals). Given these axiomatic requirements, effects of variations in patient composition,hospital size, and actual and expected mortality rates on the SMR were examined using basicalgebra and calculus. In this regard, we distinguished between standardization using expectedmortality rates derived from a different dataset (external standardization) and standardizationbased on a dataset including the considered hospitals (internal standardization). The resultswere illustrated by hypothetical examples.
Results : Under external standardization, the SMR fulfills the axiomatic requirements of strictmonotonicity and scale insensitivity but violates the requirement of case-mix insensitivity, theequivalence principle, and the dominance principle. All axiomatic requirements not fulfilledunder external standardization are also not fulfilled under internal standardization. In addition,the SMR under internal standardization is scale sensitive and violates the axiomatic requirementof strict monotonicity.
Conclusions : The SMR fulfills only two (none) out of the five proposed axiomatic requirementsunder external (internal) standardization. Generally, the SMRs of hospitals are differentlyaffected by variations in case mix and actual and expected mortality rates unless the hospitalsare identical in these characteristics. These properties hamper valid assessment and comparisonof hospital performance based on the SMR. a r X i v : . [ s t a t . A P ] S e p ntroduction Assessing quality of care in hospitals is of high interest to patients, healthcare professionals, andpolitical decision makers. Consequently, there are multiple attempts to characterize and comparehospitals based on quality indicators [1, 2, 3, 4, 5]. The conceptualization of those indicators usuallyincludes some form of risk adjustment. Utilizing statistical methods and measures, risk adjustmentaims to facilitate comparison of hospitals with differences in case mix (e.g. different shares of high-risk patient groups) that induce outcome differences between the hospitals irrespectively of the truequality of care. Such adjustment is particularly relevant for quality indicators based on in-hospitalmortality, which is one of the most frequently considered hospital outcomes.A frequently used measure of risk-adjusted mortality is the standardized mortality ratio (SMR)[6, 7, 8, 9, 10, 11, 12]. Using indirect standardization, the SMR relates the observed mortality rateof a hospital to its expected mortality rate. The latter is derived by estimating expected mortalityrates for predefined strata of patients (i.e. patients with similar risk factors characteristics) andaggregating these stratum-specific expected mortality rates according to the hospital’s case mix(details on calculation of the SMR are provided below). In this way, the SMR aims to express actualmortality relative to a benchmark that contains information on the hospital’s patient compositionand, thus, on the distribution of risk factors for in-hospital mortality.While the SMR is the dominant measure in empirical applications, some studies highlightedproblems related to its estimation. SMR values were found to be sensitive to the choice of theestimation method [13], readmission rates [14], differences between hospitals with respect to codingquality [15], and correlation between quality of care and risk factors [16]. In some cases, changes inthe SMR over time were primarily driven by changes in expected rather than actual mortality rates[17]. Moreover, violations of the assumption of identical relationships between mortality and its riskfactors across all analyzed hospitals were shown to induce bias in the estimation of the SMR [18].In addition to estimation issues, there is evidence for more general methodological problemsrelated to the construction of the SMR. Notably, the SMR was shown to be case-mix sensitive,implying that two hospitals with identical mortality rates in all patient groups may differ in theirSMRs due to differences in patient composition [19]. This property is reflected in the results ofmultiple studies examining the impact of variations in patient composition on the SMRs of hospitals[19, 20, 21, 22, 23]. It is noteworthy that all of these studies draw on empirical and/or simulationapproaches. While those analyses can provide evidence on basic properties of the SMR, there is alack of rigorous, formal analysis. Moreover, properties of the SMR other than case-mix sensitivityhave rarely been examined.Against that background, we systematically investigated and evaluated basic properties of theSMR. In a first step, we proposed general properties that characterize adequate measures of stan-dardized mortality. In a second step, we utilized these proposed characteristics to derive a set ofaxiomatic requirements that should be fulfilled by standardized mortality measures. Formulationof axiomatic requirements for adequate statistical measures is long-established in literature on mea-surement of income inequality [24] and facilitates clear evaluation of the measures’ mathematicalproperties. In a third step, we examined properties of the SMR by drawing on analytical mathe-matical methods. This approach allowed us to formally investigate the behavior of the SMR givenvariations in case mix, hospital size, and actual and expected mortality rates. The insights onproperties of the SMR were evaluated with respect to the formulated axiomatic requirements forstandardized mortality measures. In this way, this paper clarifies and extends the results of previousanalyses by providing a comprehensive, systematic, and transparent examination and assessment ofthe SMR’s basic properties.
Methods
All formal analyses relied on basic algebra and differential calculus. In preparation of these anal-yses, the following sections outline the definition and the analyzed properties of the SMR and thenotational conventions used throughout this paper.
Definition and interpretation of the standardized mortality ratio
We considered H hospitals, indexed by h = 1 , . . . , H . Each patient treated in one of these hospitalsbelonged to one of S strata, indexed by s = 1 , . . . , S . Each stratum represents a group of patientswith the same risk factor characteristics. Let n hs ∈ N denote the number of patients belonging tostratum s treated in hospital h and n h = (cid:80) Ss =1 n hs denote the total number of patients treated1n hospital h . Note that we also refer to n h as a measure of hospital size. Furthermore, assumethat each hospital was characterized by actual stratum-specific mortality rates p hs ∈ [0 , expected stratum-specific mortality rates p es ∈ [0 , h is defined as the relationbetween its actual mortality rate ¯ p h = n − h (cid:80) Ss =1 n hs p hs and its expected mortality rate ¯ p eh = n − h (cid:80) Ss =1 n hs p es , i.e. SMR h := ¯ p h ¯ p eh = (cid:80) Ss =1 n hs p hs (cid:80) Ss =1 n hs p es . (1)Note that while actual mortality rates p hs may be specific for each stratum in a hospital, expectedmortality rates p es may only vary by stratum but are the same for all considered hospitals. Hence,the SMR may be interpreted as evaluating actual mortality rates of all hospitals relative to thesame “benchmark” (i.e. expected) mortality rates, where both actual and expected mortality ratesare weighted by each hospital’s stratum-specific patient numbers. If the SMR of a hospital exceedsthe value of 1, the hospital is judged to perform worse than expected. A SMR smaller than 1 isinterpreted as better-than-expected performance. The relative performance of hospitals is oftenassessed by comparison of their SMRs.In practice, the hospital-specific mortality rates p hs are unknown and may be estimated by thehospital’s observed stratum-specific mortality rates. Under this approach, the numerator of Eq 1becomes the hospital’s observed number of deaths while the denominator is the hospital’s expectednumber of deaths. However, since our analysis does not focus on issues of estimation but examinesgeneral properties of the SMR, we treat the actual mortality rates p hs as known (or perfectlyestimated) throughout the paper. Axiomatic requirements for standardized mortality measures
The objective of this paper is to evaluate properties of the SMR in a systematic way. While Eq 1provides the basis for formal analysis, evaluation of the SMR’s properties also requires generalassumptions on desirable properties of standardized mortality measures. Those properties shouldbe relevant for fair comparison of hospital performance in terms of mortality, which, by assumption,is influenced by the hospitals’ care qualities. For this purpose, we propose that a well-behavedmeasure of standardized mortality should be characterized by the following properties: • Increases (decreases) in actual mortality rates should, ceteris paribus, always be reflected inincreased (decreased) values of the standardized mortality measure.
Rationale : Keeping (all relevant) patient-specific risk factors constant, increasing (decreasing)mortality in patients treated in a hospital indicates worse (better) performance of the hospital. • The measure should be independent of the hospital’s patient composition.
Rationale : The hospital’s case mix does not reflect the hospital’s care quality and, thus, shouldnot influence the performance assessment. • The measure should be independent of hospital size.
Rationale : Hospital size per se does not reflect quality of care and, thus, should not influencethe performance assessment. • The measure should assign the same value to hospitals with identical performance in terms ofmortality.
Rationale : Fair comparison of hospital performance requires that hospitals with identical carequality may not be evaluated differently. • The measure should always rank one hospital better than another hospital if the former un-ambiguously performs better in terms of care-quality related mortality.
Rationale : Lower mortality rates of all patient groups in one hospital compared to anotherhospital imply that each patient’s risk of death is lower when being admitted in the formerhospital.Based on these necessary properties for valid comparisons of quality of care, we postulate thefollowing five axiomatic requirements for standardized mortality measures: • Strict monotonicity: Increases (decreases) in a hospital’s stratum-specific mortality rates p hs always induce increases (decreases) in the value of the measure assigned to the hospital if thehospital treated patients belonging to that stratum ( n hs > Case-mix insensitivity: Holding actual stratum-specific mortality rates p hs expected mortal-ity rates p es and the hospital’s number of patients n h constant, the value of the measureis insensitive to the hospital’s case mix, i.e. the hospital’s stratum-specific patient shares n hs /n h , s = 1 , . . . , S . • Scale insensitivity: Holding case mix ( n hs /n h , s = 1 , . . . , S ), actual mortality rates p hs , andexpected mortality rates p es constant, the measure is insensitive to the hospital’s total numberof patients n h . • Equivalence principle: The measure assigns the same value to two hospitals with identicalstratum-specific mortality rates p hs or identical deviations of actual stratum-specific mortalityrates p hs from expected stratum-specific mortality rates p es . • Dominance principle: The measure always ranks hospital 1 better than hospital 2 if the actualmortality rates of all patient groups treated in hospital 1 are equal to or lower than themortality rates of these patient groups in hospital 2 ( p s ≤ p s ∀ s = 1 , . . . , S ) and the mortalityrate of at least one patient group is lower in hospital 1 than in hospital 2 ( ∃ k ∈ { , ..., S } : p k < p k ).Given these axiomatic requirements, we examined effects of variations in case mix, hospital size n h , actual mortality rates p hs , and expected mortality rates p es on the SMR. In this regard, it isnoteworthy that there are two general ways in which expected mortality rates p es may be derived: • External standardization: Expected mortality rates may be derived from data that is not included in the analysis of the hospitals under consideration, e.g. from a dataset of hospitalsfrom a different geographical region. This approach is refereed to as external standardization. • Internal standardization: Alternatively, expected mortality rates may be derived from thesame dataset used to calculate the SMRs of the considered hospitals. In this case, the per-formance of the hospitals usually is evaluated against their average performance in terms ofmortality rates. This approach is referred to as internal standardization.Taking the difference between external and internal standardization into account, we examinedproperties of the SMR for both standardization approaches separately.
Notation
For notational brevity, arguments of functions are stated explicitly only when they are relevant forthe analysis. For instance, the SMR of a specific hospital h , which depends on stratum-specificnumbers of patients n h , . . . , n hS , the hospital’s stratum-specific mortality rates p h , . . . , p hS , andexpected mortality rates p e , . . . , p eS is simply written as SMR h , whereSMR h = SMR h ( n h , . . . , n hS , p h , . . . , p hS , p e , . . . , p eS ):= (cid:80) Ss =1 n hs p hs (cid:80) Ss =1 n hs p es . (2)Adding η ∈ N + patients to stratum s = k while holding all other parameters constant is expressedas SMR h ( n hk + η ), whereSMR h ( n hk + η ) = SMR h ( n h , . . . , n h,k − , n hk + η, n h,k +1 , . . . , n hS , p h , . . . , p hS , p e , . . . , p eS )= ( n hk + η ) p hk + (cid:80) s (cid:54) = k n hs p hs ( n hk + η ) p ek + (cid:80) s (cid:54) = k n hs p ek . (3)In the same way, the overall mortality rate of hospital h when multiplying all stratum-specific patientnumbers n hs , s = 1 , . . . , S with a common factor λ ∈ R + is written as ¯ p h ( λn h , . . . , λn hS ), where¯ p h ( λn h , . . . , λn hS ) = ¯ p h ( λn h , . . . , λn hS , p h , . . . , p hS )= (cid:80) Ss =1 ( λn hs ) p hs (cid:80) Ss =1 ( λn hs ) . (4)To distinguish in notation between external and internal standardization, variables that are af-fected by the choice of standardization approach are tagged with the superscripts “ext” and “int”,respectively. 3 esults In the following, effects of variations in case mix, hospital size, and actual and expected mortality rateare examined formally. Analyses were first conducted for the SMR under external standardizationand subsequently for the SMR under internal standardization.
External standardization
As noted above, external standardization refers to the case in which the stratum-specific expectedmortality rates are derived from a different dataset. Letting p e, ext s denote these expected mortalityrates, the externally standardized SMR of hospital h isSMR ext h := ¯ p h ¯ p e, ext h = (cid:80) Ss =1 n hs p hs (cid:80) Ss =1 n hs p e, ext s , (5)where ¯ p e, ext h = n − h (cid:80) Ss =1 n hs p e, ext s is the externally standardized expected mortality rate of hospi-tal h . Variations in case mix under external standardization
To analyze case-mix sensitivity, we examined effects of a change in a hospital’s number of patientsbelonging to specific strata on Eq 5 while holding the total number of patients treated in the hospitalconstant. Formally, we considered a shift of η ∈ N + patients from stratum s = l , to stratum s = k ,where n hl ≥ η . The SMR of hospital h thus becomesSMR ext h ( n hk + η, n hl − η ) = ( (cid:80) Ss =1 n hs p hs ) + η · ( p hk − p hl )( (cid:80) Ss =1 n hs p e, ext s ) + η · ( p e, ext k − p e, ext l ) . (6)It is noteworthy that Eq 6 implies that the SMR generally changes due to a shift of patients fromstratum l to stratum k even if the hospital’s mortality rates in both strata are equal to the expectedmortality rates ( p hk = p e, ext k , p hl = p e, ext l ) as long as p e, ext k (cid:54) = p e, ext l . Hence, performance in linewith expected mortality for both strata generally does not imply that the SMR is insensitive to thenumber of patients belonging to these strata. This result demonstrates the SMR’s violation of theaxiomatic requirement of case-mix insensitivity.For further investigation, the change in the SMR due to the shift in case mix is defined asΩ ext hkl ( η ) := SMR ext h ( n hk + η, n hl − η ) − SMR ext h ( n hk , n hl )= ( p hk − p hl ) − ( p e, ext k − p e, ext l ) · SMR ext h ( n hk , n hl ) η − n h ¯ p e, ext h ( n hk , n hl ) + ( p e, ext k − p e, ext l ) . (7)Eq 7 shows that the change in the SMR due to a shift of patients from stratum l to stratum k is,in absolute terms, large if the number of shifted patients η is large, the number of patients treatedin the hospital n h is small, and the hospital’s overall expected mortality rate ¯ p e, ext h ( n hk , n hl ) is low.Since ¯ p e, ext h ( n hk , n hl ) depends on the stratum-specific patient numbers n hs , s = 1 , . . . , S , the latterimplies that the change in the SMR due to a variation in case mix depends on the initial case mixof the hospital.The sign of Eq 7 is determined according toΩ ext hkl ( η ) > p hk − p hl ) > ( p e, ext k − p e, ext l ) · SMR ext h ( n hk , n hl ) , (8)Ω ext hkl ( η ) = 0 if ( p hk − p hl ) = ( p e, ext k − p e, ext l ) · SMR ext h ( n hk , n hl ) , (9)Ω ext hkl ( η ) < p hk − p hl ) < ( p e, ext k − p e, ext l ) · SMR ext h ( n hk , n hl ) . (10)Hence, the direction of the change in the SMR due to a shift of patients from stratum l to stratum k depends on the difference between the hospital’s mortality rates of these strata ( p hk − p hl ), thedifference between the strata’s expected mortality rates ( p e, ext k − p e, ext l ) and the hospital’s SMR.If the hospital’s mortality rate in stratum k is higher than in stratum l ( p hk − p hl >
0) while theopposite is true for the expected mortality rates ( p e, ext k − p e, ext l < p hk − p hl > p e, ext k − p e, ext l > p hk − p hl < p e, ext k − p e, ext l <
0) imply that the SMR of a hospital increases (decreases) if the initial SMRof the hospital is high (low). The SMR generally changes in accordance with the relation betweenactual and expected mortality rate differences only if SMR ext h ( n hk , n hl ) = 1, as this implies thatΩ ext hkl ( η ) (cid:82) p hk − p hl ) (cid:84) ( p e, ext k − p e, ext l ).As noted above, performance in line with expected mortality rates ( p hk = p e, ext k and p hl = p e, ext l )does not imply that the SMR is insensitive to the number of patients belonging to the consideredstrata. Under this condition, p hk − p hl > ext hkl ( η ) (cid:82) ext h ( n hk , n hl ) (cid:81)
1. Thus,a shift of patients from a stratum with a lower to a stratum with a higher mortality rate leads toan increase (decrease) in the SMR if the hospital’s SMR initially is smaller (greater) than 1. By thesame token, a shift of patients from a stratum with a higher to a stratum with a lower mortalityrate p hk − p hl < ext hkl ( η ) (cid:82) ext h ( n hk , n hl ) (cid:82) k ( n hk = n h ),the hospital’s SMR equals the relation between the actual and the observed mortality rate of thatstratum, i.e.SMR ext h ( n h = 0 , . . . , n h,k − = 0 , n hk = n h , n h,k +1 = 0 , . . . , n hS = 0) = p hk p e, ext k . (11)For illustration of case-mix sensitivity under external standardization, we considered two hospitalsand three strata of patients (Table 1). Both hospitals had the same case mix, with 20 − η patientsbelonging to stratum 1, η patients belonging to stratum 2 and 5 patients belonging to stratum 3.The parameter η is used to determine the allocation of patients to stratum 1 and stratum 2. If η = 0,both hospitals had 20 patients in stratum 1 and 0 patients in stratum 2. If η = 20, 20 patients wereallocated to stratum 2 while the hospitals have no patient in stratum 1. Furthermore, both hospitalsperformed in line with expected mortality rates in strata 1 and 2. The only difference between thehospitals is that hospital 1 had a higher-than-expected mortality rate in stratum 3 (0 . > .
15) whilehospital 2 performed better than expected in this stratum (0 . < . η . Although both hospitals were identical in case mix and performedin line with expected mortality rates in both affected strata, their SMRs are affected by a shift ofpatients from stratum 1 to stratum 2. As indicated by Eqs 8-10, hospital 1 experiences an increase inits SMR whereas the SMR of hospital 2 decreases when the number of patients allocated to stratum2 is increased (i.e. η is increased). This is because mortality in stratum 2 was lower than mortalityin stratum 1 and the SMR of hospital 1 exceeds unity while the SMR of hospital 2 is below unity.Table 1: Example of variations in case mix under external standardization: parameter valuesStratum Hospital 1 ( H ) Hospital 2 ( H ) Exp. mortality rates n s p s n s p s p e, ext s − η − η η η Variations in hospital size under external standardization
To examine variations in hospital size, we considered a proportional shift in the numbers of patientstreated in all strata of a hospital by the scale factor λ ∈ R + , where λ = 1 is the initial scale of thehospital. For λ >
1, this reflects a situation in which the total number of patients treated in thehospital is increased by factor λ while the case mix (i.e. the shares of the strata in the hospital’stotal number patients) is held constant. For the SMR under external standardization is follows thatSMR ext h ( λn h , . . . , λn hS ) = (cid:80) Ss =1 λn hs p hs (cid:80) Ss =1 λn hs p e, ext s = (cid:80) Ss =1 n hs p hs (cid:80) Ss =1 n hs p e, ext s = SMR ext h ( n h , . . . , n hS ) . (12)5 .900.951.001.051.10 0 5 10 15 20 Number of patients h shifted from stratum 1 to stratum 2 S M R Hospital H H Figure 1: SMRs for different numbers of patients η shifted from stratum 1 to stratum 2 in hospital1, holding the number of patients belonging to stratum 3 constantSince the value of the scaled SMR is the same as the value of the original SMR, the SMR fulfills theaxiomatic requirement of scale insensitivity under external standardization. Increases in hospitalsize do not change the value of the SMR, ceteris paribus.Scale insensitivity under external standardization is illustrated by the example of a hospitalwith two strata, containing 20 and 40 patients, respectively, in the initial situation (Table 2). Themortality rates were assumed to be 0.05 in the first and 0.15 in the second stratum. Expectedmortality rates in both strata were fixed at the value of 0.1. In the initial situation, this correspondsto 7 observed and 6 expected deaths, which results in a SMR of 1.17. Doubling the size of thehospital while holding case mix constant ( λ = 2) doubles both, the number of patients and thenumber of deaths in each stratum. However, the SMR of the hospital remains constant at the valueof 1.17. The same is true for further increases of hospital size as induced by higher values of λ .Table 2: Example of variations in hospital size under external standardizationQuantity initial λ = 2 λ = 3 λ = 4 λ = 5Patients in stratum 1 n h
20 40 60 80 100Patients in stratum 2 n h
40 80 120 160 200Actual number of deaths (cid:80) Ss =1 n hs p hs (cid:80) Ss =1 n hs p e, ext s ext h p h = 0 . , p h = 0 . , p e, ext1 = p e, ext2 = 0 . Variations in actual mortality rates under external standardization
Effects of variations in actual mortality rates were examined by calculating the marginal effect (i.e.the partial derivative) [25] of an increase in the mortality rate of stratum k in hospital h on thehospital’s SMR: ME ext h,p hk := ∂ SMR ext h ∂p hk = n hk n h · p e, ext h . (13)If n hk >
0, Eq 13 implies that ME ext h,p hk >
0, i.e. an increase in the mortality rate of a specific stratumincreases the SMR of the hospital. The SMR under external standardization therefore fulfills theaxiomatic requirement of strict monotonicity. The increase in the SMR induced by an increase inthe stratum-specific mortality rate is relatively large (small) if the patients included in the stratum6ccount for a large (small) share n hk /n h of patients treated in the hospital. Furthermore, themarginal effect decreases in the hospital’s expected overall mortality rate ¯ p e, ext h . The latter impliesthat an increase in stratum-specific mortality generally affects hospitals differently, as ¯ p e, ext h dependson a hospital’s case mix.This result also applies in the case in which the hospital’s actual mortality rates of all strataare increased by the absolute amount of d p . This corresponds to a situation in which the overallmortality rate of the hospital is increased by d p . Calculating the differential of Eq 5 in all actualmortality rates and using d p hs = d p, s = 1 , . . . , S yieldsdSMR ext h = S (cid:88) s =1 ∂ SMR ext h ∂p hs d p hs = d p ¯ p e, ext h . (14)Similar to an increase in the mortality rate of a single stratum, increases in the mortality rates ofall strata have a large (small) impact on the hospital’s SMR when the hospital’s overall expectedmortality rate is small (large).The results on mortality rate variations under external standardization are illustrated by theexample of two hospitals and three strata of patients (Table 3). Both hospitals treated 10 patients,with 5 belonging to stratum 1. The difference between the hospitals was that the remaining 5patients of hospital 1 belonged to stratum 2 whereas those of hospital 2 belonged to stratum 3. Inall strata, the hospitals performed in line with expected mortality rates, such that the SMR of bothhospitals in the initial situation is 1.Holding the remaining parameter values constant, Fig 2 shows the SMRs of the hospitals fordifferent mortality rates in stratum 1. Note that the mortality rates in stratum 1 were variedsimultaneously for hospital 1 and hospital 2 in each scenario ( p = p ), such that there is nodifference in the performance of the hospitals with respect to stratum 1. While the SMRs ofboth hospitals are equal in the initial situation, lower-than-expected mortality rates in stratum 1( p = p < .
1) imply that the SMR of hospital 1 is lower than the SMR of hospital 2. For higher-than-expected mortality rates ( p = p > . H ) Hospital 2 ( H ) Exp. mortality rate s n s p s n s p s p ext s p = 0.1 5 p = 0.1 0.12 5 0.15 - - 0.153 - - 5 0.3 0.3 Variations in expected mortality rates under external standardization
Effects of variations in expected mortality rates on the SMR under external standardization wererevealed by calculating the marginal effect of an increase in the stratum-specific expected mortalityrate p e, ext k : ME ext h,p ek := ∂ SMR ext h ∂p e, ext k = − SMR ext h · n hk n h · p e, ext h . (15)According to Eq 15, n hk > ext h,p ek <
0, i.e. an increase in the expected mortalityrate of a stratum reduces the hospital’s SMR if it treated patients belonging to this stratum. Thisreduction is (in absolute terms) larger for hospitals with higher SMRs, a larger share n hk /n k ofpatients belonging the considered stratum, and lower expected overall mortality rates ¯ p e, ext h . Thus,effects of variations in stratum-specific expected mortality rates depend on the hospital’s case mixand the initial value of the hospital’s SMR. 7 .01.5 0.0 0.1 0.2 0.3 Actual mortality rate of stratum 1 S M R Hospital H H Figure 2: SMRs for different mortality rates p = p The same applies to an increase in all stratum-specific expected mortality rates by the absoluteamount of d p e, ext s = d p, s = 1 , . . . , S as the associated change in the SMR depends on both the sizeof the hospital’s SMR and the expected overall mortality rate:dSMR ext h = S (cid:88) s =1 ∂ SMR ext h ∂p e, ext s d p e, ext s = − SMR ext h · d p ¯ p e, ext h . (16)To illustrate the effects of changes in expected mortality rates under external standardization, weconsidered two hospitals and two strata of patients (Table 4). Both hospitals had 5 patients with amortality rate of 0.1 in stratum 1. The hospitals differed with respect to stratum 2, where hospitalhospital 1 had 5 patients with a mortality rate of 0.2 and hospital hospital 2 had 15 patients witha mortality rate of 0.15. Note that both hospitals performed worse than expected in stratum 2 asthe expected mortality rate was 0.1. Overall, hospital 2 was performing better than hospital 1 dueto equal actual mortality rates in stratum 1 and a lower mortality rate in stratum 2.Fig 3 shows the effect of varying the expected mortality rate of stratum 1 p e, ext1 on the SMRsof the hospitals. Starting at low expected mortality rates of stratum 1, the SMR of hospital 1 ishigher than the SMR of hospital 2, implicating that hospital 2 performed better than hospital 1.Increasing the expected mortality rate of stratum 1 reduces the SMRs of both hospitals. However,since hospital 1 has a higher share of patients in stratum 1 and a higher initial SMR, it experiencesa stronger decrease in its SMR. At an expected mortality rate of p e, ext1 = 0 .
14, the SMRs of bothhospitals are equal. For further increased expected mortality rates of stratum 1, the SMR of hospital1 becomes lower than the SMR of hospital 2 although the overall performance of hospital 2 wasbetter than the performance of hospital 1. This result is driven by the fact that stratum 1 accountsfor a higher share of patients in hospital 1 than in hospital 2. By the virtue of Eq 15, this impliesthat hospital 1 “benefits” more from increases in the expected mortality rate of this stratum interms of reductions in the SMR even if the SMRs of both hospitals are equal. With respect tothe formulated axiomatic requirements, the example therefore demonstrates that the SMR underexternal standardization violates the dominance principle.Table 4: Example of variations in expected mortality rate under external standardization: parametervalues Stratum Hospital 1 ( H ) Hospital 2 ( H ) Exp. mortality s n s p s n s p s p e, ext s p e, ext1 .81.01.21.41.61.82.0 0.1 0.2 0.3 Expected mortality rate of stratum 1 S M R Hospital H H Figure 3: SMRs for different expected mortality rates p e, ext1 Internal standardization
The stratum-specific mortality rates p e, int s were calculated from the same dataset used for lateranalysis when derived by internal standardization. The SMR of hospital h based on the internalstandard therefore is expressed asSMR int h = ¯ p h ¯ p e, int h = (cid:80) Ss =1 n hs p hs (cid:80) Ss =1 n hs p e, int s , (17)where ¯ p e, int h = n − h (cid:80) Ss =1 n hs p e, int s is the hospital’s internally standardized expected mortality rate.Since expected mortality rates were derived from the same dataset used for calculation of thehospitals’ SMRs, they implicitly depend on the stratum-specific mortality rates p js and the numberof patients n js of the included hospitals j = 1 , . . . , H . The internal standard may be chosen suchthat the expected mortality rate of each stratum equals the weighted average mortality rate of thatstratum across all hospitals, i.e. p e, int s := ¯ p s = 1 n s H (cid:88) j =1 n js p js , (18)where n s = (cid:80) Hj =1 n js is the total number of patients in stratum s . In terms of interpretability, thisapproach to standardization has the advantage that a hospital with average mortality rates in allstrata ( p hs = ¯ p s , s = 1 , . . . , S ) has a SMR int h = 1. Variations in case mix under internal standardization
Given a shift of η patients from stratum l to stratum k , the change in the SMR under internalstandardization is defined asΩ int hkl ( η ) := SMR int h ( n hk + η, n hl − η ) − SMR int h ( n hk , n hl )= ( p hk − p hl ) − (˜ p hk − ˜ p hl ) · SMR int h ( n hk , n hl ) η − n h ¯ p e, int h ( n hk , n hl ) + (˜ p hk − ˜ p hl ) . (19)This expression analogous to Eq 7 with the exogenous expected mortality rates p e, ext k , p e, ext l replacedby ˜ p hk = α hk p hk + (1 − α hk )¯ p k ( n hk ) , (20)˜ p hl = α hl p hl + (1 − α hl )¯ p l ( n hl ) . (21)9ence, ˜ p hk and ˜ p hl represent weighted averages of the hospital’s stratum-specific mortality rates( p hk , p hl ) and the respective average stratum-specific mortality rates (¯ p k , ¯ p l ). The weights α hk =( n hk + η ) / ( n k + η ) and α hl = ( n hl − η ) / ( n l − η ) reflect the degree to which hospital h accountsfor the total number of patients in the considered strata. Similar to the SMR under externalstandardization, the SMR under internal standardization generally changes due to a change in casemix. Thus, it does not fulfill the axiomatic requirement of case-mix insensitivity.From Eq 19 follows thatΩ int hkl ( η ) > p hk − p hl ) > (˜ p hk − ˜ p hl ) · SMR int h ( n hk , n hl ) , (22)Ω int hkl ( η ) = 0 if ( p hk − p hl ) = (˜ p hk − ˜ p hl ) · SMR int h ( n hk , n hl ) , (23)Ω int hkl ( η ) < p hk − p hl ) < (˜ p hk − ˜ p hl ) · SMR int h ( n hk , n hl ) . (24)Similar to the results for case-mix variations under external standardization (Eqs 8-10), the directionof change in the SMR induced by a shift of patients from stratum l to stratum k depends on thedifference of the actual stratum-specific mortality rates ( p hk − p hl ) and the SMR-weighted differencein the endogenous threshold mortality rates (˜ p hk − ˜ p hl ).In the extreme case in which the hospital accounts for the total number of patients in both strata( n hk = n k , n hl = n l ), it holds that α hk = α hl = 1, which implies that ˜ p hk = p hk and ˜ p hl = p hl . ForSMR int h ( n hk , n hl ) > int hkl ( η ) ≷ p hk − p hl > int h ( n hk , n hl ) ≶ , (25)Ω int hkl ( η ) = 0 if p hk − p hl = 0 , (26)Ω int hkl ( η ) ≷ p hk − p hl < int h ( n hk , n hl ) ≷ . (27)Hence, a shift of patients from a stratum with a lower to a stratum with a higher mortality rateincreases (decreases) the SMR of hospitals with below-average (above-average) SMRs. Similarly, ashift of patients from a stratum with a higher to a stratum with a lower mortality rate decreases(increases) the SMR of hospitals with below-average (above-average) SMRs. These results aredriven by assumption that the hospital fully serves as its own reference in both strata. Hence, aconcentration of patients in a stratum with a relatively high actual mortality rate implies a greater“benefit” in terms of a lower SMR for hospitals with above-average SMRs and vice versa.In the other extreme case, the hospital accounts for a negligible share of the strata’s total numberof patients. Holding n hk and n hl constant, it can be derived that lim n k →∞ α hk = lim n k →∞ α hl = 0,which implies that lim n l →∞ ˜ p hk = ¯ p k ( n hk ) and lim n l →∞ ˜ p hl = ¯ p l ( n hl ). Thus, SMR int h ( n hk , n hl ) > int hkl ( η ) ≷ p k ( n hk ) − ¯ p l ( n hl ) > int h ( n hk , n hl ) ≶ , (28)Ω int hkl ( η ) = 0 if ¯ p k ( n hk ) − ¯ p l ( n hl ) = 0 , (29)Ω int hkl ( η ) ≷ p k ( n hk ) − ¯ p l ( n hl ) < int h ( n hk , n hl ) ≷ , (30)For large values of n k and n h relative to n hk and n hl , respectively, the stratum-specific averagemortality rates ¯ p k ( n hk ) and ¯ p l ( n hl ) are almost exclusively determined by the mortality rates ofhospitals other than h . Hence, the behavior of the SMR under internal standardization is similarto the behavior of the SMR under external standardization if the hospital accounts for negligibleshares of the strata’s total numbers of patients because the hospital has little influence in the internalstandard.For illustration, we considered the case of two hospitals and two strata of patients (Table 5).With regard to stratum 1, both hospitals were characterized by a mortality rate of 0.1, implyingthat the expected mortality rate of stratum 1 is also 0.1. With respect to stratum 2, hospital 1 wascharacterized by a higher mortality rate than hospital 2 (0 . > . η determined the number of patients treated in hospital 1 belonging to stratum 1 and 2, respectively.If η = 0, all patients of hospital 1 were allocated to stratum 1 and no patient was allocated tostratum 2. If η = 50, no patient of stratum 1 was treated in hospital 2 while the hospital treated50 patients belonging to stratum 2.The SMRs of both hospitals for different values of η are shown by Figure 4. As the mortalityrates of hospital 1 were higher or equal to those of hospital 2, the SMR of hospital 1 exceeds unitywhile the SMR of hospital 2 is below unity. If all patients of hospital 1 are allocated to stratum 1( η = 0), increases in η lead to an increase in the SMR of hospital 1 and a decrease in the SMR ofhospital 2. This behavior is in line with the fact that higher values of η imply that more patients10f hospital 1 are shifted from a stratum with a mortality rate equal to expected mortality to thestratum with a higher-than-expected mortality rate. However, at a certain number of patientsallocated from stratum 1 to stratum 2, the SMR of hospital 1 does not change due to a change in η . At this point, the size of the hospital’s SMR and its influence on the expected mortality rate ofstratum 2 has become sufficiently large to meet the condition stated by Eq 23. When the numberof patients treated in hospital 1 that is allocated from stratum 1 to stratum 2 is further increased,the SMR of hospital 1 even starts to decrease as implied by Eq 24.Table 5: Example of variations in case mix under internal standardization: parameter valuesStratum Hospital 1 ( H ) Hospital 2 ( H )s n s p s n s p s − η η Number of patients h of H shifted from stratum 1 to stratum 2 S M R Hospital H H Figure 4: SMRs for different numbers of patients η shifted from stratum 1 to stratum 2 in hospital 1 Variations in hospital size under internal standardization
In case of internal standardization, increasing the number of patients treated in hospital h in allstrata by factor λ yields SMR int h ( λn h , . . . , λn hS ) = (cid:80) Ss =1 n hs p hs (cid:80) Ss =1 n hs p e, int s ( λn hs ) , (31)where p e, int s ( λn hs ) = ( (cid:80) Hj =1 n js p js ) + ( λ − n hs p hs ( (cid:80) Hj =1 n js ) + ( λ − n hs . (32)Since Eq 32 shows that the stratum-specific expected mortality rates depend on λ , the SMR does notfulfill the axiomatic requirement of scale insensitivity under internal standardization. For furtherinvestigation, the change in the SMR due to increasing the number of patients by factor λ is definedas ∆SMR int h := SMR int h ( λn h , . . . , λn hS ) − SMR int h ( n h , . . . , n hS )= SMR int h ( n h , . . . , n hS ) · (cid:34) ¯ p e, int h ( n h , . . . , n hS )¯ p e, int h ( λn h , . . . , λn hS ) − (cid:35) . (33)11q 33 implies that the SMR of hospital h increases (decreases) due to an increase in hospital sizeif the hospital’s expected overall mortality rate decreases (increases) due to scaling. Furthermore,the magnitude of change induced by scaling is (in absolute terms) higher (lower) for hospitals withhigher (lower) initial SMRs.The condition determining the direction of change in the SMR may be expressed as∆SMR int h > S (cid:88) s =1 n hs n h [ p e, int s ( n hs ) − p e, int s ( λn hs )] > , (34)∆SMR int h = 0 if S (cid:88) s =1 n hs n h [ p e, int s ( n hs ) − p e, int s ( λn hs )] = 0 , (35)∆SMR int h < S (cid:88) s =1 n hs n h [ p e, int s ( n hs ) − p e, int s ( λn hs )] < . (36)Hence, the sign of Eq 33 depends on the patient-share-weighted average of the differences p e, int s ( n hs ) − p e, int s ( λn hs ) in stratum-specific expected mortality rates before and after scaling. Further analysisreveals that p e, int s ( n hs ) − p e, int s ( λn hs ) > p hs < ¯ p s ( n hs ) , (37) p e, int s ( n hs ) − p e, int s ( λn hs ) = 0 if p hs = ¯ p s ( n hs ) , (38) p e, int s ( n hs ) − p e, int s ( λn hs ) < p hs > ¯ p s ( n hs ) . (39)For a hospital with above-average stratum-specific mortality rates in all strata, Eqs 37-39 imply adecrease in the SMR when the scale of those hospital is increased. On the contrary, the SMR ofa hospital performing better than average in all strata increases when its size is increased whileholding case mix constant. In accordance with these results, it can be derived thatlim λ →∞ SMR int h ( λn h , . . . , λn hS ) = (cid:80) Ss =1 n hs p hs (cid:80) Ss =1 n hs lim λ →∞ p e, int s ( λn hs ) = 1 , (40)since Eq 32 implies that lim λ →∞ p e, int s ( λn hs ) = p hs . Hence, the SMR under internal standardizationapproaches (but does not cross) unity when the scale of a hospital is increased. These results reflectsthat the hospital is increasingly becoming its own reference when its size is increased because itincreasingly dominates the value of the stratum-specific expected mortality rates.For illustration of scale sensitivity under internal standardization, we considered three hospitalsand three strata of patients (Table 6). Hospitals 1 and 2 had patients in strata 1 and 2 and nopatient belonging to stratum 3. Hospital 3 treated patients belonging to strata 2 and 3 but nopatient belonging to stratum 1. In terms of mortality rates, hospital 2 performed better than theother hospitals in all strata. Hospital 2 performed better than hospital 3 in stratum 2. The patientnumbers of hospital 1 in all strata were scaled by factor λ .As depicted by Fig 5, in the initial situation ( λ = 1) hospital 1 has the highest SMR whilehospital 2 has the lowest SMR. The SMR of hospital 3 exceeds unity, indicating worse-than-averageperformance, but is lower than the SMR of hospital 1. Doubling the size of hospital 1 ( λ = 2) leadsto a decrease in the SMRs of all three hospitals. This is due to the increased weight of hospital1 in the calculation of the expected mortality rates of strata 1 and 2. Since the stratum-specificmortality rates of hospital 1 are higher than the average mortality rates in the initial situation, thisresults in an increase in expected mortality rates (see Eqs 37-39). However, the induced decreasein the SMR is strongest for hospital 1, implying that it becomes more close to hospital 3 in termsof the overall performance assessment. For further increased scales of hospital 1 ( λ ≥ Variations in actual mortality rates under internal standardization
The marginal effect of an increase in the mortality rate of stratum k in hospital h can be derived asME int h,p hk := ∂ SMR int h ∂p hk = n hk n h · p e, int h − SMR int h · n hk n h · p e, int h · ∂p e, int k ∂p hk (41)= n hk n h · p e, int h · (cid:18) − SMR int h · n hk n k (cid:19) (42)12able 6: Example of variations of hospital size under internal standardization: parameter valuesStratum Hospital 1 ( H ) Hospital 2 ( H ) Hospital 3 ( H )s n s p s n s p s n s p s · λ · λ Scale factor l S M R Hospital H H H Figure 5: SMRs for different scale factors λ affecting the size of hospital 1Note that the first term on the right-hand side of Eq 41 is similar to the first term on the righthand side of Eq 13. Thus, this term represents the direct effect of an increase in the stratum-specificmortality on the SMR of hospital h . However, when using an internal standard there is also anindirect effect, represented by the second term on the right hand side of Eq 41. This indirect effectemerges from the fact that the expected mortality rate p e, int k depends on the the mortality rate p hk of patients included in this stratum treated in hospital h . Given that ∂p e, int k /∂p hk = n hk /n k > n hk >
0, the second term on the right-hand side of Eq 41 is negative, which indicates that theindirect effect counteracts the positive direct effect of an increase in the stratum-specific mortalityrate.It follows that ME int h,p hk > int h < n k n hk , (43)ME int h,p hk = 0 if SMR int h = n k n hk , (44)ME int h,p hk < int h > n k n hk . (45)As shown by Eq 45, the marginal effect of p hk may even be negative if the hospital’s SMR is highand the hospital accounts for a large share of patients in stratum k , i.e. if n k /n hk is small. Thiscorresponds to the paradoxical situation in which increasing mortality in a stratum of patientstreated in a specific hospital reduces the SMR of that hospital. Hence, the SMR under internalstandardization does not fulfill the axiomatic requirement of strict monotonicity. Since n k /n hk ≥ int h > p hs = d p, s = 1 , . . . , S yields dSMR int h = S (cid:88) s =1 ∂ SMR int h ∂p hs d p hs = d p ¯ p e, int h (cid:32) − SMR int h S (cid:88) s =1 n hs n h · n hs n k (cid:33) . (46)13he expression describing the change in the internally standardized SMR (Eq 46) differs from theexpression for the change in the externally standardized SMR (Eq 14) due to the factor in parenthesesincluded in Eq 46. This factor is smaller than 1 if SMR int h >
0, implying that the increase in theSMR of a hospital due to an increase in its overall mortality rate is, generally, smaller under internalthan under external standardization. Moreover, the sign of Eq 46 is ambiguous sincedSMR int h > int h < (cid:80) Ss =1 n hs n h · n hs n k , (47)dSMR int h = 0 if SMR int h = 1 (cid:80) Ss =1 n hs n h · n hs n k , (48)dSMR int h < int h > (cid:80) Ss =1 n hs n h · n hs n k . (49)An increase in the overall mortality rate of a hospital therefore may reduce its SMR if the hospital’sSMR exceeds the threshold ( (cid:80) Ss =1 n hs n h · n hs n k ) − ≥
1. This threshold takes on the value of 1 if allpatients of the hospital are concentrated in a specific stratum k ( n hk /n h = 1) and the hospitalaccounts for all patients belonging to this stratum ( n hk /n k = 1). Thus, paradoxical effects ofmortality rate increases on the SMR may particularly arise in specialized hospitals treating specificpatient groups that are seldom treated in other hospitals.For illustration, we considered three hospitals and two strata of patients (Table 7). Hospitals1 and 2 treated patients belonging to strata 1 and 2 whereas hospital 3 treated patients belongingto stratum 2 only. With respect to stratum 2, hospital 1 had the highest mortality rate whereashospital 3 had the lowest mortality rate. In the following, the mortality rate p ∈ [0 ,
1] in stratum1 of hospital 1 is varied for different shares w ∈ { . , . , } of patients in stratum 1 treated inhospital 1. The larger w , the higher the share of patients belonging to stratum 1 that were treatedin hospital 1.The results are shown in Fig 6. If 60% of all patients in stratum 1 are allocated to hospital 1( w = 0 . w = 0 . p . Thisillustrates the paradoxical situation captured by Eq 45, in which increasing mortality in a stratumof patients reduces the SMR of the considered hospital. In the extreme case in which all patientsin stratum 1 are treated in hospital 1 ( w = 1), the inverse relationship between stratum-specificmortality and SMR of hospital 1 gets even more pronounced. If the mortality rate of stratum 1in hospital 1 reaches 100%, the SMR of hospital 1 gets close to unity. Note that this scenario alsoillustrates that the SMR of hospital 1 can become lower than the SMR of hospital 3 although themortality rate in stratum 2 (the only stratum with a positive number of patients treated in hospital3) is 10% lower in hospital 3 than in hospital 1.Table 7: Example of variations in actual mortality rates under internal standardization: parametervalues Stratum Hospital 1 ( H ) Hospital 2 ( H ) Hospital 3 ( H )s n s p s n s p s n s p s · w p · (1 − w ) 0.1 0 -2 50 0.4 50 0.1 40 0.3 Variations in expected mortality rates under internal standardization
Analogous to the SMR under external standardization, the SMR under internal standardization isaffected by changes in expected mortality rates, which leads to a violation of the dominance principle.However, due to the endogeneity of the stratum-specific expected mortality rates p e, int s under internalstandardization, variations in these expected mortality rates may be driven by variations in themortality rates and patient compositions of all hospitals in the sample. The analyses shown abovealready highlighted effects of variations in mortality rates and patient composition of a hospital onits own SMR. In the following, we examine the influence of other hospitals.14 H H H H H H H H H H H H H H H H H H H H H H H H H H H H H H H H H H H H H H H H H H H H H H H H H H H H H H H H H H H H H H H H H H H H H H H H H H H H H H H H H H H H H H H H H H H H H H H H H H H H H H H H H H H H H H H H H H H H H H H H H H H H H H H H H H H H H H H H H H H H H H H H H H H H H H H H H H H H H H H H H H H H H H H H H H H H H H H H H H H H H H H H H H H H H H H H H H H H H H H H H H H H H H H H H H H H H H H H H H H H H H H H H H H H H H H H H H H H H H H H H H H H H H H H H H H H H H H H H H H H H H H H H H H H H H H H H H H H H H H H H H H H H H H H H H H H H H H H H H H H H H H H H H H H H H H H H H H H H H H H H H H H H H H H H H H H H H H H H H H H H H H H H H H H H H H H H H H H H H H H H H H H H H H H H H H H H H H H H H H H H H H H H H H H H H H H H H H H H H H H H H H H H H H H H H H H H H H H H H H H H H H H H H H H H H H H H H H H H H H H H H H H H H H H H H H H H H H H H H Mortality rate of stratum 1 in hospital 1 p S M R w Figure 6: SMRs for different mortality rates p and patient shares w of hospital 1First, the change in the SMR due to a change in the expected mortality rate of stratum k isexpressed as dSMR int h = − SMR int h · n hk n k · p e, int h · d p e, int k . (50)Eq 50 indicates that the SMR decreases (increases) if the expected mortality rate increases (de-creases). Using Eq 18, the marginal effect of an increase in the mortality rate of stratum k inhospital i (cid:54) = h is ∂p e, int k ∂p ik = n ik n k > n ik > . (51)In combination, Eqs 50-51 imply that the SMR of a hospital decreases when the stratum-specificmortality rates of other hospitals in the sample are increased. The reason is that such increases inmortality rates unambiguously increase the expected mortality rates of the affected strata.This effect is illustrated by the behavior of the SMR of hospital 2 in Fig 6, which decreases inthe mortality rate of stratum 1 in hospital 1 as long as hospital 2 accounts for a positive number ofpatients in that stratum.Second, adding η patients to stratum k in hospital i implies p e, int k ( n ik + η ) − p e, int k ( n ik ) = p ik − ¯ p k ( n ik )1 + η − n k . (52)Hence, increasing the size of stratum k in hospital i (and, thus, its share in the total number ofpatients belonging to stratum k ) increases (decreases) the expected mortality rate of that stratumif hospital i ’s mortality rate of that stratum p ik is higher (lower) than the average mortality rate ofthat stratum ¯ p k ( n ik ). The direction of change in the SMR of a hospital h (Eq 50) due to a changein other hospitals’ stratum-specific patient numbers therefore depends on whether those hospitalsperform better or worse than average in the affected strata.An illustration of this result is given by the variation in the SMR of hospital 2 in Fig 4. Sincehospital 1 performs worse than average in stratum 2, increasing the number of patients η in thishospital belonging to that stratum reduces the SMR of hospital 2. Summary of results
Evaluating the derived properties of the SMR using the five axiomatic requirements formulatedabove yielded differences between external and internal standardization (Table 8). Under externalstandardization, the SMR fulfills the requirements of strict monotonicity and scale insensitivitybut violates the requirement of case-mix insensitivity, the equivalence principle, and the dominanceprinciple. All axiomatic requirements not fulfilled by the SMR under external standardization are15lso not fulfilled by the SMR under internal standardization due to similarity in their mathematicalstructure. Additionally, higher mortality rates may induce lower SMR values and the SMR of largehospitals is driven towards unity under internal standardization. The internally standardized SMRtherefore also violates the requirements of strict monotonicity and scale insensitivity and, thus,fulfills none of the postulated axiomatic requirements.Table 8: Fulfillment of axiomatic requirements by standardization approachSMR under SMR underAxiomatic requirement external standardization internal standardizationStrict monotonicity yes noCase-mix insensitivity no noScale insensitivity yes noEquivalence principle no noDominance principle no no
Discussion
This paper proposed five axiomatic requirements for risk standardized mortality measures (strictmonotonicity, case-mix insensitivity, scale insensitivity, equivalence principle, dominance principle).Given these axiomatic requirements, properties of the SMR were formally investigated and evaluated.The results of our analyses indicate that several properties of the SMR hamper valid assessmentand comparison of hospital performance based on this measure. This finding has very high publichealth relevance, as clinicians, healthcare decision makers, the public, and all users of quality of careinformation based on SMRs are confronted with potentially biased information and, thus, may drawinappropriate conclusions. Effects of variations in case mix on the SMR were found to depend notonly on hospital size and the initial patient composition of a hospital but also on the size of its SMR.Variations in actual mortality rates depend on the hospital’s expected overall mortality rate and,thus, on its case mix. Under external standardization, the stratum-specific expected mortality rateshave crucial influence on the size of the SMR. Paradoxically, variations in these expected mortalityrates may reverse the rank of two hospitals although one of the hospitals unambiguously performsbetter than the other in terms of actual mortality rates.While hospital size has no effect on the SMR under external standardization, this desirableproperty of scale insensitivity is absent under internal standardization. In this case, the SMR oflarge hospitals is, ceteris paribus, more close to 1 than the hospital of small hospitals. This resultsis driven by the fact that large hospitals have more influence on expected mortality rates than smallhospitals under internal standardization. This influence on expected mortality rates also modifies theeffect of variations in actual mortality rates on the SMR. In extreme cases, higher actual mortalityrates may be related to a lower SMR of the considered hospital. This paradoxical effect particularlymay arise in specialized hospitals that almost exclusively treated specific patient groups.In summary, our findings significantly extend previous research on properties of the SMR [19,20, 21, 22, 23] by formally deriving expressions and conditions describing the behavior of the SMR.In this way, this study provides a comprehensive and exact characterization of this commonly usedhospital performance measure.
Limitations and prospects
The analyses presented in this paper provide a clear description of central properties of the SMR.However, although we constructed hypothetical examples illustrating these properties, we did notprovide empirical examples based on real-world data. Presumably, the extent to which the describeddrawbacks of the SMR are empirically relevant depends on the considered indication, the choice ofrisk factors used to define patient strata, and the similarity of the considered hospitals with respectto case mix and mortality rates. While investigating these issues in specific settings is beyond thescope of this paper, future studies may examine the insights highlighted in this paper empirically.Furthermore, while this study revealed several undesirable properties of the SMR under bothexternal and internal standardization, it did not provide an alternative measure of hospital perfor-mance. Some studies point to certain advantages of measures like the comparative mortality figure(CMF) or excess risk (ER) [16, 26]. However, there is a lack of formal analysis and comparison16n the literature. Systematically analyzing and developing suitable hospital performance measurestherefore may be a promising route for further research.Finally, there may be relevant properties of the SMR that were not investigated above. Formalexamination of those properties may be a valuable complement to the analyses presented in thispaper.
Practical implications
Contrary to internal standardization, external standardization ensures strict monotonicity and scaleinsensitivity. Hence, external standardization should generally be preferred over internal standard-ization in practical applications. This is particularly true when the number of analyzed hospitals issmall or when there are large and/or specialized hospitals that almost exclusively treated specificpatient groups. Nonetheless, practitioners should be aware of the potential drawbacks related to theuse of the SMR under both standardization approaches. The SMR generally violates the require-ment of case-mix insensitivity, the equivalence principle, and the dominance principle. Particularlyin the presence of large heterogeneity of the analyzed hospitals in terms of case mix and mortal-ity rates, the SMR cannot be trusted. As a general recommendation, empirical studies thereforeshould assess and report the degree of heterogeneity of the considered hospitals and take effects ofheterogeneity into account when interpreting calculated SMRs.
References [1] Agency for Healthcare Research and Quality (AHRQ. QHRQ Quality Indicators; 2020. Avail-able from: .[2] Centers for Medicare & Medicaid Services. Hospital Compare; 2020. Available from: .[3] IQTIG. Qualit¨atsindikatoren; 2020. Available from: https://iqtig.org/qs-instrumente/qualitaetsindikatoren/ .[4] Mansky T, Vlzke T, Nimptsch U. Improving outcomes using German Inpatient Quality In-dicators in conjunction with peer review procedures. Zeitschrift fr Evidenz, Fortbildung undQualitt im Gesundheitswesen. 2015;109(9-10):662–670.[5] U S News & World Report. America’s Best Hospitals; 2020. Available from: https://health.usnews.com/best-hospitalshttps://health.usnews.com/best-hospitals