Andrew Wey

Effects of High‐Risk Kidneys on Scientific Registry of Transplant Recipients Program Quality Reports

There is a perception that transplanting high‐risk kidneys causes programs to be identified as underperforming, thereby increasing the frequency of discards and diminishing access to transplant. Thus, the Organ Procurement and Transplantation Network (OPTN) has considered excluding transplants using kidneys from donors with high Kidney Donor Profile Index (KDPI) scores (≥0.85) when assessing program performance. We examined whether accepting high‐risk kidneys (KDPI ≥0.85) for transplant yields worse outcome evaluations. Despite a clear relationship between KDPI and graft failure and mortality, there was no relationship between a programs use of high‐KDPI kidneys and poor performance evaluations after risk adjustment. Excluding high‐KDPI donor transplants from the June 2015 evaluations did not alter the programs identified as underperforming, because in every case underperforming programs also had worse‐than‐expected outcomes among lower‐risk donor transplants. Finally, we found that hypothetically accepting and transplanting additional kidneys with KDPI similar to that of kidneys currently discarded would not adversely affect program evaluations. Based on the study findings, there is no evidence that programs that accept higher‐KDPI kidneys are at greater risk for low performance evaluations, and risk aversion may limit access to transplant for candidates while providing no measurable benefit to program evaluations.

Potential Implications of Recent and Proposed Changes in the Regulatory Oversight of Solid Organ Transplantation in the United States

Every 6 months, the Scientific Registry of Transplant Recipients (SRTR) publishes evaluations of every solid organ transplant program in the United States, including evaluations of 1‐year patient and graft survival. The Centers for Medicare & Medicaid Services (CMS) and the Organ Procurement and Transplantation Network (OPTN) Membership and Professional Standards Committee (MPSC) use SRTRs 1‐year evaluations for regulatory review of transplant programs. Concern has been growing that the regulatory scrutiny of transplant programs with lower‐than‐expected outcomes is harmful, causing programs to undertake fewer high‐risk transplants and leading to unnecessary organ discards. As a result, CMS raised its threshold for a “Condition‐Level Deficiency” designation of observed relative to expected 1‐year graft or patient survival from 1.50 to 1.85. Exceeding this threshold in the current SRTR outcomes report and in one of the four previous reports leads to scrutiny that may result in loss of Medicare funding. For its part, OPTN is reviewing a proposal from the MPSC to also change its performance criteria thresholds for program review, to review programs with “substantive clinical differences.” We review the details and implications of these changes in transplant program oversight.

Influence of kidney offer acceptance behavior on metrics of allocation efficiency

We investigated associations of deceased donor kidney offer acceptance with likelihood of the kidney being discarded, cold ischemia time at transplant (CIT), and likelihood of the kidney being exported outside the donation service area (DSA). We used kidney offers from donors in the Scientific Registry of Transplant Recipients July 1, 2015‐June 30, 2016, and a stratified logistic regression to estimate odds ratios of acceptance for candidates wait‐listed in a DSA. We estimated associations between these ratios and likelihood of discard or export and CIT at transplant. Approximately 0.50 kidneys were discarded per donor; lower DSA‐specific offer acceptance ratios were associated with more discards (R=−0.20; P=0.006). For a median donor, the DSA with the highest acceptance ratio would place 0.12 more kidneys per donor than the DSA with the lowest ratio. Low acceptance ratios were associated with higher CIT (R=−0.23; P<0.001). For the median donor, CIT was 2.9 hours shorter for the DSA with the highest versus lowest acceptance ratio. Low acceptance ratios were associated with more exports (R=−0.43; P<0.001); the probability was 15% higher for a median donor in the DSA with the lowest versus highest acceptance ratio. Improving lower‐than‐expected offer acceptance would likely reduce discards, CIT, and exports.

A Five‐Tier System for Improving the Categorization of Transplant Program Performance

OBJECTIVE To better inform health care consumers by better identifying differences in transplant program performance. DATA SOURCE Adult kidney transplants performed in the United States, January 1, 2012-June 30, 2014. STUDY DESIGN In December 2016, the Scientific Registry of Transplant Recipients instituted a five-tier system for reporting transplant program performance. We compare the differentiation of program performance and the simulated misclassification rate of the five-tier system with the previous three-tier system based on the 95 percent credible interval. DATA COLLECTION Scientific Registry of Transplant Recipients database. PRINCIPAL FINDINGS The five-tier system improved differentiation and maintained a low misclassification rate of less than 22 percent for programs differing by two tiers. CONCLUSION The five-tier system will better inform health care consumers of transplant program performance.

A kidney offer acceptance decision tool to inform the decision to accept an offer or wait for a better kidney

We developed a kidney offer acceptance decision tool to predict the probability of graft survival and patient survival for first‐time kidney‐alone candidates after an offer is accepted or declined, and we characterized the effect of restricting the donor pool with a maximum acceptable kidney donor profile index (KDPI). For accepted offers, Cox proportional hazards models estimated these probabilities using transplanted kidneys. For declined offers, these probabilities were estimated by considering the experience of similar candidates who declined offers and the probability that declining would lead to these outcomes. We randomly selected 5000 declined offers and estimated these probabilities 3 years post‐offer had the offers been accepted or declined. Predicted outcomes for declined offers were well calibrated (<3% error) with good predictive accuracy (area under the curve: graft survival, 0.69; patient survival, 0.69). Had the offers been accepted, the probabilities of graft survival and patient survival were typically higher. However, these advantages attenuated or disappeared with higher KDPI, candidate priority, and local donor supply. Donor pool restrictions were associated with worse 3‐year outcomes, especially for candidates with high allocation priority. The kidney offer acceptance decision tool could inform offer acceptance by characterizing the potential risk–benefit trade‐off associated with accepting or declining an offer.

Offer acceptance practices and geographic variability in allocation model for end‐stage liver disease at transplant

Offer acceptance practices may cause geographic variability in allocation Model for End‐Stage Liver Disease (aMELD) score at transplant and could magnify the effect of donor supply and demand on aMELD variability. To evaluate these issues, offer acceptance practices of liver transplant programs and donation service areas (DSAs) were estimated using offers of livers from donors recovered between January 1, 2016, and December 31, 2016. Offer acceptance practices were compared with liver yield, local placement of transplanted livers, donor supply and demand, and aMELD at transplant. Offer acceptance was associated with liver yield (odds ratio, 1.32; P < 0.001), local placement of transplanted livers (odds ratio, 1.34; P < 0.001), and aMELD at transplant (average aMELD difference, –1.62; P < 0.001). However, the ratio of donated livers to listed candidates in a DSA (ie, donor‐to‐candidate ratio) was associated with median aMELD at transplant (r = −0.45; P < 0.001), but not with offer acceptance (r = 0.09; P = 0.50). Additionally, the association between DSA‐level donor‐to‐candidate ratios and aMELD at transplant did not change after adjustment for offer acceptance. The average squared difference in median aMELD at transplant across DSAs was 24.6; removing the effect of donor‐to‐candidate ratios reduced the average squared differences more than removing the effect of program‐level offer acceptance (33% and 15% reduction, respectively). Offer acceptance practices and donor‐to‐candidate ratios independently contributed to geographic variability in aMELD at transplant. Thus, neither offer acceptance nor donor‐to‐candidate ratios can explain all of the geographic variability in aMELD at transplant. Liver Transplantation 24 478–487 2018 AASLD.

Comparing Scientific Registry of Transplant Recipients posttransplant program-specific outcome ratings at listing with subsequent recipient outcomes after transplant

To improve accessibility of program‐specific reports to patients, the Scientific Registry of Transplant Recipients released a 5‐tier system for categorizing 1‐year posttransplant program evaluations. Whether this system predicts subsequent posttransplant outcomes at the time patients are waitlisted has been questioned. We investigated the association of tier at listing and the corresponding continuous score used for tier assignment, which ranges from 0 (poor outcomes) to 1 (good outcomes), with eventual 1‐year posttransplant graft survival for candidates listed between July 12, 2011, and June 16, 2014, who underwent transplant before December 31, 2016. One additional tier at listing was associated with better 1‐year posttransplant outcomes in liver (hazard ratio [HR], 0.93; 95% confidence interval [CI], 0.89–0.97) and lung transplant (HR, 0.90; 95% CI, 0.84–0.97) but not kidney (HR, 0.96; 95% CI, 0.92–1.01) or heart transplant (HR, 1.02; 95% CI, 0.93–1.10). In liver and lung transplant, longer time between listing and transplant was associated with stronger protective effects for high‐tier programs. In kidney, liver, and lung transplant, posttransplant evaluations at listing had nonlinear associations with eventual posttransplant outcomes: relatively flat for 5‐tier scores <0.5 and decreasing for scores >0.5. After adjustment for measured recipient and donor risk factors, posttransplant evaluations at listing predicted differences in eventual outcomes in liver and lung transplant, providing useful information to patients.

The clinical relevance of Organ Procurement and Transplantation Network screening criteria for program performance review in the United States

The Organ Procurement and Transplantation Network is charged with overseeing the quality of transplant programs in the United States. However, there has been controversy over whether too many programs are being identified as underperforming. It has also been suggested that dramatic improvements in outcomes throughout the United States have made the thresholds for determining which deceased donor transplant programs are underperforming no longer clinically relevant. The Scientific Registry of Transplant Recipients compared actual and expected 1‐y graft survival for transplant programs identified as underperforming in the most recent cohort (transplants from July 1, 2012 to December 31, 2014). For most organs, actual 1‐y graft survival was substantially lower for programs identified as underperforming than for programs identified as performing as expected. Differences were smallest for kidney programs: median 1‐y graft survival 89.2% vs 95.4% in large‐volume programs identified and not identified for Membership and Professional Standards Committee review, respectively. Median expected graft survival was only slightly lower (94.8% vs 95.1%, respectively), suggesting that identified and not identified programs tend to have similar risk tolerances. An excess of 143 grafts were lost from kidney programs identified as underperforming. Transplant programs identified as underperforming generally have reduced 1‐y graft survival that stakeholders may consider clinically relevant.

Seeking new answers to old questions about public reporting of transplant program performance in the United States

The Scientific Registry of Transplant Recipients (SRTR) is mandated by the National Organ Transplant Act, the Final Rule, and the SRTR contract with the Health Resources and Services Administration to report program‐specific information on the performance of transplant programs. Following a consensus conference in 2012, SRTR developed a new version of the public website to improve public reporting of often complex metrics, including changing from a 3‐tier to a 5‐tier summary metric for first‐year posttransplant survival. After its release in December 2016, the new presentation was moved to a “beta” website to allow collection of additional feedback. SRTR made further improvements and released a new beta website in May 2018. In response to feedback, SRTR added 5‐tier summaries for standardized waitlist mortality and deceased donor transplant rate ratios, along with an indicator of which metric most affects survival after listing. Presentation of results was made more understandable with input from patients and families from surveys and focus groups. Room for improvement remains, including continuing to make the data more useful to patients, deciding what additional data elements should be collected to improve risk adjustment, and developing new metrics that better reflect outcomes most relevant to patients.

The importance of transplant program measures: Surveys of 3 national patient advocacy groups

The Scientific Registry of Transplant Recipients (SRTR) provides federally mandated program‐specific transplant data to the public. Currently, there is little understanding of how different program measures are prioritized by patients in selecting a program for transplantation. This study recruited 479 transplant advocacy group members from mailing lists and social media of the National Kidney Foundation (NKF), transplant families (TF), and Transplant Recipient International Organization (TRIO). Survey participants identified how many different programs would be reasonable to consider and viewed four measures that have recently been displayed on SRTR public search result websites and six measures not recently displayed and indicated importance on a 5‐point scale. Four hundred two completed the survey (TF = 26; TRIO = 34; NKF = 342). Seventy‐eight percent indicated that considering more than one program would be reasonable. Linear mixed models adjusted for organization, education, and gender. Likert scores for pretransplant (transplant rate) and transplant volume measures were similar and were very or extremely important to over 80% of participants. Posttransplant (survival after transplant) was rated as 0.52 points higher, confidence interval (0.41, 0.64). Results indicate that many patient advocacy group members find a choice between two or more programs reasonable and value multiple measures when assessing programs where they may want to undergo transplantation.