Maarten J.G. Leening

Evaluation of newer risk markers for coronary heart disease risk classification: a cohort study.

BACKGROUND Whether newer risk markers for coronary heart disease (CHD) improve CHD risk prediction remains unclear. OBJECTIVE To assess whether newer risk markers for CHD risk prediction and stratification improve Framingham risk score (FRS) predictions. DESIGN Prospective population-based study. SETTING The Rotterdam Study, Rotterdam, the Netherlands. PARTICIPANTS 5933 asymptomatic, community-dwelling participants (mean age, 69.1 years [SD, 8.5]). MEASUREMENTS Traditional CHD risk factors used in the FRS (age, sex, systolic blood pressure, treatment of hypertension, total and high-density lipoprotein cholesterol levels, smoking, and diabetes) and newer CHD risk factors (N-terminal fragment of prohormone B-type natriuretic peptide levels, von Willebrand factor antigen levels, fibrinogen levels, chronic kidney disease, leukocyte count, C-reactive protein levels, homocysteine levels, uric acid levels, coronary artery calcium [CAC] scores, carotid intima-media thickness, peripheral arterial disease, and pulse wave velocity). RESULTS Adding CAC scores to the FRS improved the accuracy of risk predictions (c-statistic increase, 0.05 [95% CI, 0.02 to 0.06]; net reclassification index, 19.3% overall [39.3% in those at intermediate risk, by FRS]). Levels of N-terminal fragment of prohormone B-type natriuretic peptide also improved risk predictions but to a lesser extent (c-statistic increase, 0.02 [CI, 0.01 to 0.04]; net reclassification index, 7.6% overall [33.0% in those at intermediate risk, by FRS]). Improvements in predictions with other newer markers were marginal. LIMITATION The findings may not be generalizable to younger or nonwhite populations. CONCLUSION Among 12 CHD risk markers, improvements in FRS predictions were most statistically and clinically significant with the addition of CAC scores. Further investigation is needed to assess whether risk refinements using CAC scores lead to a meaningful change in clinical outcome. Whether to use CAC score screening as a more routine test for risk prediction requires full consideration of the financial and clinical costs of performing versus not performing the test for both persons and health systems. PRIMARY FUNDING SOURCE Netherlands Organization for Health Research and Development (ZonMw).

Net Reclassification Improvement: Computation, Interpretation, and Controversies A Literature Review and Clinician's Guide

Since the introduction of the term risk factor more than 50 years ago in this journal (1), many such factors have been identified. Risk factors have been incorporated into statistical models to predict occurrence of disease, to more adequately diagnose patients, and to predict outcomes after disease has been diagnosed. A substantial number of clinical guidelines have incorporated risk prediction models to aid clinicians in everyday decision making in various fields of medicine, including cardiology, oncology, and respiratory medicine (28). Many markers, such as biomarkers, genetic factors, and imaging results, have been proposed to improve these prediction models. In the past 3 decades, the most commonly used measure to quantify these improvements has been the change in the c-statistic, also known as the area under the receiver-operating characteristic curve (AUC). Studies have emphasized the limitations of the AUC, including the difficulty in interpreting the usually small changes in this statistic and the relation of the magnitude of improvement to the performance of the baseline model (912). A more relevant criterion may be to assess whether the addition of the marker to an existing model will influence clinical practice (13), which is the case if the newly predicted risk crosses a clinically meaningful threshold for an individual. This has led to the introduction of the concept of risk reclassification (14), which involves cross-tabulating categories of predicted risk for 2 models usually one with the new marker under study and the other without itto see how persons are classified differently when these models are used. The subsequent changes in risk classification can be quantified by the net reclassification improvement (NRI) (15). Risk reclassification analysis with the NRI has become popular: More than 1000 publications have cited the 2008 article that introduced the NRI (15). However, reporting of the methods used is of heterogeneous quality (16), and misconceptions are common in interpreting the NRI (17). In this article, we aim to provide a systematic assessment of the reporting practices in analyses involving the NRI and address some controversies relating to its use and interpretation. We also make recommendations on how to report and interpret the NRI (18). Overview of Current Reporting Literature Search and Data Extraction We systematically collected studies that computed the NRI or discussed results from NRI analysis. We used the Thomson Reuters Web of Knowledge (version 5.9) to identify all publications that cited 1 of 4 methodological articles by Pencina and colleagues (15, 1921) or a methodological review on reclassification measures by Cook and Ridker (22). The search was last updated on 23 April 2013 and yielded 1250 unique citations (Appendix Figure 1). We selected all 67 citations in the 4 general clinical journals with the highest impact factors (New England Journal of Medicine, The Lancet, Journal of the American Medical Association, and Annals of Internal Medicine) (2288) for data extraction (Appendix Tables 1 and 2). Our rationale was that these articles may be expected to have broad impact and be used as examples for others. Appendix Figure 1. Summary of evidence search and selection. The search was last updated on 23 April 2013. Appendix Table 1. List and Main Characteristics of the 67 Articles Appendix Table 2. Summary Characteristics of the 67 Articles Two evaluators independently extracted data from the publications. Cases on which the evaluators disagreed were discussed with a third evaluator to reach consensus. All publications were searched for NRI calculations or results. If these were found, we checked which version of the NRI was used: the category-based NRI (15) or the continuous (category-free) NRI (20) (Table 1). Next, we reviewed all articles to determine whether risk categories corresponding to diagnostic or treatment thresholds from clinical guidelines were used to evaluate the category-based NRI or whether other categorization was justified. We determined which NRI components were reported: solely the overall NRI, or the event NRI and the nonevent NRI (Table 1). Moreover, we categorized studies that reported estimates of the overall NRI on the basis of whether they reported it as a unitless statistic or a percentage. Table 1. Formulas and Interpretation of the NRI Results The predominant reason for citing one of the methodological articles was the computation of NRI estimates (n= 39) (Table 2). In 2 (5%) articles, only the continuous NRI was computed. In 5 articles, the NRI was used to compare 2 different models instead of the nested addition of 1 or more new risk markers to a simpler model. Table 2. Results From the Literature Review on Reporting of the NRI Of the 37 articles that computed category-based NRI results, 34 (92%) detailed the cutoffs for the risk categories chosen. The number of risk categories defined in the computation of the NRI varied between 2 and 6, with 3 being the most common number (Appendix Table 1). These risk categories were justified in the text, by references, or both ways in 15 (41%) instances and fully matched clinically meaningful categories with clear implications from guidelines in 4 (11%) instances (Table 2). For outcomes other than atherosclerotic cardiovascular disease, the rationale for the risk categorization could not be traced in 10 of 12 instances. Another 8 studies on the prediction of various manifestations of cardiovascular disease used cutoffs for the NRI that are the subject of ongoing debate (28, 60, 70, 89, 90)for example, a 10-year risk cutoff of 6% (rather than 10%) for low risk for coronary heart disease. Fourteen publications applied cutoffs for coronary risk stratification to broader definitions of cardiovascular disease (Appendix Table 1). Among 38 prospective studies that calculated the NRI, 30 (79%) clearly reported the time horizon at which the risk predictions were evaluated. In 7 of 30 (23%) instances where both predicted horizon and observed follow-up were detailed, we could infer that the authors studied a predicted horizon beyond the observed follow-up time (Table 2). We identified another 7 studies that used events occurring beyond the predicted horizon in the reclassification analysis. Nearly all studies reported the overall NRI. Only 11 (28%) articles presented its componentsthe event NRI and the nonevent NRIin the results section. However, 25 (68%) presented reclassification tables stratified for events and nonevents (Table 2), which allowed for computation of both NRI components by a knowledgeable reader. By combining the components presented in the text and the reclassification tables, we identified 29 (74%) studies with information on the event NRI and nonevent NRI presented for at least 1 reclassification analysis. Of note, 1 study claimed to have calculated the NRI, but no such results could be traced. Another study presented P values but no point estimates of the NRI. Of the 36 studies presenting estimates of the overall NRI, 24 (67%) expressed it as a percentage (Table 2). Eight (22%) articles in our review interpreted the overall NRI as a percentage or proportion of the entire study population that was correctly reclassified or used similar wording, such as interpreting an overall NRI of 0.29 as 29% of patients were correctly reclassified (17, 39). NRI Computation, Components, and Interpretation Predicted Time Horizons and Follow-up When prospective data are involved, such as cardiovascular events occurring during follow-up, the time horizon used to calculate the predicted risks should be clear. Because virtually every prospective study has some loss to follow-up, it is important to adequately handle observations with incomplete follow-up in the analysis. In our review, we found that studies published shortly after the introduction of the NRI often did not report how incomplete follow-up was handled. Some studies classified censored observations as nonevents (naive extrapolation) or excluded persons with incomplete follow-up. Better methods have been proposed to limit loss of useful information, including KaplanMeier estimates of the expected number of events and nonevents (prospective NRI) (20, 78) and inverse-probability weighting (91). Similarly, not every study has sufficient follow-up available for the predicted time horizons used in clinical guidelines (for example, 10-year risk for coronary heart disease [89]). In the articles we reviewed, authors made various attempts to overcome this problem, such as using Weibull extrapolation (48, 53), adjusting the predicted risk cutoffs by the ratio of actual to desired follow-up (24), or extrapolating the observed rates on the KaplanMeier survival estimates to the predicted time horizon for presentation purposes (22). Risk Categories The NRI was introduced with the example of the added value of high-density lipoprotein cholesterol level to coronary risk prediction in the Framingham Heart Study (15). Current clinical guidelines on primary prevention of cardiovascular disease recommend clear cutoffs for initiation of statin treatment (2, 3, 89, 90). These recommendations are supported by cost-effectiveness analyses. The NRI captures the change in a persons predicted risk that crosses one of such cutoffs and thus translates into a clinically meaningful change in treatment recommendations. Our review of the literature confirms the findings of Tzoulaki and colleagues: Selected risk cutoffs are generally poorly motivated and rarely correspond to therapeutic implications. Both shortcomings have been shown to yield significantly higher NRI estimates (16, 81). In some cases, the existing clinical cutoffs may result in limited reclassification. For example, in a study of a population at very low risk for cardiovascular disease, only a small number of participants would be considered to be at high risk; therefore, few will cross the recommended risk thres

Methods of data collection and definitions of cardiac outcomes in the Rotterdam Study

The prevalence of cardiovascular diseases is rising. Therefore, adequate risk prediction and identification of its determinants is increasingly important. The Rotterdam Study is a prospective population-based cohort study ongoing since 1990 in the city of Rotterdam, The Netherlands. One of the main targets of the Rotterdam Study is to identify the determinants and prognosis of cardiovascular diseases. Case finding in epidemiological studies is strongly depending on various sources of follow-up and clear outcome definitions. The sources used for collection of data in the Rotterdam Study are diverse and the definitions of outcomes in the Rotterdam Study have changed due to the introduction of novel diagnostics and therapeutic interventions. This article gives the methods for data collection and the up-to-date definitions of the cardiac outcomes based on international guidelines, including the recently adopted cardiovascular disease mortality definitions. In all, detailed description of cardiac outcome definitions enhances the possibility to make comparisons with other studies in the field of cardiovascular research and may increase the strength of collaborations.

Sex differences in lifetime risk and first manifestation of cardiovascular disease: prospective population based cohort study

Objective To evaluate differences in first manifestations of cardiovascular disease between men and women in a competing risks framework. Design Prospective population based cohort study. Setting People living in the community in Rotterdam, the Netherlands. Participants 8419 participants (60.9% women) aged ≥55 and free from cardiovascular disease at baseline. Main outcome measures First diagnosis of coronary heart disease (myocardial infarction, revascularisation, and coronary death), cerebrovascular disease (stroke, transient ischaemic attack, and carotid revascularisation), heart failure, or other cardiovascular death; or death from non-cardiovascular causes. Data were used to calculate lifetime risks of cardiovascular disease and its first incident manifestations adjusted for competing non-cardiovascular death. Results During follow-up of up to 20.1 years, 2888 participants developed cardiovascular disease (826 coronary heart disease, 1198 cerebrovascular disease, 762 heart failure, and 102 other cardiovascular death). At age 55, overall lifetime risks of cardiovascular disease were 67.1% (95% confidence interval 64.7% to 69.5%) for men and 66.4% (64.2% to 68.7%) for women. Lifetime risks of first incident manifestations of cardiovascular disease in men were 27.2% (24.1% to 30.3%) for coronary heart disease, 22.8% (20.4% to 25.1%) for cerebrovascular disease, 14.9% (13.3% to 16.6%) for heart failure, and 2.3% (1.6% to 2.9%) for other deaths from cardiovascular disease. For women the figures were 16.9% (13.5% to 20.4%), 29.8% (27.7% to 31.9%), 17.5% (15.9% to 19.2%), and 2.1% (1.6% to 2.7%), respectively. Differences in the number of events that developed over the lifespan in women compared with men (per 1000) were −7 for any cardiovascular disease, −102 for coronary heart disease, 70 for cerebrovascular disease, 26 for heart failure, and −1 for other cardiovascular death; all outcomes manifested at a higher age in women. Patterns were similar when analyses were restricted to hard atherosclerotic cardiovascular disease outcomes, but absolute risk differences between men and women were attenuated for both coronary heart disease and stroke. Conclusions At age 55, though men and women have similar lifetime risks of cardiovascular disease, there are considerable differences in the first manifestation. Men are more likely to develop coronary heart disease as a first event, while women are more likely to have cerebrovascular disease or heart failure as their first event, although these manifestations appear most often at older ages.

Coronary Calcification and the Risk of Heart Failure in the Elderly: The Rotterdam Study

OBJECTIVES The purpose of this study was to determine the association of coronary artery calcification (CAC) with incident heart failure in the elderly and examine its independence of overt coronary heart disease (CHD). BACKGROUND Heart failure is often observed as a first manifestation of coronary atherosclerosis rather than a sequela of overt CHD. Although numerous studies have shown that CAC, an established measure of coronary atherosclerosis, is a strong predictor of CHD, the association between CAC and future heart failure has not been studied prospectively. METHODS In the Rotterdam Study, a population-based cohort, 1,897 asymptomatic participants (mean age, 69.9 years; 58% women) underwent CAC scoring and were followed for the occurrence of heart failure and CHD. RESULTS During a median follow-up of 6.8 years, there were 78 cases of heart failure and 76 cases of nonfatal CHD. After adjustment for cardiovascular risk factors, increasing CAC scores were associated with heart failure (p for trend = 0.001), with a hazard ratio of 4.1 (95% confidence interval [CI]: 1.7 to 10.1) for CAC scores >400 compared with CAC scores of 0 to 10. After censoring participants for incident nonfatal CHD, increasing extent of CAC remained associated with heart failure (p for trend = 0.046), with a hazard ratio of 2.9 (95% CI: 1.1 to 7.4) for CAC scores >400. Moreover, adding CAC to cardiovascular risk factors resulted in an optimism-corrected increase in the c-statistic by 0.030 (95% CI: 0.001 to 0.050) to 0.734 (95% CI: 0.698 to 0.770) and substantially improved the risk classification of subjects (continuous net reclassification index = 34.0%). CONCLUSIONS CAC has a clear association with the risk of heart failure, independent of overt CHD. Because heart failure is highly prevalent in the elderly, it might be worthwhile to include heart failure as an outcome in future risk assessment programs incorporating CAC.

Development and Validation of a Coronary Risk Prediction Model for Older U.S. and European Persons in the Cardiovascular Health Study and the Rotterdam Study

BACKGROUND Risk scores for prediction of coronary heart disease (CHD) in older adults are needed. OBJECTIVE To develop a sex-specific CHD risk prediction model for older adults that accounts for competing risks for death. DESIGN 2 observational cohort studies, using data from 4946 participants in the Cardiovascular Health Study (CHS) and 4303 participants in the Rotterdam Study (RS). SETTING Community settings in the United States (CHS) and Rotterdam, the Netherlands (RS). PARTICIPANTS Persons aged 65 years or older who were free of cardiovascular disease. MEASUREMENTS A composite of nonfatal myocardial infarction and coronary death. RESULTS During a median follow-up of 16.5 and 14.9 years, 1166 CHS and 698 RS participants had CHD events, respectively. Deaths from noncoronary causes largely exceeded the number of CHD events, complicating accurate CHD risk predictions. The prediction model had moderate ability to discriminate between events and nonevents (c-statistic, 0.63 in both U.S. and European men and 0.67 and 0.68 in U.S. and European women). The model was well-calibrated; predicted risks were in good agreement with observed risks. Compared with the Framingham point scores, the prediction model classified elderly U.S. persons into higher risk categories but elderly European persons into lower risk categories. Differences in classification accuracy were not consistent and depended on cohort and sex. Adding newer cardiovascular risk markers to the model did not substantially improve performance. LIMITATION The model may be less applicable in nonwhite populations, and the comparison Framingham model was not designed for adults older than 79 years. CONCLUSION A CHD risk prediction model that accounts for deaths from noncoronary causes among older adults provided well-calibrated risk estimates but was not substantially more accurate than Framingham point scores. Moreover, adding newer risk markers did not improve accuracy. These findings emphasize the difficulties of predicting CHD risk in elderly persons and the need to improve these predictions. PRIMARY FUNDING SOURCE National Heart, Lung, and Blood Institute; National Institute of Neurological Disorders and Stroke; The Netherlands Organisation for Scientific Research; and the Netherlands Organisation for Health Research and Development.

Serum Magnesium and the Risk of Death From Coronary Heart Disease and Sudden Cardiac Death

Background Low serum magnesium has been implicated in cardiovascular mortality, but results are conflicting and the pathway is unclear. We studied the association of serum magnesium with coronary heart disease (CHD) mortality and sudden cardiac death (SCD) within the prospective population‐based Rotterdam Study, with adjudicated end points and long‐term follow‐up. Methods and Results Nine‐thousand eight‐hundred and twenty participants (mean age 65.1 years, 56.8% female) were included with a median follow‐up of 8.7 years. We used multivariable Cox proportional hazard models and found that a 0.1 mmol/L increase in serum magnesium level was associated with a lower risk for CHD mortality (hazard ratio: 0.82, 95% CI 0.70–0.96). Furthermore, we divided serum magnesium in quartiles, with the second and third quartile combined as reference group (0.81–0.88 mmol/L). Low serum magnesium (≤0.80 mmol/L) was associated with an increased risk of CHD mortality (N=431, hazard ratio: 1.36, 95% CI 1.09–1.69) and SCD (N=217, hazard ratio: 1.54, 95% CI 1.12–2.11). Low serum magnesium was associated with accelerated subclinical atherosclerosis (expressed as increased carotid intima‐media thickness: +0.013 mm, 95% CI 0.005–0.020) and increased QT‐interval, mainly through an effect on heart rate (RR‐interval: −7.1 ms, 95% CI −13.5 to −0.8). Additional adjustments for carotid intima‐media thickness and heart rate did not change the associations with CHD mortality and SCD. Conclusions Low serum magnesium is associated with an increased risk of CHD mortality and SCD. Although low magnesium was associated with both carotid intima‐media thickness and heart rate, this did not explain the relationship between serum magnesium and CHD mortality or SCD. Future studies should focus on why magnesium associates with CHD mortality and SCD and whether intervention reduces these risks.

Net reclassification improvement and integrated discrimination improvement require calibrated models: relevance from a marker and model perspective

markdownabstractIntroduction For the last three decades, clinical prediction models have mainly been evaluated on the basis of their ability to discriminate between persons who develop the event of interest and persons who do not, as quantified by the c-statistic or area under the receiver operator characteristic curve (AUC). The AUC considers sensitivity and specificity of the model over all possible cut-points of predicted risk. However, prediction models are often used to classify patients into risk categories that correspond to diagnostic or therapeutic decisions. This provoked the idea of comparing models according to their ability to adequately assign clinical risk categories based on absolute risk estimates. Analyses of risk reclassification have hit the ground running: uptake of measures such as net reclassification improvement (NRI) has been enormous, and guidance documents on evaluations of markers and prediction models embraced it as a step prior to full-blown cost-effectiveness analysis.More recently, several researchers reviewed the current applications of reclassification analysis and expressed concerns about inappropriate use.

Unrecognised myocardial infarction and long-term risk of heart failure in the elderly: the Rotterdam Study

Objective To examine the association between unrecognised myocardial infarction (MI) as detected by electrocardiography and the long-term risk of heart failure. Design The Rotterdam Study is a prospective population-based cohort study of the general population of a suburb of the city of Rotterdam, The Netherlands. Participants At baseline 2581 men and 3724 women aged ≥55 years were classified on the basis of electrocardiography, interview and clinical data into those with recognised MI, those with ECG-based unrecognised MI and those without MI. The participants were followed-up for incident heart failure, death or end of the study period on 12 October 2006. Results During a median follow-up time of 13.2 years, 823 cases of heart failure occurred, of which 403 in men. Independently of cardiovascular risk factors, recognised and unrecognised MIs yielded HRs of developing heart failure in men of 2.6 (95% CI 2.0 to 3.3) and 2.1 (95% CI 1.5 to 2.9), respectively. In women, recognised MI was associated with heart failure (HR=2.8; 95% CI 1.9 to 4.1), whereas unrecognised MI was not significantly related to the risk of heart failure (HR=1.1; 95% CI 0.7 to 1.7). Conclusion Unrecognised MI detected by electrocardiography yields a long-term risk of heart failure equivalent to recognised MI in men, but is not significantly related to heart failure in women. In the light of the high incidence of both unrecognised MI and heart failure in the elderly, it may be worthwhile for both doctors and patients to improve responsiveness to typical and atypical symptoms of MI.

Declining incidence of sudden cardiac death from 1990-2010 in a general middle-aged and elderly population: The Rotterdam Study

BACKGROUND Although sudden cardiac death (SCD) is relatively common, contemporary data on its incidence are lacking. OBJECTIVE The purpose of this study was to investigate the current incidence of SCD and its trend over the past 2 decades in a general middle-aged and elderly population. METHODS This study was performed within the Rotterdam Study, a prospective population-based cohort study of persons aged 45 years and older. Age-standardized incidence rates of SCD were calculated. To study trends in incidence, we compared 2 subcohorts within the total study population, 1 followed from 1990-2000 and the other from 2001-2010. RESULTS From 1990-2010, 5512 of 14,628 participants died, of whom 583 (4.0%) were classified as SCD. The overall incidence was 4.2 per 1000 person-years. The incidence was higher in men (5.2 per 1000 person-years) than in women (3.6 per 1000 person-years). Age-adjusted hazard ratio (HR) 1.84 (95% confidence [CI] 1.56-2.17) and risk of SCD increased with age (HR 1.10 per year; 95% CI 1.09-1.11). The incidence rate from 1990-2000 was 4.7 per 1000 person-years vs 2.1 per 1000 person-years from 2001-2010 (age- and sex-adjusted HR of SCD 0.60, 95% CI 0.44-0.80). To check for cohort effects, we also analyzed the incidence of total mortality and found an age- and sex-adjusted HR of total mortality of 0.82 (95% CI 0.75-0.90) for the second compared to the first subcohort, which was significantly higher than the decline in SCD incidence. CONCLUSION We found an incidence of SCD of 4.2 per 1000 person-years. The incidence decreased from 1990-2010, a period during which the diagnosis and treatment of heart disease greatly improved.