Karen Leffondré

The MISTELS program to measure technical skill in laparoscopic surgery

BackgroundThe McGill Inanimate System for Training and Evaluation of Laparoscopic Skills (MISTELS) is a series of five tasks with an objective scoring system. The purpose of this study was to estimate the interrater and test–retest reliability of the MISTELS metrics and to assess their internal consistency.MethodsTo determine interrater reliability, two trained observers scored 10 subjects, either live or on tape. Test–retest reliability was assessed by having 12 subjects perform two tests, the second immediately following the first. Interrater and test–retest reliability were assessed using intraclass correlation coefficients. Internal consistency between tasks was estimated using Cronbach’s alpha.ResultsThe interrater and test–retest reliabilities for the total scores were both excellent at 0.998 [95% confidence interval (CI), 0.985–1.00] and 0.892 (95% CI, 0.665–0.968), respectively. Cronbach’s alpha for the first assessment of the test–retest was 0.86.ConclusionsThe MISTELS metrics have excellent reliability, which exceeds the threshold level of 0.8 required for high-stakes evaluations. These findings support the use of MISTELS for evaluation in many different settings, including residency training programs.

Competing risks analyses: objectives and approaches.

Studies in cardiology often record the time to multiple disease events such as death, myocardial infarction, or hospitalization. Competing risks methods allow for the analysis of the time to the first observed event and the type of the first event. They are also relevant if the time to a specific event is of primary interest but competing events may preclude its occurrence or greatly alter the chances to observe it. We give a non-technical overview of competing risks concepts for descriptive and regression analyses. For descriptive statistics, the cumulative incidence function is the most important tool. For regression modelling, we introduce regression models for the cumulative incidence function and the cause-specific hazard function, respectively. We stress the importance of choosing statistical methods that are appropriate if competing risks are present. We also clarify the role of competing risks for the analysis of composite endpoints.

Interval-censored time-to-event and competing risk with death: is the illness-death model more accurate than the Cox model?

BACKGROUND In survival analyses of longitudinal data, death is often a competing event for the disease of interest, and the time-to-disease onset is interval-censored when the diagnosis is made at intermittent follow-up visits. As a result, the disease status at death is unknown for subjects disease-free at the last visit before death. Standard survival analysis consists in right-censoring the time-to-disease onset at that visit, which may induce an underestimation of the disease incidence. By contrast, an illness-death model for interval-censored data accounts for the probability of developing the disease between that visit and death, and provides a better incidence estimate. However, the two approaches have never been compared for estimating the effect of exposure on disease risk. METHODS This paper compares through simulations the accuracy of the effect estimates from a semi-parametric illness-death model for interval-censored data and the standard Cox model. The approaches are also compared for estimating the effects of selected risk factors on the risk of dementia, using the French elderly PAQUID cohort data. RESULTS The illness-death model provided a more accurate effect estimate of exposures that also affected mortality. The direction and magnitude of the bias from the Cox model depended on the effects of the exposure on disease and death. The application to the PAQUID cohort confirmed the simulation results. CONCLUSION If follow-up intervals are wide and the exposure has an impact on death, then the illness-death model for interval-censored data should be preferred to the standard Cox regression analysis.

Temporal patterns of occupational asbestos exposure and risk of pleural mesothelioma.

Asbestos is the primary cause of pleural mesothelioma (PM). The objective of this study was to elucidate the importance of different temporal patterns of occupational asbestos exposure on the risk of PM using case–control data in male subjects. Cases were selected from a French case–control study conducted in 1987–1993 and the French National Mesothelioma Surveillance Program in 1998–2006. Population controls were frequency matched to cases by year of birth. Occupational asbestos exposure was evaluated with a job–exposure matrix. The dose–response relationships were estimated using restricted cubic spline functions in logistic regression models. A total of 2,466 ever-asbestos-exposed males (1,041 cases and 1,425 controls) were used. After adjustment for intensity and total duration of occupational asbestos exposure, the risk of PM was lower for subjects first exposed after the age of 20 yrs and continued to increase until 30 yrs after cessation of exposure. The effect of total duration of exposure decreased when age at first exposure and time since last exposure increased. These results, based on a large population-based case–control study, underline the need to take into account the temporal pattern of exposure on risk assessment.

Analysis of risk factors associated with renal function trajectory over time: a comparison of different statistical approaches

BACKGROUND The most commonly used methods to investigate risk factors associated with renal function trajectory over time include linear regression on individual glomerular filtration rate (GFR) slopes, linear mixed models and generalized estimating equations (GEEs). The objective of this study was to explain the principles of these three methods and to discuss their advantages and limitations in particular when renal function trajectories are not completely observable due to dropout. METHODS We generated data from a hypothetical cohort of 200 patients with chronic kidney disease at inclusion and seven subsequent annual measurements of GFR. The data were generated such that both baseline level and slope of GFR over time were associated with baseline albuminuria status. In a second version of the dataset, we assumed that patients systematically dropped out after a GFR measurement of <15 mL/min/1.73 m(2). Each dataset was analysed with the three methods. RESULTS The estimated effects of baseline albuminuria status on GFR slope were similar among the three methods when no patient dropped out. When 32.7% dropped out, standard GEE provided biased estimates of the mean GFR slope in normo-, micro- and macroalbuminuric patients. Linear regression on individual slopes and linear mixed models provided slope estimates of the same magnitude, likely because most patients had at least three GFR measurements. However, the linear mixed model was the only method to provide effect estimates on both slope and baseline level of GFR unaffected by dropout. CONCLUSION This study illustrates that the linear mixed model is the preferred method to investigate risk factors associated with renal function trajectories in studies, where patients may dropout during the study period because of initiation of renal replacement therapy.

Augmented Backward Elimination: A Pragmatic and Purposeful Way to Develop Statistical Models

Statistical models are simple mathematical rules derived from empirical data describing the association between an outcome and several explanatory variables. In a typical modeling situation statistical analysis often involves a large number of potential explanatory variables and frequently only partial subject-matter knowledge is available. Therefore, selecting the most suitable variables for a model in an objective and practical manner is usually a non-trivial task. We briefly revisit the purposeful variable selection procedure suggested by Hosmer and Lemeshow which combines significance and change-in-estimate criteria for variable selection and critically discuss the change-in-estimate criterion. We show that using a significance-based threshold for the change-in-estimate criterion reduces to a simple significance-based selection of variables, as if the change-in-estimate criterion is not considered at all. Various extensions to the purposeful variable selection procedure are suggested. We propose to use backward elimination augmented with a standardized change-in-estimate criterion on the quantity of interest usually reported and interpreted in a model for variable selection. Augmented backward elimination has been implemented in a SAS macro for linear, logistic and Cox proportional hazards regression. The algorithm and its implementation were evaluated by means of a simulation study. Augmented backward elimination tends to select larger models than backward elimination and approximates the unselected model up to negligible differences in point estimates of the regression coefficients. On average, regression coefficients obtained after applying augmented backward elimination were less biased relative to the coefficients of correctly specified models than after backward elimination. In summary, we propose augmented backward elimination as a reproducible variable selection algorithm that gives the analyst more flexibility in adopting model selection to a specific statistical modeling situation.

Regression methods for investigating risk factors of chronic kidney disease outcomes: the state of the art

BackgroundChronic kidney disease (CKD) is a progressive and usually irreversible disease. Different types of outcomes are of interest in the course of CKD such as time-to-dialysis, transplantation or decline of the glomerular filtration rate (GFR). Statistical analyses aiming at investigating the association between these outcomes and risk factors raise a number of methodological issues. The objective of this study was to give an overview of these issues and to highlight some statistical methods that can address these topics.MethodsA literature review of statistical methods published between 2002 and 2012 to investigate risk factors of CKD outcomes was conducted within the Scopus database. The results of the review were used to identify important methodological issues as well as to discuss solutions for each type of CKD outcome.ResultsThree hundred and four papers were selected. Time-to-event outcomes were more often investigated than quantitative outcome variables measuring kidney function over time. The most frequently investigated events in survival analyses were all-cause death, initiation of kidney replacement therapy, and progression to a specific value of GFR. While competing risks were commonly accounted for, interval censoring was rarely acknowledged when appropriate despite existing methods. When the outcome of interest was the quantitative decline of kidney function over time, standard linear models focussing on the slope of GFR over time were almost as often used as linear mixed models which allow various numbers of repeated measurements of kidney function per patient. Informative dropout was accounted for in some of these longitudinal analyses.ConclusionsThis study provides a broad overview of the statistical methods used in the last ten years for investigating risk factors of CKD progression, as well as a discussion of their limitations. Some existing potential alternatives that have been proposed in the context of CKD or in other contexts are also highlighted.

Cognitive Disorders and Occupational Exposure to Organophosphates: Results From the PHYTONER Study

The involvement of organophosphate insecticides in cognitive disorders is supported by epidemiologic and biological evidence, but the effects of long-term exposure remain debated. We studied the association between organophosphate exposure and cognitive performance in vine workers from the PHYTONER study cohort in the Bordeaux area of France. Results from interviews of 614 subjects conducted at the 4-year follow-up between 2001 and 2003 were analyzed. Exposure to pesticides since 1950 was assessed with cumulative exposure scores for 34 organophosphates combining an historical crop-exposure pesticide matrix and field exposure studies, taking into account the characteristics of treatment (mixing, spraying, equipment cleaning) and reentry tasks. For the 11 organophosphates retained in the analysis, exposure (ever vs. never) was associated with low cognitive performance. No dose-effect relationship was found, but an increased risk was observed with a 50-mg increase in the cumulative score, which was greater with mevinphos (Benton Visual Retention Test: odds ratio = 3.26, 95% confidence interval: 1.54, 6.88; Trail Making Test, part A: odds ratio = 3.03, 95% confidence interval: 1.39, 6.62). Our results support the hypothesis that cognitive disorders observed in vine workers may be associated with exposure to specific organophosphates.

Consultation with cardiologists for persons with new-onset chronic heart failure: A population-based study

BACKGROUND It is recommended that persons recently diagnosed with heart failure consult with a specialist in heart failure. OBJECTIVES To determine whether patients who were diagnosed with new-onset chronic heart failure (CHF) by a noncardiologist consulted with a cardiologist, and identify the factors associated with delayed consultation. METHODS Physician reimbursement administrative data were obtained for all adults with suspected new-onset CHF in the year 2000 in Quebec, defined operationally as a physician visit for CHF (based on the International Classification of Diseases, 9th Revision diagnostic codes), with no previous physician visit code for CHF in the preceding three years. Among those first diagnosed by a noncardiologist, Cox regression modelling was used to identify patient and physician characteristics associated with time to cardiology consultation. RESULTS Of the 13,523 persons coded as having incident CHF, 54.9% consulted a cardiologist within the next 2.5 to 3.5 years, and 67.4% were seen by an internist or cardiologist. Older patients, women, and those with lower comorbidity and socioeconomic status had significantly longer times to cardiology consultation. CONCLUSION The data suggest that many patients with suspected new-onset CHF do not receive prompt cardiology care, as stipulated by current recommendations. Equity of access for women and those with lower socioeconomic status appears to be problematic.

A weighted Cox model for modelling time-dependent exposures in the analysis of case-control studies

Many exposures investigated in epidemiological case-control studies may vary over time. The effects of these exposures are usually estimated using logistic regression, which does not directly account for changes in covariate values over time within individuals. By contrast, the Cox model with time-dependent covariates directly accounts for these changes over time. However, the over-sampling of cases in case-control studies, relative to controls, requires manipulating the risk sets in the Cox partial likelihood. A previous study showed that simple inclusion or exclusion of future cases in each risk set induces an under- or over-estimation bias in the regression parameters, respectively. We investigate the performance of a weighted Cox model that weights subjects according to age-conditional probabilities of developing the disease of interest in the source population. In a simulation study, the lifetime experience of a source population is first generated and a case-control study is then simulated within each population. Different characteristics of exposure are generated, including time-varying intensity. The results show that the estimates from the weighted Cox model are much less biased than the Cox models that simply include or exclude future cases, and are superior to logistic regression estimates in terms of bias and mean-squared error. An application to frequency-matched population-based case-control data on lung cancer illustrates similar differences in the estimated effects of different smoking variables. The investigated weighted Cox model is a potential alternative method to analyse matched or unmatched population-based case-control studies with time-dependent exposures.