Myriam Blanchin

Comparison of CTT and Rasch-based approaches for the analysis of longitudinal Patient Reported Outcomes.

Health sciences frequently deal with Patient Reported Outcomes (PRO) data for the evaluation of concepts, in particular health-related quality of life, which cannot be directly measured and are often called latent variables. Two approaches are commonly used for the analysis of such data: Classical Test Theory (CTT) and Item Response Theory (IRT). Longitudinal data are often collected to analyze the evolution of an outcome over time. The most adequate strategy to analyze longitudinal latent variables, which can be either based on CTT or IRT models, remains to be identified. This strategy must take into account the latent characteristic of what PROs are intended to measure as well as the specificity of longitudinal designs. A simple and widely used IRT model is the Rasch model. The purpose of our study was to compare CTT and Rasch-based approaches to analyze longitudinal PRO data regarding type I error, power, and time effect estimation bias. Four methods were compared: the Score and Mixed models (SM) method based on the CTT approach, the Rasch and Mixed models (RM), the Plausible Values (PV), and the Longitudinal Rasch model (LRM) methods all based on the Rasch model. All methods have shown comparable results in terms of type I error, all close to 5 per cent. LRM and SM methods presented comparable power and unbiased time effect estimations, whereas RM and PV methods showed low power and biased time effect estimations. This suggests that RM and PV methods should be avoided to analyze longitudinal latent variables.

RespOnse Shift ALgorithm in Item response theory (ROSALI) for response shift detection with missing data in longitudinal patient-reported outcome studies

AbstractPurposeSome IRT models have the advantage of being robust to missing data and thus can be used with complete data as well as different patterns of missing data (informative or not). The purpose of this paper was to develop an algorithm for response shift (RS) detection using IRT models allowing for non-uniform and uniform recalibration, reprioritization RS recognition and true change estimation with these forms of RS taken into consideration if appropriate. MethodsThe algorithm is described, and its implementation is shown and compared to Oort’s structural equation modeling (SEM) procedure using data from a clinical study assessing health-related quality of life in 669 hospitalized patients with chronic conditions.ResultsThe results were quite different for the two methods. Both showed that some items of the SF-36 General Health subscale were affected by response shift, but those items usually differed between IRT and SEM. The IRT algorithm found evidence of small recalibration and reprioritization effects, whereas SEM mostly found evidence of small recalibration effects.ConclusionAn algorithm has been developed for response shift analyses using IRT models and allows the investigation of non-uniform and uniform recalibration as well as reprioritization. Differences in RS detection between IRT and SEM may be due to differences between the two methods in handling missing data. However, one cannot conclude on the differences between IRT and SEM based on a single application on a dataset since the underlying truth is unknown. A next step would be to implement a simulation study to investigate those differences.

The minimal clinically important difference determined using item response theory models: an attempt to solve the issue of the association with baseline score

OBJECTIVES Determining the minimal clinically important difference (MCID) of questionnaires on an interval scale, the trait level (TL) scale, using item response theory (IRT) models could overcome its association with baseline severity. The aim of this study was to compare the sensitivity (Se), specificity (Sp), and predictive values (PVs) of the MCID determined on the score scale (MCID-Sc) or the TL scale (MCID-TL). STUDY DESIGN AND SETTING The MCID-Sc and MCID-TL of the MOS-SF36 general health subscale were determined for deterioration and improvement on a cohort of 1,170 patients using an anchor-based method and a partial credit model. The Se, Sp, and PV were calculated using the global rating of change (the anchor) as the gold standard test. RESULTS The MCID-Sc magnitude was smaller for improvement (1.58 points) than for deterioration (-7.91 points). The Se, Sp, and PV were similar for MCID-Sc and MCID-TL in both cases. However, if the MCID was defined on the score scale as a function of a range of baseline scores, its Se, Sp, and PV were consistently higher. CONCLUSION This study reinforces the recommendations concerning the use of an MCID-Sc defined as a function of a range of baseline scores.

Power and Sample Size Determination in the Rasch Model: Evaluation of the Robustness of a Numerical Method to Non-Normality of the Latent Trait

Patient-reported outcomes (PRO) have gained importance in clinical and epidemiological research and aim at assessing quality of life, anxiety or fatigue for instance. Item Response Theory (IRT) models are increasingly used to validate and analyse PRO. Such models relate observed variables to a latent variable (unobservable variable) which is commonly assumed to be normally distributed. A priori sample size determination is important to obtain adequately powered studies to determine clinically important changes in PRO. In previous developments, the Raschpower method has been proposed for the determination of the power of the test of group effect for the comparison of PRO in cross-sectional studies with an IRT model, the Rasch model. The objective of this work was to evaluate the robustness of this method (which assumes a normal distribution for the latent variable) to violations of distributional assumption. The statistical power of the test of group effect was estimated by the empirical rejection rate in data sets simulated using a non-normally distributed latent variable. It was compared to the power obtained with the Raschpower method. In both cases, the data were analyzed using a latent regression Rasch model including a binary covariate for group effect. For all situations, both methods gave comparable results whatever the deviations from the model assumptions. Given the results, the Raschpower method seems to be robust to the non-normality of the latent trait for determining the power of the test of group effect.

Rasch-family models are more valuable than score-based approaches for analysing longitudinal patient-reported outcomes with missing data

The objective was to compare classical test theory and Rasch-family models derived from item response theory for the analysis of longitudinal patient-reported outcomes data with possibly informative intermittent missing items. A simulation study was performed in order to assess and compare the performance of classical test theory and Rasch model in terms of bias, control of the type I error and power of the test of time effect. The type I error was controlled for classical test theory and Rasch model whether data were complete or some items were missing. Both methods were unbiased and displayed similar power with complete data. When items were missing, Rasch model remained unbiased and displayed higher power than classical test theory. Rasch model performed better than the classical test theory approach regarding the analysis of longitudinal patient-reported outcomes with possibly informative intermittent missing items mainly for power. This study highlights the interest of Rasch-based models in clinical research and epidemiology for the analysis of incomplete patient-reported outcomes data.

Power and Sample Size Determination for the Group Comparison of Patient-Reported Outcomes with Rasch Family Models

Background Patient-reported outcomes (PRO) that comprise all self-reported measures by the patient are important as endpoint in clinical trials and epidemiological studies. Models from the Item Response Theory (IRT) are increasingly used to analyze these particular outcomes that bring into play a latent variable as these outcomes cannot be directly observed. Preliminary developments have been proposed for sample size and power determination for the comparison of PRO in cross-sectional studies comparing two groups of patients when an IRT model is intended to be used for analysis. The objective of this work was to validate these developments in a large number of situations reflecting real-life studies. Methodology The method to determine the power relies on the characteristics of the latent trait and of the questionnaire (distribution of the items), the difference between the latent variable mean in each group and the variance of this difference estimated using Cramer-Rao bound. Different scenarios were considered to evaluate the impact of the characteristics of the questionnaire and of the variance of the latent trait on performances of the Cramer-Rao method. The power obtained using Cramer-Rao method was compared to simulations. Principal Findings Powers achieved with the Cramer-Rao method were close to powers obtained from simulations when the questionnaire was suitable for the studied population. Nevertheless, we have shown an underestimation of power with the Cramer-Rao method when the questionnaire was less suitable for the population. Besides, the Cramer-Rao method stays valid whatever the values of the variance of the latent trait. Conclusions The Cramer-Rao method is adequate to determine the power of a test of group effect at design stage for two-group comparison studies including patient-reported outcomes in health sciences. At the design stage, the questionnaire used to measure the intended PRO should be carefully chosen in relation to the studied population.

Validation of the French translation-adaptation of the impact of cancer questionnaire version 2 (IOCv2) in a breast cancer survivor population

BackgroundThe Impact of Cancer version 2 (IOCv2) was designed to assess the physical and psychosocial health experience of cancer survivors through its positive and negative impacts. Although the IOCv2 is available in English and Dutch, it has not yet been validated for use in French-speaking populations. The current study was undertaken to provide a comprehensive assessment of the reliability and validity of the French language version of the IOCv2 in a sample of breast cancer survivors.MethodsAn adapted French version of the IOCv2 as well as demographic and medical information were completed by 243 women to validate the factor structure divergent/divergent validities and reliability. Concurrent validity was assessed by correlating the IOCv2 scales with measures from the SF-12, PostTraumatic Growth Inventory and Fear of Cancer Recurrence Inventory.ResultsThe French version of the IOCv2 supports the structure of the original version, with four positive impact dimensions and four negative impact dimensions. This result was suggested by the good fit of the confirmatory factor analysis and the adequate reliability revealed by Cronbachs alpha coefficients and other psychometric indices. The concurrent validity analysis revealed patterns of association between IOCv2 scale scores and other measures.Unlike the original version, a structure with a Positive Impact domain consisting in the IOCv2 positive dimensions and a Negative Impact domain consisting in the negative ones has not been clearly evidenced in this study. The limited practical use of the conditional dimensions Employment Concerns and Relationship Concerns, whether the patient is partnered or not, did not make possible to provide evidence of validity and reliability of these dimensions as the subsets of sample to work with were not large enough. The scores of these conditional dimensions have to be used with full knowledge of the facts of this limitation of the study.ConclusionsIntegrating IOCv2 into studies will contribute to evaluate the psychosocial health experience of the growing population of cancer survivors, enabling better understanding of the multi-dimensional impact of cancer.

Does the relationship between health-related quality of life and subjective well-being change over time? An exploratory study among breast cancer patients

It has been suggested recently that measures of Subjective Well-Being (SWB) instead of preferences could be employed to determine relative weights for the dimensions of health-related quality of life (HRQol) with the aim of developing health utility indexes for economic evaluation purposes. In this context, this paper addresses the possibility of reprioritization response shift in SWB. It examines whether the association between dimensions of HRQol and SWB changes over time in chronically ill patients. 215 women newly diagnosed for breast cancer in a French hospital between 2010 and 2012 completed the Satisfaction with Life Scale (SWLS) and the EORTC QLQ-C30 HRQol questionnaires over a two-year period. We estimated hierarchical random coefficients models for the repeated SWLS measures while allowing for time-varying parameters for the scales of the QLQ-C30 to test for reprioritization. Our findings suggest that women adapt to breast cancer by giving greater weight over time to the social dimension of HRQol. This possibility of reprioritization response shift should be considered in researches trying to develop SWB-based health utility values to inform the allocation of resources in health care.

Power and sample size determination for group comparison of patient‐reported outcomes using polytomous Rasch models

The analysis of patient-reported outcomes or other psychological traits can be realized using the Rasch measurement model. When the objective of a study is to compare groups of individuals, it is important, before the study, to define a sample size such that the group comparison test will attain a given power. The Raschpower procedure (RP) allows doing so with dichotomous items. The RP is extended to polytomous items. Several computational issues were identified, and adaptations have been proposed. The performance of this new version of RP is assessed using simulations. This adaptation of RP allows obtaining a good estimate of the expected power of a test to compare groups of patients in a large number of practical situations. A Stata module, as well as its implementation online, is proposed to perform the RP. Two versions of the RP for polytomous items are proposed (deterministic and stochastic versions). These two versions produce similar results in all of the tested cases. We recommend the use of the deterministic version, when the measure is obtained using small questionnaires or items with a few number of response categories, and the stochastic version elsewhere, so as to optimize computing time.

Identifying patterns of adaptation in breast cancer patients with cancer-related fatigue using response shift analyses at subgroup level

Fatigue is the most prevalent symptom in breast cancer. It might be perceived differently among patients over time as a consequence of the differing patients’ adaptation and psychological adjustment to their cancer experience which can be related to response shift (RS). RS analyses can provide important insights on patients’ adaptation to cancer but it is usually assumed that RS occurs in the same way in all individuals which is unrealistic. This study aimed to identify patients’ subgroups in which different RS effects on self‐reported fatigue could occur over time using a combination of methods for manifest and latent variables. The FATSEIN study comprised 466 breast cancer patients followed over a 2‐year period. Fatigue was measured with the Multidimensional Fatigue Inventory questionnaire (MFI‐20) during 10 visits. A novel combination of Mixed Models, Growth Mixture Modeling, and Structural Equation Modeling was used to assess the occurrence of RS in fatigue changes to identify subgroups displaying different RS patterns over time. An increase in fatigue was evidenced over the 8‐month follow‐up, followed by a decrease between the 8‐ and 24‐month. Four latent classes of patients were identified. Different RS patterns were detected in all latent classes between the inclusion and 8 months (last cycle of chemotherapy). No RS was evidenced between 8‐ and 24‐month. Several RS effects were evidenced in different groups of patients. Women seemed to adapt differently to their treatment and breast cancer experience possibly indicating differing needs for medical/psychological support.