Antoine Regnault

Sample size used to validate a scale: a review of publications on newly-developed patient reported outcomes measures

PurposeNew patient reported outcome (PRO) measures are regularly developed to assess various aspects of the patients’ perspective on their disease and treatment. For these instruments to be useful in clinical research, they must undergo a proper psychometric validation, including demonstration of cross-sectional and longitudinal measurement properties. This quantitative evaluation requires a study to be conducted on an appropriate sample size. The aim of this research was to list and describe practices in PRO and proxy PRO primary psychometric validation studies, focusing primarily on the practices used to determine sample size.MethodsA literature review of articles published in PubMed between January 2009 and September 2011 was conducted. Three selection criteria were applied including a search strategy, an article selection strategy, and data extraction. Agreements between authors were assessed, and practices of validation were described.ResultsData were extracted from 114 relevant articles. Within these, sample size determination was low (9.6%, 11/114), and were reported as either an arbitrary minimum sample size (n = 2), a subject to item ratio (n = 4), or the method was not explicitly stated (n = 5). Very few articles (4%, 5/114) compared a posteriori their sample size to a subject to item ratio. Content validity, construct validity, criterion validity and internal consistency were the most frequently measurement properties assessed in the validation studies.Approximately 92% of the articles reported a subject to item ratio greater than or equal to 2, whereas 25% had a ratio greater than or equal to 20. About 90% of articles had a sample size greater than or equal to 100, whereas 7% had a sample size greater than or equal to 1000.ConclusionsThe sample size determination for psychometric validation studies is rarely ever justified a priori. This emphasizes the lack of clear scientifically sound recommendations on this topic. Existing methods to determine the sample size needed to assess the various measurement properties of interest should be made more easily available.

Validation of the Treatment Satisfaction Questionnaire for Medication in patients with cystic fibrosis

BACKGROUND Our objective was to confirm the measurement properties of the Treatment Satisfaction Questionnaire with Medication (TQSM) in patients with cystic fibrosis (CF) receiving inhaled antibiotics. METHODS The TSQM was included in the EAGER study, a clinical trial comparing a nebulized and a dry powder device for inhaled tobramycin in a CF population with chronic Pseudomonas aeruginosa (Pa) lung infection, aged 6 years and above (N=553). Reliability and validity of the questionnaire were investigated using Cronbachs α and multitrait-multimethod approach. RESULTS The TSQM demonstrated very good reliability and construct validity: all Cronbachs α were above 0.86 and all items met the convergent and discriminant validity criteria. In multivariate regressions, higher patient satisfaction and lower perceived impact of side-effects were associated with better treatment compliance. CONCLUSIONS The TSQM showed very good measurement properties that strongly support its use to assess satisfaction of patients with CF taking inhaled antibiotics.

Impact on parents of bronchiolitis hospitalization of full-term, preterm and congenital heart disease infants

BackgroundThe objective of this work was to explore the impact on parents of the bronchiolitis hospitalization of their infant using the Impact of Bronchiolitis Hospitalization Questionnaire (IBHQ©).MethodsFour hundred sixty-three infants aged less than 1 year and hospitalized for bronchiolitis were included in a French observational study during the 2008–2009 season. Parents were asked to complete the IBHQ at hospital discharge and 3 months later. IBHQ scores, ranging from 0 (no impact) to 100 (highest impact), were compared according to gestational age (full-term, 33–36 wGA, ≤ 32 wGA) and the presence of congenital heart disease (CHD). The potential drivers of impact were explored using multivariate linear regressions.ResultsThe study included 332 full-terms, 71 infants born at 33–36 wGA, and 60 at ≤ 32 wGA; 28 infants had a CHD. At hospital discharge, 9 of the 12 IBHQ mean scores were above 40, indicating a marked impact on parents. Three months later, all mean scores were lower but 5 were still greater than 40. At discharge, the length of hospitalization had a significant effect on IBHQ worries and distress, fear for future, guilt and impact on daily organization scores (p<0.01); the parents’ educational level had a significant effect on IBHQ worries and distress, fear for future, impact on daily organization and financial impact scores (p<0.05). The only statistically significant difference found between the parents of preterm and full-term infants was for the physical impact score at discharge (p=0.004).ConclusionsBronchiolitis hospitalization has conspicuous emotional, physical and organizational consequences on parents and siblings, which persist 3 months after hospital discharge. The main drivers of the impact were length of hospital stay and parents’ educational level, while infants’ gestational age or the presence of a CHD had little influence.

Development and psychometric validation of measures to assess the impact of phenylketonuria and its dietary treatment on patients' and parents' quality of life: the phenylketonuria - quality of life (PKU-QOL) questionnaires.

BackgroundThe aim of our study was to develop and validate the first set of PKU-specific Health-related Quality of Life (HRQoL) questionnaires that: 1) were developed for patients with PKU and their parents, 2) cover the physical, emotional, and social impacts of PKU and its treatment on patients’ lives, 3) are age specific (Child PKU-QOL, Adolescent PKU-QOL, Adult PKU-QOL), 4) enable the evaluation of the HRQoL of children by their parents (Parent PKU-QOL), and 5) have been cross-culturally adapted for use in seven countries (i.e. France, Germany, Italy, The Netherlands, Spain, Turkey and the UK).MethodsThe PKU-QOL questionnaires were developed according to reference methods including patients’, parents’ and healthcare professionals’ interviews; testing in a pilot study (qualitative step in six countries), and linguistic validation of the finalised pilot versions in Turkish. For finalisation and psychometric validation, the pilot versions were included in a multicentre, prospective, non-interventional, observational study conducted in 34 sites in France, Germany, Italy, The Netherlands, Spain, Turkey and the UK. Iterative multi-trait analyses were conducted. Psychometric properties were assessed (concurrent and clinical validity, internal consistency reliability and test-retest reliability).ResultsData from 559 subjects (306 patients, 253 parents) were analysed. After finalisation, the PKU-QOL questionnaires included 40 items (Child PKU-QOL), 58 items (Adolescent PKU-QOL), 65 items (Adult PKU-QOL) and 54 items (Parent PKU-QOL), distributed in four modules: PKU symptoms, PKU in general, administration of Phe-free protein supplements and dietary protein restriction. The measurement properties of the Adolescent, Adult and Parent PKU-QOL questionnaires were overall fairly satisfactory, but weaker for the Child questionnaire.ConclusionsThe four PKU-QOL questionnaires developed for different ages (Child PKU-QOL, Adolescent PKU-QOL, Adult PKU-QOL), and for parents of children with PKU (Parent PKU-QOL) are valid and reliable instruments for assessing the multifaceted impact of PKU on patients of different age groups (children, adolescents and adults) and their parents, and are available for use in seven countries. They are very promising tools to explore how patients’ perceptions evolve with age, to increase knowledge of the impact of PKU on patients and parents in different countries, and to help monitor the effect of therapeutic strategies.

Assessing the impact on caregivers of patients with schizophrenia: psychometric validation of the Schizophrenia Caregiver Questionnaire (SCQ)

BackgroundThe Schizophrenia Caregiver Questionnaire (SCQ) was developed to assess the impact on caregivers of caring for patients with schizophrenia. The objective of this study was to develop a scoring algorithm for the SCQ, and evaluate its measurement properties.MethodsThe SCQ was administered to 358 caregivers of patients with schizophrenia included in the observational PATTERN study of stabilized patients with persistent symptoms of schizophrenia receiving outpatient care. SCQ item selection and creation of scores were based on exploration of item response distribution, factor analyses, and Rasch model. Construct validity, reliability, and ability to detect change of the SCQ scores were investigated.ResultsThe final questionnaire comprised a ‘Humanistic impact’ supra-domain composed of a global score and four subdomain scores (‘Physical’; ‘Emotional’; ‘Social’; ‘Daily life’), and eight other domain scores related to the caregiving role (‘Exhaustion with caregiving’; ‘Feeling alone’; ‘Patient Dependence’; ‘Worries for the patient’; ‘Perception of caregiving’; ‘Financial dependence of the patient’; ‘Financial impact of caregiving’; ‘Overall difficulty of caregiving’). Two items from the SCQ were deleted. SCQ scores showed very good construct validity: Item convergent/discriminant validity were satisfactory; SCQ scores of caregivers of patients with more severe symptoms were higher indicating more impact (p < 0.05 for all scores); SCQ scores were meaningfully associated with measures of schizophrenia severity (PANSS and PSP) and caregivers’ Health-Related Quality of Life (Medical Outcome Survey Short Form 36 items). The SCQ Humanistic impact supra-domain scores demonstrated very good internal consistency reliability (Cronbach’s alphas between 0.80 and 0.96) and test-retest reliability (Intraclass Coefficient correlations ranging from 0.75 and 0.87); Other SCQ domain scores showed lower but still acceptable reliability coefficients. SCQ scores clearly increased for caregivers of patients whose schizophrenia worsened.ConclusionsOverall, the 30-item SCQ demonstrated very good measurement properties supporting its relevance to comprehensively measure the experience of caregivers of patients with schizophrenia.

332* Validation of the treatment satisfaction questionnaire for medication (TSQM) in cystic fibrosis (CF)

Objectives: To assess the measurement properties of the TSQM, a 14-item questionnaire measuring 4 dimensions related to treatment satisfaction (Effectiveness; Side-effects; Convenience; Global satisfaction) in CF. Methods: The TSQM is a generic treatment satisfaction instrument validated in several chronic diseases but not used before in CF. It was included in the EAGER study, a randomized open-label clinical trial comparing a nebulized and a dry powder device for inhaled tobramycin in a CF population aged 6 years and above. The two treatment arms (N= 553) were pooled together to conduct psychometric analyses. Construct validity was assessed using confirmatory factor analysis (CFA) and multi-trait analysis. Cronbach’s a were calculated to assess internal consistency reliability. Differential item functioning between responders who were assisted by a relative (patients below 12 years of age) and those who were not was sought for using logistic regression. Results: The quality of completion of the 14 TSQM items was excellent since all had less than 2% missing data. The CFA goodness-of-fit of the 4-dimension structure met commonly used standards (e.g. Goodness of Fit index = 0.92). In the multitrait analysis, all items met the convergent validity criterion and all but one met the discriminant validity. All Cronbach’s a were above 0.86, indicating very good reliability for all 4 dimensions. No item showed differential functioning related to mode of administration (alone vs. with the assistance of a relative). Conclusion: The TSQM showed excellent measurement properties that strongly support the use of this instrument to assess patient satisfaction with inhaled CF treatments.

Patient-reported outcomes in drug development for hematology

Patient-reported outcomes (PROs) are any outcome evaluated directly by the patient himself and based on the patients perception of a disease and its treatment(s). PROs are direct outcome measures that can be used as clinical meaningful endpoints to characterize treatment benefit. They provide unique and important information about the effect of treatment from a patients view. However, PROs will only be considered adequate if the assessment is well-defined and reliable. In 2009, the FDA has issued a guidance, which defines good measurement principles to consider for PRO measures intended to give evidence of treatment benefit in drug development. In hematologic clinical trials, when applied rigorously, they may be used to evaluate overall treatment effectiveness, treatment toxicity, and quality of patients well-being at short-term and long-term after treatment from a patients perspective. In situations in which multiple treatment options exist with similar survival outcome or if a new therapeutic strategy needs to be evaluated, the inclusion of PROs as an endpoint can provide additional data and help in clinical decision making. Given the diversity of the hematological field, the approach to measurement needs to be tailored for each specific situation. The importance of PROs in hematologic diseases has been highlighted in a number of international recommendations. In addition, new perspectives in the regulatory field will enhance the inclusion of PRO endpoints in clinical trials in hematology, allowing the voice of the patients with hematologic diseases to be taken into greater consideration in the development of new drugs.