Rob Arbuckle

The reliability, validity, and preliminary responsiveness of the Eye Allergy Patient Impact Questionnaire (EAPIQ)

BackgroundThe Eye Allergy Patient Impact Questionnaire (EAPIQ) was developed based on a pilot study conducted in the US and focus groups with eye allergy sufferers in Europe. The purpose of this study was to present the results of the psychometric validation of the EAPIQ.MethodsOne hundred forty six patients from two allergy clinics completed the EAPIQ twice over a two-week period during the fall and winter allergy seasons, along with concurrent measures of health status, work productivity, and utility. Construct validity, reliability (internal consistency and test-retest), concurrent, known-group, and clinical validities, and responsiveness of the EAPIQ were assessed. Known-group validity was assessed by comparing EAPIQ scale scores between patients grouped according to their self-rating of ocular allergy severity (no symptoms, very mild, mild, moderate, severe, very severe). Clinical validity was assessed by assessing differences in EAPIQ scores between groups of patients rated by their clinician as non-symptomatic, mild, moderate, and severe.Results and DiscussionResults from the validation study suggested the deletion of 14 of 43 items (including embedded questions) that required patients to complete the percentage of time they were troubled by something (daily activity limitations/emotional troubles). These items yielded a significant amount of missing or inconsistent data (50%). The resulting factor analysis suggested four domains: symptoms, daily life impact, psychosocial impact, and treatment satisfaction. When included as separate scales, the symptom-bother and symptom-frequency scales were highly correlated (> 0.9). As a consequence, and due to superior discriminative validity, the symptom bother and frequency items were summed. All items met the tests for item convergent validity (item-scale correlation = 0.4). The success rate for item discriminant validity testing was 97% (item-scale correlation greater with own scale than with any other). The criterion for internal consistency reliability (alpha coefficient ≥ 0.70) was met for all EAPIQ scales (range 0.89–0.93), as was the criterion for test-retest reliability (intraclass correlation [ICC] ≥ 0.70). Largely moderate correlations between the scales of the EAPIQ and the mini Rhinoconjunctivitis Quality of Life Questionnaire (miniRQLQ) and low correlations with the Health Utilities Index 2/3 (HUI2/3) were indicative of satisfactory concurrent validity. The EAPIQ symptoms, Daily Life Impact, and Psychosocial Impact scales were able to distinguish between patients differing in eye allergy symptom severity, as rated by patients and clinicians, providing evidence of satisfactory known-group and clinical validities, respectively. Preliminary analyses indicated the EAPIQ Symptoms, Daily Life Impact, and Psychosocial Impact scales to be responsive to changes in eye allergies.ConclusionFollowing item reduction, construct validity, reliability, concurrent validity, known-group validity, and preliminary responsiveness were satisfactory for the EAPIQ in this population of ocular allergy patients.

The bladder pain/interstitial cystitis symptom score: development, validation, and identification of a cut score.

BACKGROUND There is a need to develop a self-report measure that reliably identifies moderate to severe bladder pain syndrome (BPS) patients for inclusion into clinical trials to assess the efficacy of new BPS treatments. OBJECTIVE To develop and validate a patient-reported symptom-based instrument, the Bladder Pain/Interstitial Cystitis Symptom Score (BPIC-SS), for clinical trial eligibility of BPS patients. DESIGN, SETTING, AND PARTICIPANTS Stage 1: Qualitative concept elicitation (CE) interviews were conducted with BPS patients in France (n=12), Germany (n=12), and the United States (US) (n=20), and overactive bladder (OAB) (n=10) patients in the US for comparison. Stage 2: Cognitive debriefing (CD) interviews were performed with US BPS patients (n=20). Stage 3: An observational study with 99 BPS, 99 OAB, and 100 healthy participants in the US was used to perform item reduction, identify cut scores, and validate the measure. A cut score was defined using logistic regression and receiver operating characteristic curves. Psychometric properties, including test-retest reliability, were assessed. MEASUREMENTS In addition to the BPIC-SS, the Pelvic Pain and Urgency/Frequency Patient Symptom Scale, the Interstitial Cystitis Symptom Index, a Clinician Global Impression of Severity, and a Patient Global Impression of Change were included in the observational study. RESULTS AND LIMITATIONS In CE, reported symptoms were bladder pain, persistent urge to urinate, and high urinary frequency. In CD, 13 items were deleted, and 15 were retained. Based on validation analyses, qualitative findings, and clinical relevance, the instrument was reduced to eight items that had strong sensitivity (0.72) and specificity (0.86) with a cut score ≥19 to determine clinical trial inclusion. Psychometric properties were strong. CONCLUSIONS The BPIC-SS is a reliable, valid, and appropriate questionnaire to select BPS/interstitial cystitis patients for clinical trials.

“Not Just Little Adults”: Qualitative Methods to Support the Development of Pediatric Patient-Reported Outcomes

The US FDA and the European Medicines Agency (EMA) have issued incentives and laws mandating clinical research in pediatrics. While guidances for the development and validation of patient-reported outcomes (PROs) or health-related quality of life (HRQL) measures have been issued by these agencies, little attention has focused on pediatric PRO development methods. With reference to the literature, this article provides an overview of specific considerations that should be made with regard to the development of pediatric PRO measures, with a focus on performing qualitative research to ensure content validity. Throughout the questionnaire development process it is critical to use developmentally appropriate language and techniques to ensure outcomes have content validity, and will be reliable and valid within narrow age bands (0–2, 3–5, 6–8, 9–11, 12–14, 15–17 years). For qualitative research, sample sizes within those age bands must be adequate to demonstrate saturation while taking into account children’s rapid growth and development. Interview methods, interview guides, and length of interview must all take developmental stage into account. Drawings, play-doh, or props can be used to engage the child. Care needs to be taken during cognitive debriefing, where repeated questioning can lead a child to change their answers, due to thinking their answer is incorrect. For the PROs themselves, the greatest challenge is in measuring outcomes in children aged 5–8 years. In this age range, while self-report is generally more valid, parent reports of observable behaviors are generally more reliable. As such, ‘team completion’ or a parent-administered child report is often the best option for children aged 5–8 years. For infants and very young children (aged 0–4 years), patient rating of observable behaviors is necessary, and, for adolescents and children aged 9 years and older, self-reported outcomes are generally valid and reliable. In conclusion, the development of PRO measures for use in children requires careful tailoring of qualitative methods, and performing research within narrow age bands. The best reporter should be carefully considered dependent on the child’s age, developmental ability, and the concept being measured, and team completion should be considered alongside self-completion and observer measures.

What Outcomes are Important for Gout Patients? In-Depth Qualitative Research into the Gout Patient Experience to Determine Optimal Endpoints for Evaluating Therapeutic Interventions.

Background and ObjectivesCharacterized by sudden onset of severe joint pain, swelling, redness, and tenderness to touch, gout ‘flare ups’ have a substantial impact on quality of life (QoL). This research employed a patient-centered approach to explore the symptoms and impacts of gout, and assess the content validity of existing patient-reported outcomes (PROs).MethodsQualitative interviews were conducted with 30 US gout patients (non-tophaceous: n = 20, tophaceous: n = 10) and five expert rheumatologists. Each interview included both concept elicitation (CE) questioning to learn about the patient experience and cognitive debriefing to assess the content validity of three PRO instruments (HAQ-DI, GAQ, and TIQ-20). Nine of the patients provided further real-time qualitative data through a smart phone application. All qualitative data were subject to thematic analysis using Atlas.ti. Two patient advisors and three expert clinicians were engaged as advisors at key stages throughout the research.ResultsInterview and real-time data identified the same core symptoms and proximal impact concepts. Severe pain (typically in joints of extremities) was described as the cardinal symptom, often accompanied by swelling, redness, heat, sensitivity to touch, and stiffness. Domains of QoL impacted included physical functioning, sleep, daily activities, and work. The PRO instruments were generally well-understood by patients, but each included items with questionable relevance to at least some of the sample, dependent on the specific joints affected.ConclusionsGout patients experience severe pain in affected joints, resulting in substantial limitations in physical functioning. Both the HAQ-DI and the TIQ-20 are useful for specific research purposes in the gout population, although modifications are recommended.

Qualitative cross-cultural exploration of vaginal bleeding/spotting symptoms and impacts associated with hormone therapy in post-menopausal women to inform the development of new patient-reported measurement tools

OBJECTIVES To understand the vaginal bleeding/spotting experiences of postmenopausal (PM) women taking estrogen plus progestin therapies (EPT) and develop measures to assess these symptoms and their impact on womens daily lives in four countries. DESIGN (1) Concept elicitation interviews were conducted with PM women in the US (n=14), Italy (n=15), Mexico (n=15) and China (n=15) to explore vaginal bleeding/spotting symptoms associated with EPT. The Post-Menopausal Bleeding Questionnaire (PMBQ) was also debriefed to evaluate understanding and comprehensiveness. (2) Based on concept elicitation, a single item electronic daily diary was developed and the PMBQ modified to form a 12-item impact measure. (3) The measures were pilot-tested and then cognitively debriefed with US women receiving EPT. All qualitative data was subject to thematic analysis. MAIN OUTCOME MEASURES The Vaginal Bleeding/Spotting Daily Diary, (VBS-DD) and Post-Menopausal Bleeding Impact Questionnaire (PMBIQ) were developed in this study. RESULTS Concept elicitation identified vaginal bleeding and spotting as important symptoms for women taking EPT, impacting their emotional wellbeing, social life, ability to move freely, clothing and sexual activity. Based on pilot testing and cognitive debriefing, women demonstrated good understanding of the VBS-DD and the PMBQ was reduced to 10 items due to conceptual redundancy. CONCLUSIONS Women taking EPT in the US, China, Mexico and Italy reported vaginal bleeding/spotting symptoms that have a detrimental impact on their quality of life. Two new measures were developed to assess the severity and impact of vaginal bleeding/spotting specific to EPT. This work highlights the need for EPT-related symptoms to be a part of treatment decision-making.

Development and preliminary psychometric evaluation of an owner-completed measure of feline quality of life

Due to improved healthcare and pet longevity, measurement of health-related quality of life (HRQoL) is increasingly important in companion animal medicine. The aim of this study was to develop and evaluate the content and psychometric properties of an owner-completed assessment of health and quality of life (QoL) in cats for use in general veterinary clinical practice. A 23-item feline QoL measure, drafted based on findings from an online survey completed by 45 pet owners, was revised following qualitative interviews with 10 pet owners of healthy cats to assess content validity. The resulting 22-item measure was completed twice by 199 owners of healthy cats to assess the reliability and validity of the measure via psychometric evaluation, including assessment of missing data, item response distributions, item correlations, factor analysis, internal consistency, test-retest reliability, multi-trait analysis, known groups analyses and estimation of minimally important differences. There were no missing data. Responses for all items were heavily skewed due to the sample being healthy. Analysis of items and factor analysis supported deletion of six items and calculation of two domain scores and a total score. Internal consistency and test-retest reliability were strong for all domains (0.70-0.80), indicating good reliability. All but three items demonstrated strong item convergent validity (item-scale correlation>0.40) and correlated highest with their respective domain (item discriminant validity). Significant between-group differences in scores differing according to a global impression of feline health item provided evidence of discriminative validity. Findings provide evidence that the final 16-item feline QoL measure has strong cross-sectional psychometric properties.

Psychometric assessment of the IBS-D Daily Symptom Diary and Symptom Event Log

PurposeDiarrhea-predominant irritable bowel syndrome (IBS-D) can considerably impact patients’ lives. Patient-reported symptoms are crucial in understanding the diagnosis and progression of IBS-D. This study psychometrically evaluates the newly developed IBS-D Daily Symptom Diary and Symptom Event Log (hereafter, “Event Log”) according to US regulatory recommendations.MethodsA US-based observational field study was conducted to understand cross-sectional psychometric properties of the IBS-D Daily Symptom Diary and Event Log. Analyses included item descriptive statistics, item-to-item correlations, reliability, and construct validity.ResultsThe IBS-D Daily Symptom Diary and Event Log had no items with excessive missing data. With the exception of two items (“frequency of gas” and “accidents”), moderate to high inter-item correlations were observed among all items of the IBS-D Daily Symptom Diary and Event Log (day 1 range 0.67–0.90). Item scores demonstrated reliability, with the exception of the “frequency of gas” and “accidents” items of the Diary and “incomplete evacuation” item of the Event Log. The pattern of correlations of the IBS-D Daily Symptom Diary and Event Log item scores with generic and disease-specific measures was as expected, moderate for similar constructs and low for dissimilar constructs, supporting construct validity. Known-groups methods showed statistically significant differences and monotonic trends in each of the IBS-D Daily Symptom Diary item scores among groups defined by patients’ IBS-D severity ratings (“none”/“mild,” “moderate,” or “severe”/“very severe”), supporting construct validity.ConclusionsInitial psychometric results support the reliability and validity of the items of the IBS-D Daily Symptom Diary and Event Log.

Cultural adaptation: translatability assessment and linguistic validation of the patient-reported outcome instrument for irritable bowel syndrome with diarrhea

Background and objective Following a 2009 US Food and Drug Administration guidance, a new patient-reported outcome (PRO) instrument was developed to support end points in multinational clinical trials assessing irritable bowel syndrome with diarrhea (IBS-D) symptom severity. Our objective was to assess the translatability of the IBS-D PRO instrument into ten languages, and subsequently perform a cultural adaptation/linguistic validation of the questionnaire into Japanese and US Spanish. Materials and methods Translatability assessments of the US English version of the IBS-D PRO were performed by experienced PRO translators who were native speakers of each target language and currently residing in target-language countries. Languages were Chinese (People’s Republic of China), Dutch (the Netherlands), French (Belgium), German (Germany), Japanese (Japan), Polish (Poland), Portuguese (Brazil), Russian (Russia), Spanish (Mexico), and Spanish (US). The project team assessed the instrument to identify potential linguistic and/or cultural adaptation issues. After the issues identified were resolved, the instrument was translated into Spanish (US) and Japanese through a process of two forward translations, one reconciled translation, and one backward translation. The project team reviewed the translated versions before the instruments were evaluated by cognitive debriefing interviews with samples of five Spanish (US) and five Japanese IBS-D patients. Results Linguistic and cultural adaptation concerns identified during the translatability assessment required minor revisions, mainly the presentation of dates/times and word structure. During the cognitive debriefing interviews, two of five Spanish respondents misunderstood the term “bowel movement” to mean only diarrhea in the Spanish version. Consequently, the term was changed from “movimiento intestinal” to “evacuaciones”. None of the Japanese respondents identified issues with the Japanese version. Conclusion The translatability of the IBS-D PRO instrument into ten target languages was confirmed, with only minor changes made to the translations of the instrument. The translation and linguistic validation into Spanish (US) and Japanese provide evidence that this instrument can be used in multinational trials and clinical settings.

Qualitative cross-cultural exploration of breast symptoms and impacts associated with hormonal treatments for menopausal symptoms to inform the development of new patient-reported measurement tools

To explore cross-cultural experiences of women taking estrogen plus progestin therapies (EPT) and develop a symptom-based electronic diary and impact questionnaire for EPT-related breast symptoms. (1) Concept elicitation interviews were conducted with women in the US (n=14), Italy (n=15), Mexico (n=15) and China (n=15) to explore breast symptoms associated with EPT. Patients completed the Breast Sensitivity Questionnaire (BSQ) to evaluate understanding and comprehensiveness. (2) Based on concept elicitation, a 6-item eDiary (Breast Pain/Tenderness Daily Diary - BPT-DD) was generated and the BSQ modified forming the 18-item Breast Sensations Impact Questionnaire (BSIQ). (3) The measures were pilot-tested and then cognitively debriefed with US women receiving EPT. All qualitative data was subject to thematic analysis. Concept elicitation identified breast pain/tenderness, swollen breasts and sensitivity to contact as important symptoms, impacting womens emotional well-being, relationships with family/friends, social life, sleep, ability to move freely, contact, clothing and sexual activity. Experiences were relatively consistent across the country samples. Based on pilot testing and cognitive debriefing, the BPT-DD was reduced to 4 items (and renamed the Breast Pain Daily Diary - BP-DD) and the BSIQ was reduced to 13 items due to conceptual redundancy. Women taking EPT in the US, China, Mexico and Italy reported breast sensations that have a detrimental impact on quality of life. Two new measures were developed to assess the severity and impact of breast pain specific to EPT. This work highlights that EPT-related symptoms should be part of treatment decision-making, and treatments with less burdensome side effects are needed.

Development and preliminary evidence of the psychometric properties of the GNE myopathy functional activity scale

AIM GNE myopathy, a rare, severe, progressive myopathy, presents with lower extremity distal muscle weakness. The GNE myopathy functional activity scale (GNEM-FAS) evaluates the impact of GNE myopathy on functioning in adults. This paper presents the psychometric validation of the GNEM-FAS. PATIENTS & METHODS Validation of the GNEM-FAS was performed using data from a randomized, double-blind, placebo-controlled Phase-II study (n = 46). RESULTS Domain score distributions were acceptable. Moderate inter-item correlations (typical range, 0.40-0.70), strong item convergent and discriminant validity and high internal consistency reliability (α = 0.88-0.92) supported the instrument structure. Test-retest reliability was strong (ICC range: 0.87-0.95). Scale scores distinguished among subjects with differing disease severity (p < 0.05). CONCLUSION This study provides preliminary evidence of the GNEM-FAS as a valid, reliable assessment.