Christine Cheng

Psychometric properties of the PROMIS physical function item bank in patients with spinal disorders.

Study Design. Patient-reported outcomes provide vital information when assessing effectiveness of clinical care. Yet, most patient-reported outcome instruments are limited by lack of validation and reliability to measure PF adequately. As part of the Patient-Reported Outcomes Measurement Information System (PROMIS), a PF item bank consisting of 124 items has been developed. Objective. There is validation evidence for the PROMIS PF item bank in the general orthopedic patient population in general, but has yet to be validated in the patient with spinal disorders. This study aims to evaluate the psychometric properties of the PROMIS PF item bank specifically for patients presenting with spine-related complaints. Summary of Background Data. Data were collected from adult patients visiting a university spine clinic for back and neck problems. All patients older than 18 years were eligible to participate. A total of 438 patients (49% male) were enrolled in this prospective study. Patients were 18- to 89-year old and presented with back problems (n = 286) and neck problems (n = 152). All patients were administered a 131 item questionnaire. Methods. Conventional descriptive statistics such as means, standard deviations, and proportions were conducted to examine patient characteristics. A Rasch model was used to examine the psychometric properties of the instrument including dimensionality, floor/ceiling effects, reliabilities, and item bias. Results. Results showed that a single PF dimension was supported by the data (i.e., unexplained variance was 2.9%). The instrument had 1.7% ceiling effect and 0.2% floor effect. Item reliability was 1.00 and person reliability was 0.99. We found evidence of item response bias associated with sex, age, and education in some items. Conclusion. The PROMIS PF item bank adequately addressed outcomes of patients with spinal disorders as reliabilities were excellent, minimal ceiling/floor effect existed, and item bias was limited. Future effort should be focused on eliminating, rescaling, or modifying those items that had item bias. Level of Evidence: 2

Psychometric Comparison of the PROMIS Physical Function CAT With the FAAM and FFI for Measuring Patient-Reported Outcomes

Background: Selecting optimal patient-reported outcome (PRO) instruments is critical to improving the quality of health care. The purpose of this study was to compare the reliability, responsiveness, and efficiency of three PRO measures: the Foot and Ankle Ability Measure–Activity of Daily Living subscale (FAAM_ADL), the Foot Function Index 5-point verbal rating scale (FFI-5pt), and the PROMIS Physical Function computerized adaptive test (PF CAT). Methods: Data were aggregated from 10 clinical sites in the AOFAS’s National Orthopaedic Foot and Ankle Research (OFAR) Network from 311 patients who underwent elective surgery for a disorder of the foot or ankle. Patients were administered the FAAM_ADL, FFI-5pt, and PF CAT at their preoperative visit and at 6 months after surgery. Reliabilities were evaluated using a Rasch model. Responsiveness was calculated using paired samples t test and efficiency was recorded as number of seconds to complete the instrument. Results: Similar reliabilities were found for the three instruments. Item reliabilities for FAAM_ADL, FFI-5pt, and PF CAT were all .99. Pearson reliabilities for FAAM_ADL, FFI-5pt, and PF CAT were .95, .93, and .96, respectively. On average, patients completed the FAAM_ADL in 179 seconds, the FFI-5pt in 194 seconds, and the PF CAT in 44 seconds, (P < .001). The PF CAT and FAAM_ADL showed significant improvement (P = .01 and P = .001, respectively) in patients’ physical function after treatment; the FFI-5pt did not show improvement. Conclusions: Overall, the PF CAT performed best in terms of reliability, responsiveness, and efficiency in this broad sample of foot and ankle patients. It can be a potential replacement for the conventional PRO measures, but further validation is needed in conjunction with the PROMIS Pain instruments. Level of Evidence: Level I, prospective comparative outcome study.

Time for a paradigm shift with computerized adaptive testing of general physical function outcomes measurements.

Background: Patient-reported outcomes (PRO) are critical to understanding the value of orthopedic treatments. We hypothesized that use of the computerized adaptive testing from a well-characterized physical function item bank would show superiority in assessing all levels of physical function compared to current standard generic physical function outcomes instruments for foot and ankle patients. Methods: In a population of 126 foot and ankle patients we compared the psychometric properties for three PROs: the Physical Function subscale of the Medical Outcomes General Health Survey (SF-36 PF) version 2, the Patient-Reported Outcomes Measurement Information System (PROMIS) Physical Function (PF) Computerized Adaptive Test (CAT), and the Lower Extremity (LE) CAT. A Rasch item response theory (IRT) model was applied to assess and compare the fit, dimensionality, reliability, validity, and coverage. Results: The unexplained variance for the PF CAT was 3.9% and the LE CAT was 2.1%, suggesting each instrument explained a single concept. The SF-36 PF had more concerning unexplained variance of 7.6%. We found no floor or ceiling effects for the PF CAT, a minimal floor effect (1.6%) but no ceiling effect for the LE CAT, and an 11.1% floor effect and 9.5% ceiling effect for the SF-36 PF. Conclusion: Foot and ankle clinicians and researchers interested in measuring patient perceived functional outcomes with a generic instrument should consider using either the PF CAT or the LE CAT rather than the SF-36 PF. Further studies comparing these CATs to anatomic specific instruments are needed. Level of Evidence: Level I, diagnostic study.

Psychometric Evaluation of the Lower Extremity Computerized Adaptive Test, the Modified Harris Hip Score, and the Hip Outcome Score

Background: The applicability and validity of many patient-reported outcome measures in the high-functioning population are not well understood. Purpose: To compare the psychometric properties of the modified Harris Hip Score (mHHS), the Hip Outcome Score activities of daily living subscale (HOS-ADL) and sports (HOS-sports), and the Lower Extremity Computerized Adaptive Test (LE CAT). The hypotheses was that all instruments would perform well but that the LE CAT would show superiority psychometrically because a combination of CAT and a large item bank allows for a high degree of measurement precision. Study Design: Cohort study (diagnosis); Level of evidence, 2. Methods: Data were collected from 472 advanced-age, active participants from the Huntsman World Senior Games in 2012. Validity evidences were examined through item fit, dimensionality, monotonicity, local independence, differential item functioning, person raw score to measure correlation, and instrument coverage (ie, ceiling and floor effects), and reliability evidences were examined through Cronbach alpha and person separation index. Results: All instruments demonstrated good item fit, unidimensionality, monotonicity, local independence, and person raw score to measure correlations. The HOS-ADL had high ceiling effects of 36.02%, and the mHHS had ceiling effects of 27.54%. The LE CAT had ceiling effects of 8.47%, and the HOS-sports had no ceiling effects. None of the instruments had any floor effects. The mHHS had a very low Cronbach alpha of 0.41 and an extremely low person separation index of 0.08. Reliabilities for the LE CAT were excellent and for the HOS-ADL and HOS-sports were good. Conclusion: The LE CAT showed better psychometric properties overall than the HOS-ADL, HOS-sports, and mHHS for the senior population. The mHHS demonstrated pronounced ceiling effects and poor reliabilities that should be of concern. The high ceiling effects for the HOS-ADL were also of concern. The LE CAT was superior in all psychometric aspects examined in this study. Future research should investigate the LE CAT for wider use in different populations.

Challenging the Norm: Further Psychometric Investigation of the Neck Disability Index

BACKGROUND CONTEXT The neck disability index (NDI) was the first patient-reported outcome (PRO) instrument specific to patients with neck pain, and it remains one of the most widely used PROs for the neck population. The NDI is an appealing measure as it is a short and well-known PRO measure. Currently, there are conflicting data on the performance and applicability of the NDI in patients undergoing either operative or nonoperative treatment for neck-related conditions. PURPOSE This study investigates the psychometric properties, performance, and applicability of the NDI in the spine patient population. STUDY DESIGN A total of 865 patients visiting a university-based spine clinic with neck complaints, with or without radiating upper extremity pain, numbness, or weakness were enrolled in the study. Visit types included new and follow-up visits to both operative and nonoperative treatments. Questionnaires were administered electronically on a tablet computer, and all patients answered all 10 questions of the NDI. METHODS Standard descriptive statistics were performed to describe the demographic characteristics of the patients. Rasch modeling was applied to examine the psychometric properties of the NDI. RESULTS The NDI demonstrated insufficient unidimensionality (ie, unexplained variance after accounting for the first dimension=9.4%). Person reliability was 0.85 and item reliability was 1.00 for the NDI. The overall item fit for the NDI was good with an outfit mean square of 1.03. The NDI had a floor effect of 35.5% and ceiling effect of 4.6%. The raw score to measure correlation of the NDI was 0.019. CONCLUSIONS Although the NDI had good person and item reliability, it did not demonstrate strong evidence of unidimensionality. The NDI exhibited a very large floor effect. Because of the poor raw score to measure correlation, the sum score should not be used in interpretation of findings. Despite great investment by physicians and other stakeholders in the NDI, this evaluation and previous research have demonstrated that the NDI needs further investigation and refinement.

A Normative Dataset of the Balance Error Scoring System in Children Aged Between 5 and 14

Objective:Pediatric head injuries occur commonly and are being reported in increasing numbers. Balance testing is a key component in the evaluation of suspected concussion, and the balance error scoring system (BESS) is likely the most well-known and widely used measure. To date, normative BESS scores for adults have been reported but not for children. Design:Normative data for BESS scores and modified BESS scores were created in a cohort of healthy children. Potential variables were analyzed as predictors of BESS performance. Setting:Local elementary and junior high schools. Participants:A total of 373 healthy children between the ages of 5 and 14. Interventions:The BESS was performed on all children. Assessment of Risk Factors:Gender, body mass index percentile, previous concussions, athletic participation, age, and the parental opinion of childs balance ability were examined as factors associated with the BESS score. Main Outcome Measures:BESS scores. Results:Normative data are reported, stratified by age groups of 5 to 7 years, 8 to 10 years, and 11 to 14 years of age, for both BESS and modified BESS. Median BESS scores are 23 for children aged 5 to 7, 18 for children aged 8 to 10, and 16 for children aged 11 to 14. Median modified BESS scores are 8 for children age 5 to 7, 5 for children age 8 to 10, and 4 for children age 11 to 14. Increasing age and positive parental opinion regarding their childs balance ability were independently correlated with decreasing BESS scores (P < 0.01). Conclusions:The normative data on BESS scores for healthy children reported here provide age-stratified reference values for suspected balance alterations.

Patient-Reported Outcomes and Total Health Care Expenditure in Prediction of Patient Satisfaction: Results From a National Study.

Background Health care quality is often linked to patient satisfaction. Yet, there is a lack of national studies examining the relationship between patient satisfaction, patient-reported outcomes, and medical expenditure. Objective The aim of this study is to examine the contribution of physical health, mental health, general health, and total health care expenditures to patient satisfaction using a longitudinal, nationally representative sample. Methods Using data from the 2010-2011 Medical Expenditure Panel Survey, analyses were conducted to predict patient satisfaction from patient-reported outcomes and total health care expenditures. The study sample consisted of adult participants (N=10,157), with sampling weights representative of 233.26 million people in the United States. Results The results indicated that patient-reported outcomes and total health care expenditure were associated with patient satisfaction such that higher physical and mental function, higher general health status, and higher total health care expenditure were associated with higher patient satisfaction. Conclusions We found that patient-reported outcomes and total health care expenditure had a significant relationship with patient satisfaction. As more emphasis is placed on health care value and quality, this area of research will become increasingly needed and critical questions should be asked about what we value in health care and whether we can find a balance between patient satisfaction, outcomes, and expenditures. Future research should apply big data analytics to investigate whether there is a differential effect of patient-reported outcomes and medical expenditures on patient satisfaction across different medical specialties.

Predicting the DRAM mZDI using the PROMIS anxiety and depression.

Study Design. Data were collected at a university orthopedic clinic in 2012. The final sample consisted of 316 patients. Sample included new and annual follow-up adult patients for all operative and nonoperative spine care. Outcome measures consisted of the patient-reported outcome measurement information system (PROMIS) anxiety 4-item short form (SF-4), the PROMIS depression SF-4, and the distress and risk assessment method modified Zung Depression Index (mZDI). Objective. The objective of this study was to assess whether the PROMIS anxiety 4-item short form (anxiety SF-4) or the PROMIS depression SF-4 (depression SF-4) can be used in place of the distress and risk assessment method mZDI. Summary of Background Data. Studies comparing the PROMIS short forms with the existing instruments are limited. In particular, there is not any published study comparing the PROMIS anxiety and depression short forms with the distress and risk assessment method mZDI. Methods. Regression analyses were carried out to predict the mZDI total scores from the PROMIS anxiety SF-4 T scores and the PROMIS depression SF-4 T scores. Intraclass correlation coefficients were computed on the actual and predicted mZDI scores. Results. All 3 instruments were highly correlated with each other. The PROMIS anxiety SF-4 and the PROMIS depression SF-4 were both able to explain a significant amount of variance in the mZDI. The intraclass correlation coefficients for the actual and predicted mZDI scores were high. The actual mZDI scores and predicted mZDI scores using either the PROMIS anxiety SF-4 or the PROMIS depression SF-4 across age and sex were similar. Conclusion. This study indicates that the PROMIS anxiety SF-4 scores and the depression SF-4 scores can accurately predict the mZDI scores. The PROMIS anxiety and depression short forms can be used as surrogates for the mZDI. Level of Evidence: 3

The Factor Structure of the Hospital Anxiety and Depression Scale in Orthopedic Trauma Patients

Background Many instruments exist to assess mental disorders and anxiety, such as the hospital anxiety and depression scale (HADS). Nothing has been evaluated on the HADS factor structure for use with orthopedic trauma patients. The aim of this study was to validate the underlying structure of the HADS. Specifically, we sought to understand which of the factor structures found in the literature is appropriate for the orthopedic trauma patient population. Methods This study included 348 patients with an average age of 49.8 years (SD: 18.4; range: 18 - 95). Confirmatory data analysis was performed to analyze the latent structure of the HADS. Akaike information criterion (AIC) was used to compare all the models, with the lowest AIC being the best fitting model. Results We found that both the anxiety and the depression factors were highly correlated (with Pearson correlations greater than 0.700). After removing one item from each subscale, we found that a two-factor model was the best fitting one (AIC: 8,298.901); all other models had an AIC over 10,000. Conclusion Our results support a satisfactory two-factor structure for the HADS in the orthopedic trauma patients. Further studies are needed to test for higher factor structures in larger samples and in a different population.