Man Hung

Evaluation of the PROMIS physical function item bank in orthopaedic patients

The patient‐reported outcomes measurement information system (PROMIS) physical function item bank v1 (PPFIB) contains 124 item response theory (IRT) calibrated items (Rose et al. 2008. J Clin Epidemiol 61:17–33). We report the psychometric properties of these items within an outpatient, orthopaedic patient population. In particular, we investigated whether a single unidimensional IRT scale can adequately define physical function of patients presenting with primarily upper or lower extremity orthopaedic complaints. We conducted a prospective study at an orthopaedic outpatient clinic to collect data from 865 adult patients with all 124 PROMIS physical function items and seven demographic items. Items were evaluated by a Rasch model. Total variance (60.6%) across the 124 items was explained by a single Rasch dimension. The variance explained by the second dimension was 7.7%, reflecting differential item functioning in the upper and lower extremity patients. The upper extremity physical function items had a pronounced ceiling effect. A single physical function dimension accounts for most of the item variance in the PPFIB, suggesting that the items are measuring predominately one single construct. Separate subscales for lower versus upper extremities, especially with additional items at the upper trait level of the upper extremity subscale, may further enhance evaluation of physical function in orthopaedic patients.

Computerized Adaptive Testing Using the PROMIS Physical Function Item Bank Reduces Test Burden With Less Ceiling Effects Compared With the Short Musculoskeletal Function Assessment in Orthopaedic Trauma Patients

Purpose: Patient-reported outcomes are important to assess effectiveness of clinical interventions. For orthopaedic trauma patients, the short Musculoskeletal Function Assessment (sMFA) is a commonly used questionnaire. Recently, the Patient-Reported Outcome Measurement Information System (PROMIS) PF Function Computer Adaptive Test (PF CAT) was developed using item response theory to efficiently administer questions from a calibrated bank of 124 PF questions using computerized adaptive testing. In this study, we compared the sMFA versus the PROMIS PF CAT for trauma patients. Methods: Orthopaedic trauma patients completed the sMFA and the PROMIS PF CAT on a tablet wirelessly connected to the PROMIS Assessment Center. The time for each test administration was recorded. A 1-parameter item response theory model was used to examine the psychometric properties of the instruments, including precision and floor/ceiling effects. Results: One hundred fifty-three orthopaedic trauma patients participated in the study. Mean test administration time for PROMIS PF CAT was 44 seconds versus 599 seconds for sMFA (P < 0.05). Both instruments showed extremely high item reliability (Cronbach alpha = 0.98). In terms of instrument coverage, neither instrument showed any floor effect; however, the sMFA revealed 14.4% ceiling effect, whereas the PROMIS PF CAT had no appreciable ceiling effect. Conclusions: Administered by electronic means, the PROMIS PF CAT required less than one-tenth the amount of time for patients to complete than the sMFA while achieving equally high reliability and less ceiling effects. The PROMIS PF CAT is a very attractive and innovative method for assessing patient-reported outcomes with minimal burden to patients.

Psychometric properties of the PROMIS physical function item bank in patients with spinal disorders.

Study Design. Patient-reported outcomes provide vital information when assessing effectiveness of clinical care. Yet, most patient-reported outcome instruments are limited by lack of validation and reliability to measure PF adequately. As part of the Patient-Reported Outcomes Measurement Information System (PROMIS), a PF item bank consisting of 124 items has been developed. Objective. There is validation evidence for the PROMIS PF item bank in the general orthopedic patient population in general, but has yet to be validated in the patient with spinal disorders. This study aims to evaluate the psychometric properties of the PROMIS PF item bank specifically for patients presenting with spine-related complaints. Summary of Background Data. Data were collected from adult patients visiting a university spine clinic for back and neck problems. All patients older than 18 years were eligible to participate. A total of 438 patients (49% male) were enrolled in this prospective study. Patients were 18- to 89-year old and presented with back problems (n = 286) and neck problems (n = 152). All patients were administered a 131 item questionnaire. Methods. Conventional descriptive statistics such as means, standard deviations, and proportions were conducted to examine patient characteristics. A Rasch model was used to examine the psychometric properties of the instrument including dimensionality, floor/ceiling effects, reliabilities, and item bias. Results. Results showed that a single PF dimension was supported by the data (i.e., unexplained variance was 2.9%). The instrument had 1.7% ceiling effect and 0.2% floor effect. Item reliability was 1.00 and person reliability was 0.99. We found evidence of item response bias associated with sex, age, and education in some items. Conclusion. The PROMIS PF item bank adequately addressed outcomes of patients with spinal disorders as reliabilities were excellent, minimal ceiling/floor effect existed, and item bias was limited. Future effort should be focused on eliminating, rescaling, or modifying those items that had item bias. Level of Evidence: 2

Psychometric Comparison of the PROMIS Physical Function CAT With the FAAM and FFI for Measuring Patient-Reported Outcomes

Background: Selecting optimal patient-reported outcome (PRO) instruments is critical to improving the quality of health care. The purpose of this study was to compare the reliability, responsiveness, and efficiency of three PRO measures: the Foot and Ankle Ability Measure–Activity of Daily Living subscale (FAAM_ADL), the Foot Function Index 5-point verbal rating scale (FFI-5pt), and the PROMIS Physical Function computerized adaptive test (PF CAT). Methods: Data were aggregated from 10 clinical sites in the AOFAS’s National Orthopaedic Foot and Ankle Research (OFAR) Network from 311 patients who underwent elective surgery for a disorder of the foot or ankle. Patients were administered the FAAM_ADL, FFI-5pt, and PF CAT at their preoperative visit and at 6 months after surgery. Reliabilities were evaluated using a Rasch model. Responsiveness was calculated using paired samples t test and efficiency was recorded as number of seconds to complete the instrument. Results: Similar reliabilities were found for the three instruments. Item reliabilities for FAAM_ADL, FFI-5pt, and PF CAT were all .99. Pearson reliabilities for FAAM_ADL, FFI-5pt, and PF CAT were .95, .93, and .96, respectively. On average, patients completed the FAAM_ADL in 179 seconds, the FFI-5pt in 194 seconds, and the PF CAT in 44 seconds, (P < .001). The PF CAT and FAAM_ADL showed significant improvement (P = .01 and P = .001, respectively) in patients’ physical function after treatment; the FFI-5pt did not show improvement. Conclusions: Overall, the PF CAT performed best in terms of reliability, responsiveness, and efficiency in this broad sample of foot and ankle patients. It can be a potential replacement for the conventional PRO measures, but further validation is needed in conjunction with the PROMIS Pain instruments. Level of Evidence: Level I, prospective comparative outcome study.

New paradigm for patient-reported outcomes assessment in foot & ankle research: computerized adaptive testing.

Background: Accurately measuring, reporting and comparing outcomes is essential for improving health care delivery. Current challenges with available health status scales include patient fatigue, floor/ceiling effects and validity/reliability. Methods: This study compared Patient Reported Outcomes Measurement Information System (PROMIS)-based Lower Extremity Physical Function Computerized Adaptive Test (LE CAT) and two legacy scales -the Foot and Function Index (FFI) and the sport module from the Foot and Ankle Ability Measure (spFAAM) -for 287 patients scheduled for elective foot and ankle surgery. We documented the time required by patients to complete the instrument, instrument precision, and the extent to which each instrument covered the full range of physical functioning across the patient sample. Results: Average time of test administration: 66 seconds for LE CAT, 130 seconds for spFAAM and 239 seconds for FFI. All three instruments were fairly precise at intermediate physical functioning levels (i.e., Standard Error of Measurement < 0.35), were relatively less precise at the higher trait levels and the LE CAT maintained precision in the lower range while the spFAAM and FFIs had decreased precision. The LE CAT had less floor/ceiling effects than the FFI and the spFAAM. Conclusion: The LE CAT showed considerable advantage compared to legacy scales for measuring patient-reported outcomes in orthopaedic patients with foot and ankle problems. Clinical Relevance: A paradigm shift to broader use of PROMIS-based CATs should be considered to improve precision and reduce patient burden with patient-reported outcome measurement for foot and ankle patients.

Psychometric evaluation of the PROMIS Physical Function Computerized Adaptive Test in comparison to the American Shoulder and Elbow Surgeons score and Simple Shoulder Test in patients with rotator cuff disease

BACKGROUND The Patient-Reported Outcomes Measurement Information System (PROMIS) Physical Function Computerized Adaptive Test (PF CAT) is a newly developed patient-reported outcome instrument designed by the National Institutes of Health to measure generalized physical function. However, the measurement properties of the PF CAT have not been compared with established shoulder-specific patient-reported outcomes. METHODS Patients with clinical diagnosis of rotator cuff disease completed the American Shoulder and Elbow Surgeons (ASES) score, Simple Shoulder Test (SST), and PF CAT. Responses to each of the 3 instruments were statistically analyzed with a Rasch partial credit model. Associations between instruments, convergent validity, item and person reliability, ceiling and floor effects, dimensionality, and survey length were determined. RESULTS Responses from 187 patients were analyzed. The PF CAT required fewer questions than the ASES or SST (PF CAT, 4.3; ASES, 11; SST, 12). Correlation between all instruments was moderately high. Item reliability was excellent for all instruments, but person reliability of the PF CAT was superior (0.93, excellent) to the SST (0.71, moderate) and ASES (0.48, fair). Ceiling effects were similar among all instruments (PF CAT, 0.53%; SST, 6.1%; ASES, 2.3%). Floor effects were found in 21% of respondents to the SST but in only 3.2% of PF CAT and 2.3% of ASES respondents. CONCLUSION The measurement properties of the PROMIS PF CAT compared favorably with the ASES and SST despite requiring fewer questions to complete. The PROMIS PF CAT had improved person reliability compared with the ASES score and fewer floor effects compared with the SST.

Time for a paradigm shift with computerized adaptive testing of general physical function outcomes measurements.

Background: Patient-reported outcomes (PRO) are critical to understanding the value of orthopedic treatments. We hypothesized that use of the computerized adaptive testing from a well-characterized physical function item bank would show superiority in assessing all levels of physical function compared to current standard generic physical function outcomes instruments for foot and ankle patients. Methods: In a population of 126 foot and ankle patients we compared the psychometric properties for three PROs: the Physical Function subscale of the Medical Outcomes General Health Survey (SF-36 PF) version 2, the Patient-Reported Outcomes Measurement Information System (PROMIS) Physical Function (PF) Computerized Adaptive Test (CAT), and the Lower Extremity (LE) CAT. A Rasch item response theory (IRT) model was applied to assess and compare the fit, dimensionality, reliability, validity, and coverage. Results: The unexplained variance for the PF CAT was 3.9% and the LE CAT was 2.1%, suggesting each instrument explained a single concept. The SF-36 PF had more concerning unexplained variance of 7.6%. We found no floor or ceiling effects for the PF CAT, a minimal floor effect (1.6%) but no ceiling effect for the LE CAT, and an 11.1% floor effect and 9.5% ceiling effect for the SF-36 PF. Conclusion: Foot and ankle clinicians and researchers interested in measuring patient perceived functional outcomes with a generic instrument should consider using either the PF CAT or the LE CAT rather than the SF-36 PF. Further studies comparing these CATs to anatomic specific instruments are needed. Level of Evidence: Level I, diagnostic study.

Determining the minimal clinically important difference for the American Shoulder and Elbow Surgeons score, Simple Shoulder Test, and visual analog scale (VAS) measuring pain after shoulder arthroplasty.

BACKGROUND Minimal clinically important differences (MCIDs) for the American Shoulder and Elbow Surgeons (ASES) score, the Simple Shoulder Test (SST), and a visual analog scale (VAS) measuring pain have not been previously described using an anchor-based method after shoulder arthroplasty. The purpose of this study was to determine the MCIDs for these measures after shoulder arthroplasty for glenohumeral arthritis or advanced rotator cuff disease. METHODS Primary anatomic total shoulder arthroplasty (TSA), primary reverse TSA, or hemiarthroplasty was performed in 326 patients by 1 of 5 shoulder and elbow surgeons. The SST score, ASES score, and VAS pain score were collected preoperatively and at a minimum of 2 years postoperatively (mean, 3.5 years). The MCIDs were calculated for the ASES score, SST score, and VAS pain score using an anchor-based method. RESULTS The MCIDs for the ASES score, SST score, and VAS pain score were 20.9 (P < .001), 2.4 (P < .0001), and 1.4 (P = .0158), respectively. Duration of follow-up and type of arthroplasty (anatomic TSA vs reverse TSA) did not have a significant effect on the MCIDs (P > .1) except shorter follow-up correlated with a larger MCID for the ASES score (P = .0081). Younger age correlated with larger MCIDs for all scores (P < .024). Female sex correlated with larger MCIDs for the VAS pain score (P = .123) and ASES score (P = .05). CONCLUSIONS Patients treated with a shoulder arthroplasty require a 1.4-point improvement in the VAS pain score, a 2.4-point improvement in the SST score, and a 21-point improvement in the ASES score to achieve a minimal clinical importance difference from the procedure.

A lower extremity physical function computerized adaptive testing instrument for orthopaedic patients.

Background: The aim of this study was to develop a lower extremity (LE) physical function computerized adaptive testing (CAT) instrument based on Patient Reported Outcomes Measurement Information System (PROMIS) physical function items. Methods The PROMIS physical function item bank was administered to adult outpatient orthopaedic patients. Three hundred eighty-two patients presenting with LE disorders were analyzed using item response theory modeling. A LE physical function item bank was developed by distilling relevant and psychometrically sound items from the full PROMIS physical function bank. Real data CAT simulations were conducted to examine specifications for a live CAT. Results The LE physical function item bank was sufficiently unidimensional and free of item bias. It demonstrated high reliability along with content and construct validity. The flexible length LE CAT was highly correlated with the full LE instrument and showed uniformly high precision across the entire measurement continuum. The average CAT length was 6 to 7 items when standard error of measurement was 0.3 or less. Conclusion This LE physical function CAT is a valid, reliable and feasible physical function assessment tool for orthopaedic patients with LE problems that has the potential to reduce patient burden as well as administrative costs associated with data collection.

Psychometric assessment of the patient activation measure short form (PAM-13) in rural settings

PurposeThe patient activation measure short form (PAM-13) assesses patients’ self-reported health management skills, knowledge, confidence, and motivation. We used item response theory to evaluate the psychometric properties of the PAM-13 utilized in rural settings.MethodsA Rasch partial credit model analysis was conducted on the PAM-13 instrument using a sample of 812 rural patients recruited by providers and our research staff. Specially, we examined dimensionality, item fit, and quality of measures, category response curves, and item differential functioning. Convergent and divergent validities were also examined.FindingsThe PAM-13 instrument has excellent convergent and divergent validities. It is fairly unidimensional, and all items fit the Rasch model well. It has relatively high person and item reliability indices. Majority of the items were free of item differential functioning. There were, however, some issues with ceiling effects. Additionally, there was a lack of responses for category one across all items.ConclusionsPatient activation measure short form (PAM-13) performs well in some areas, but not all. In general, more items need to be added to cover the upper end of the trait. The four response categories of PAM-13 should be collapsed into three.