Susan Magasi

Content validity of patient-reported outcome measures: perspectives from a PROMIS meeting

Content validity of patient-reported outcome measures (PROs) has been a focus of debate since the 2006 publication of the U.S. FDA Draft Guidance for Industry in Patient Reported Outcome Measurement. Under the auspices of the Patient Reported Outcomes Measurement Information System (PROMIS) initiative, a working meeting on content validity was convened with leading PRO measurement experts. Platform presentations and participant discussion highlighted key issues in the content validity debate, including inconsistency in the definition and evaluation of content validity, the need for empirical research to support methodological approaches to the evaluation of content validity, and concerns that continual re-evaluation of content validity slows the pace of science and leads to the proliferation of study-specific PROs. We advocate an approach to the evaluation of content validity, which includes meticulously documented qualitative and advanced quantitative methods. To advance the science of content validity in PROs, we recommend (1) development of a consensus definition of content validity; (2) development of content validity guidelines that delineate the role of qualitative and quantitative methods and the integration of multiple perspectives; (3) empirical evaluation of generalizability of content validity across applications; and (4) use of generic measures as the foundation for PROs assessment.

Assessing dexterity function: A comparison of two alternatives for the NIH toolbox

STUDY DESIGN Clinical measurement. INTRODUCTION Manual dexterity is an important aspect of motor function across the age span. PURPOSE OF THE STUDY To identify a single measure of manual dexterity for inclusion in the National Institutes of Health (NIH) Toolbox Assessment of Neurological and Behavioral Function. METHODS A total of 340 subjects participated in our study. Two alternatives, Rolyan® 9-Hole Peg Test (9-HPT) and Grooved Pegboard test, were compared by assessing their score range across age groups (3-85 yr) and their test-retest reliability, concurrent, and known groups validity. RESULTS The 9-HPT was a simple, efficient, and low-cost measure of manual dexterity appropriate for administration across the age range. Test-retest reliability coefficients were 0.95 and 0.92 for right and left hands, respectively. The 9-HPT correlated with Bruininks-Oseretsky Test (BOT) of Motor Proficiency, dexterity subscale, at -0.87 to -0.89 and with Purdue Pegboard at -0.74 to -0.75. The Grooved Pegboard had good test-retest reliability (0.91 and 0.85 for right and left hands, respectively). The Grooved Pegboard correlated with BOT at -0.50 to -0.63 and with Purdue Pegboard at -0.73 to -0.78. However, the Grooved Pegboard required longer administration time and was challenging for the youngest children and oldest adults. CONCLUSIONS Based on its feasibility and measurement properties, the 9-HPT was recommended for inclusion in the motor battery of the NIH Toolbox. LEVEL OF EVIDENCE NA.

Environmental Barriers and Supports to Everyday Participation: A Qualitative Insider Perspective From People With Disabilities

OBJECTIVE To describe environmental factors that influence participation of people with disabilities. DESIGN Constant comparative, qualitative analyses of transcripts from 36 focus groups across 5 research projects. SETTING Home, community, work, and social participation settings. PARTICIPANTS Community-dwelling people (N=201) with diverse disabilities (primarily spinal cord injury, traumatic brain injury, and stroke) from 8 states. INTERVENTIONS None. MAIN OUTCOME MEASURES Environmental barriers and supports to participation. RESULTS We developed a conceptual framework to describe how environmental factors influence the participation of people with disabilities, highlighting 8 domains of environmental facilitators and barriers (built, natural, assistive technology, transportation, information and technology access, social support and attitudes, systems and policies, economics) and a transactional model showing the influence of environmental factors on participation at the micro (individual), mesa (community), and macro (societal) levels. Focus group data validated some International Classification of Functioning, Disability and Health environmental categories while also bringing unique factors (eg, information and technology access, economic quality of life) to the fore. Data were used to construct items to enable people with disabilities to assess the impact of environmental factors on everyday participation from their firsthand experience. CONCLUSIONS Participants with disabilities voiced the need to evaluate the impact of the environment on their participation at the immediate, community, and societal levels. The results have implications for assessing environmental facilitators and barriers to participation within rehabilitation and community settings, evaluating outcomes of environmental interventions, and effecting system and policy changes to target environmental barriers that may result in societal participation disparities versus opportunities.

Grip and knee extension muscle strength reflect a common construct among adults.

Introduction: Both grip and knee extension strength are often used to characterize overall limb muscle strength. We sought to determine if the measures actually reflect a common construct. Methods: The isometric grip and knee extension strength of 164 healthy men and women (range, 18–85 years) were measured bilaterally using standard procedures. Pearson correlations (r), Cronbach alpha, principal components analysis, and multiple regression/correlation were used to investigate the dimensionality of the measures. Results: Left and right grip forces and knee extension torques were highly correlated, internally consistent, and loaded on a single component. Gender and age explained the variance in both measures, but height added to the explanation of grip strength, whereas weight added to the explanation of knee extension strength. Conclusions: Among healthy adults, grip and knee extension strength reflect a common underlying construct. The measures, however, are affected differently by height and weight. Muscle Nerve 46: 555–558, 2012

Theoretical Foundations for the Measurement of Environmental Factors and Their Impact on Participation Among People With Disabilities

The ascendance of the World Health Organizations International Classification of Functioning, Disability and Heath (ICF) as the global standard for describing and characterizing aspects of disability has refocused attention on the role that environmental factors (EFs) have on the health and participation of people with disabilities, both as individuals and as a group. There has been a rise in the development of instruments designed to measure EFs alone and in relation to participation. Some instrument developers have used the ICF as a theoretical base for instrument development and to substantiate content validity claims. We contend that this is a misapplication of the ICF. There is a need to step back and reexamine the role that environmental theories can play in developing a conceptually driven approach to measuring the interaction between EFs and participation. For this review, we draw on the fields of social, community, and developmental psychology; disability studies; gerontology; public health; and rehabilitation. We discuss different approaches to the measurement of EFs. We suggest that given the complex nature of EFs and their influence on participation, there is a need for a fresh approach to EF measurement. The thoughtful application of theories and the use of advanced psychometric, measurement, and e-technologies and data visualization methods may enable researchers and clinicians to better quantify, document, and communicate the dynamic interrelationship between EFs and participation and health outcomes for people with disabilities at the individual, group, and population levels.

Measuring Participation Enfranchisement

OBJECTIVE To reflect the perspectives of rehabilitation stakeholders in a measure of participation enfranchisement that can be used by people with and without disabilities. DESIGN Survey. SETTING Community settings. PARTICIPANTS We pilot-tested a draft instrument with 326 adults who had sustained stroke, spinal cord injury, traumatic brain injury, or other disabling condition, as well as a general population sample. We administered a revised version of the instrument to a statewide sample drawn from the 2006 Colorado Behavioral Risk Factors Surveillance System that included persons with (N=461) and without (N=451) self-identified activity limitations. INTERVENTIONS None. MAIN OUTCOME MEASURE Participation enfranchisement. RESULTS We used multidimensional scaling, exploratory factor analysis (EFA) and confirmatory factor analysis (CFA), followed by rating scale analysis to evaluate the psychometric properties of the instrument. EFA identified 3 participation enfranchisement factors that describe perceived choice and control, contributing to ones community, and feeling valued; the factors were supported marginally by CFA. Rating scale analysis revealed marginal person separation and no misfitting items. CONCLUSIONS Participation enfranchisement constitutes a new, previously unmeasured aspect of participation-one that addresses subjective perceptions rather than objective performance-with items that are clearly distinct from more generalized satisfaction with participation. The 19 enfranchisement items describe aspects of participation that may prove useful in characterizing longer-term rehabilitation outcomes.

Barriers to mental health treatment among obstetric patients at risk for depression

OBJECTIVE The objective of the study was to examine mental health referrals outcomes among obstetric patients at risk for depression. STUDY DESIGN Fifty-one perinatal women who were offered mental health referrals were queried about their behaviors at 4 steps in the treatment engagement process and factors facilitating or impeding each step. RESULTS Although 59% of at-risk women accepted mental health referrals, only 27% ultimately engaged in treatment. Women who proactively sought help via a hotline were more likely to accept referrals (P < .001), contact a referred provider (P < .001), and engage in treatment (P < .05) than those who received unsolicited referrals after screening at-risk for depression. Barriers to successful treatment linkage were identified at the patient, provider, and system levels. CONCLUSION Only a minority of women who are at risk for perinatal depression and receive mental health referrals ultimately engage in treatment. Successful linkage may be enhanced via interventions targeting identified barriers; such interventions require prospective evaluation.

Environmental factors item development for persons with stroke, traumatic brain injury, and spinal cord injury.

OBJECTIVES To describe methods used in operationalizing environmental factors; to describe the results of a research project to develop measures of environmental factors that affect participation; and to define an initial item set of facilitators and barriers to participation after stroke, traumatic brain injury, and spinal cord injury. DESIGN Instrument development included an extensive literature review, item classification and selection, item writing, and cognitive testing following the approach of the Patient-Reported Outcomes Measurement Information System. SETTING Community. PARTICIPANTS Content area and outcome measurement experts (n=10) contributed to instrument development; individuals (n=200) with the target conditions participated in focus groups and in cognitive testing (n=15). INTERVENTIONS None. MAIN OUTCOME MEASURES Environmental factor items were categorized in 6 domains: assistive technology; built and natural environment; social environment; services, systems, and policies; access to information and technology; and economic quality of life. RESULTS We binned 2273 items across the 6 domains, winnowed this pool to 291 items for cognitive testing, and recommended 274 items for pilot data collection. CONCLUSIONS Five of the 6 domains correspond closely to the International Classification of Functioning, Disability and Health taxonomy of environmental factors; the sixth domain, economic quality of life, reflects an important construct that reflects financial resources that affect participation. Testing with a new and larger sample is underway to evaluate reliability, validity, and sensitivity.

Improving Measurement Methods in Rehabilitation: Core Concepts and Recommendations for Scale Development

Validated measurement scales are essential to evaluating clinical outcomes and conducting meaningful and reliable research. The purpose of this article is to present the clinician and researcher with a contemporary 8-stage framework for measurement scale development based on a mixed-methods qualitative and quantitative approach. Core concepts related to item response theory are presented. Qualitative methods are described to conceptualize scale constructs; obtain patient, family, and other stakeholder perspectives; and develop item pools. Item response theory statistical methodologies are presented, including approaches for testing the assumptions of unidimensionality, local independence, monotonicity, and indices of model fit. Lastly, challenges faced by scale developers in implementing these methodologies are discussed. While rehabilitation research has recently started to apply mixed-methods qualitative and quantitative methodologies to scale development, these approaches show considerable promise in advancing rehabilitation measurement.

Participation Following Traumatic Spinal Cord Injury: An Evidence-Based Review for Research: Report on the National Institute on Disability and Rehabilitation Research Spinal Cord Injury Measures Meeting

Abstract Background/Objective: To report an evidence-based review of participation instruments that have been used in spinal cord injury (SCI) clinical practice and research. Methods: Rehabilitation literature was searched for instruments used by at least 2 independent SCI researchers since 2000. Each instrument was reviewed by 2 committee members. One person reviewed the scale and documented the level of use and psychometric properties. The second committee member verified the values and made suggestions for changes. Results: Three instruments met the review criteria: Craig Handicap Assessment and Reporting Technique (CHART), Assessment of Life Habits (LIFE-H), and the Impact on Participation and Autonomy (IPA). Each instrument incorporates different perspectives in the measurement of participation. The LIFE-H uses a qualitative approach, whereas the CHART adopts a quantitative approach; both are based on societal norms of participation. In contrast, the IPA integrates individual choice and control in defining participation. CHART is the most widely used instrument, although its development predates the development of the ICF. The IPA is a relatively new instrument, and its psychometric properties have only recently published. Conclusions: Continuing research is needed to develop conceptually and psychometric valid measures of participation for use with people with SCI. Priorities include understanding the relationship between objective and subjective indicators of participation, describing the dimensions of participation, and identifying appropriate measurement models and psychometric approaches to evaluate the nonhierarchical character of participation. Researchers and clinicians should be aware of the strengths and limitations of existing measures to make informed decisions about appropriate instruments.