Diane E. Brandt

Connecting contemporary paradigms to the Social Security Administration’s disability evaluation process

From 1998 to 2008, the Social Security Administration’s (SSA) disability insurance program (DI) applications rose from 1.2 million to 2.3 million and exceeded 3 million in 2009. Given these large and growing numbers, even small changes in processing disability applications may reduce processing time, lower program costs, and improve performance of SSA’s disability programs. A literature review examining current conceptual models of disability and SSA’s disability evaluation process for adults was conducted. A gap exists between contemporary models of disability and how SSA defines and operationalizes disability. This is complicated by substantial variation in the timing, quantity, and quality of applicant functional information and workplace demands. A focus on impairment marginalizes more comprehensive assessment of function necessary to assess capacity for work. Novel assessment methodologies, such as computer adaptive testing to measure human functioning may hold promise for SSA’s data collection methods and disability assessment.

Conceptual Foundation for Measures of Physical Function and Behavioral Health Function for Social Security Work Disability Evaluation

Physical and mental impairments represent the 2 largest health condition categories for which workers receive Social Security disability benefits. Comprehensive assessment of physical and mental impairments should include aspects beyond medical conditions such as a persons underlying capabilities as well as activity demands relevant to the context of work. The objective of this article is to describe the initial conceptual stages of developing new measurement instruments of behavioral health and physical functioning relevant for Social Security work disability evaluation purposes. To outline a clear conceptualization of the constructs to be measured, 2 content models were developed using structured and informal qualitative approaches. We performed a structured literature review focusing on work disability and incorporating aspects of the International Classification of Functioning, Disability and Health as a unifying taxonomy for framework development. Expert interviews provided advice and consultation to enhance face validity of the resulting content models. The content model for work-related behavioral health function identifies 5 major domains: (1) behavior control, (2) basic interactions, (3) temperament and personality, (4) adaptability, and (5) workplace behaviors. The content model describing physical functioning includes 3 domains: (1) changing and maintaining body position, (2) whole-body mobility, and (3) carrying, moving, and handling objects. These content models informed subsequent measurement properties including item development and measurement scale construction, and provided conceptual coherence guiding future empirical inquiry. The proposed measurement approaches show promise to comprehensively and systematically assess physical and behavioral health functioning relevant to work.

Development of a computer-adaptive physical function instrument for Social Security Administration disability determination.

OBJECTIVES To develop and test an instrument to assess physical function for Social Security Administration (SSA) disability programs, the SSA-Physical Function (SSA-PF) instrument. Item response theory (IRT) analyses were used to (1) create a calibrated item bank for each of the factors identified in prior factor analyses, (2) assess the fit of the items within each scale, (3) develop separate computer-adaptive testing (CAT) instruments for each scale, and (4) conduct initial psychometric testing. DESIGN Cross-sectional data collection; IRT analyses; CAT simulation. SETTING Telephone and Internet survey. PARTICIPANTS Two samples: SSA claimants (n=1017) and adults from the U.S. general population (n=999). INTERVENTIONS None. MAIN OUTCOME MEASURES Model fit statistics, correlation, and reliability coefficients. RESULTS IRT analyses resulted in 5 unidimensional SSA-PF scales: Changing & Maintaining Body Position, Whole Body Mobility, Upper Body Function, Upper Extremity Fine Motor, and Wheelchair Mobility for a total of 102 items. High CAT accuracy was demonstrated by strong correlations between simulated CAT scores and those from the full item banks. On comparing the simulated CATs with the full item banks, very little loss of reliability or precision was noted, except at the lower and upper ranges of each scale. No difference in response patterns by age or sex was noted. The distributions of claimant scores were shifted to the lower end of each scale compared with those of a sample of U.S. adults. CONCLUSIONS The SSA-PF instrument contributes important new methodology for measuring the physical function of adults applying to the SSA disability programs. Initial evaluation revealed that the SSA-PF instrument achieved considerable breadth of coverage in each content domain and demonstrated noteworthy psychometric properties.

Development of an Instrument to Measure Behavioral Health Function for Work Disability: Item Pool Construction and Factor Analysis

OBJECTIVES To develop a broad set of claimant-reported items to assess behavioral health functioning relevant to the Social Security disability determination processes, and to evaluate the underlying structure of behavioral health functioning for use in development of a new functional assessment instrument. DESIGN Cross-sectional. SETTING Community. PARTICIPANTS Item pools of behavioral health functioning were developed, refined, and field tested in a sample of persons applying for Social Security disability benefits (N=1015) who reported difficulties working because of mental or both mental and physical conditions. INTERVENTIONS None. MAIN OUTCOME MEASURE Social Security Administration Behavioral Health (SSA-BH) measurement instrument. RESULTS Confirmatory factor analysis (CFA) specified that a 4-factor model (self-efficacy, mood and emotions, behavioral control, social interactions) had the optimal fit with the data and was also consistent with our hypothesized conceptual framework for characterizing behavioral health functioning. When the items within each of the 4 scales were tested in CFA, the fit statistics indicated adequate support for characterizing behavioral health as a unidimensional construct along these 4 distinct scales of function. CONCLUSIONS This work represents a significant advance both conceptually and psychometrically in assessment methodologies for work-related behavioral health. The measurement of behavioral health functioning relevant to the context of work requires the assessment of multiple dimensions of behavioral health functioning. Specifically, we identified a 4-factor model solution that represented key domains of work-related behavioral health functioning. These results guided the development and scale formation of a new SSA-BH instrument.

Development of a Self-Report Physical Function Instrument for Disability Assessment: Item Pool Construction and Factor Analysis

OBJECTIVES To build a comprehensive item pool representing work-relevant physical functioning and to test the factor structure of the item pool. These developmental steps represent initial outcomes of a broader project to develop instruments for the assessment of function within the context of Social Security Administration (SSA) disability programs. DESIGN Comprehensive literature review; gap analysis; item generation with expert panel input; stakeholder interviews; cognitive interviews; cross-sectional survey administration; and exploratory and confirmatory factor analyses to assess item pool structure. SETTING In-person and semistructured interviews and Internet and telephone surveys. PARTICIPANTS Sample of SSA claimants (n=1017) and a normative sample of adults from the U.S. general population (n=999). INTERVENTIONS Not applicable. MAIN OUTCOME MEASURE Model fit statistics. RESULTS The final item pool consisted of 139 items. Within the claimant sample, 58.7% were white; 31.8% were black; 46.6% were women; and the mean age was 49.7 years. Initial factor analyses revealed a 4-factor solution, which included more items and allowed separate characterization of: (1) changing and maintaining body position, (2) whole body mobility, (3) upper body function, and (4) upper extremity fine motor. The final 4-factor model included 91 items. Confirmatory factor analyses for the 4-factor models for the claimant and the normative samples demonstrated very good fit. Fit statistics for claimant and normative samples, respectively, were: Comparative Fit Index=.93 and .98; Tucker-Lewis Index=.92 and .98; and root mean square error approximation=.05 and .04. CONCLUSIONS The factor structure of the physical function item pool closely resembled the hypothesized content model. The 4 scales relevant to work activities offer promise for providing reliable information about claimant physical functioning relevant to work disability.

Scale refinement and initial evaluation of a behavioral health function measurement tool for work disability evaluation.

OBJECTIVE To use item response theory (IRT) data simulations to construct and perform initial psychometric testing of a newly developed instrument, the Social Security Administration Behavioral Health Function (SSA-BH) instrument, that aims to assess behavioral health functioning relevant to the context of work. DESIGN Cross-sectional survey followed by IRT calibration data simulations. SETTING Community. PARTICIPANTS Sample of individuals applying for Social Security Administration disability benefits: claimants (n=1015) and a normative comparative sample of U.S. adults (n=1000). INTERVENTIONS None. MAIN OUTCOME MEASURE SSA-BH measurement instrument. RESULTS IRT analyses supported the unidimensionality of 4 SSA-BH scales: mood and emotions (35 items), self-efficacy (23 items), social interactions (6 items), and behavioral control (15 items). All SSA-BH scales demonstrated strong psychometric properties including reliability, accuracy, and breadth of coverage. High correlations of the simulated 5- or 10-item computer adaptive tests with the full item bank indicated robust ability of the computer adaptive testing approach to comprehensively characterize behavioral health function along 4 distinct dimensions. CONCLUSIONS Initial testing and evaluation of the SSA-BH instrument demonstrated good accuracy, reliability, and content coverage along all 4 scales. Behavioral function profiles of Social Security Administration claimants were generated and compared with age- and sex-matched norms along 4 scales: mood and emotions, behavioral control, social interactions, and self-efficacy. Using the computer adaptive test-based approach offers the ability to collect standardized, comprehensive functional information about claimants in an efficient way, which may prove useful in the context of the Social Security Administrations work disability programs.

Conceptualizing disability in US national surveys: application of the World Health Organization's (WHO) International Classification of Functioning, Disability, and Health (ICF) framework.

PurposeDisability data inform resource allocation and utilization, characterize functioning and changes over time, and provide a mechanism to monitor progress toward promoting and protecting the rights of individuals with disability. Data collection efforts, however, define and measure disability in varied ways. Our objective was to see how the content of disability measures differed in five US national surveys and over time.MethodsUsing the WHO ICF as a conceptual framework for measuring disability, we assessed the National Health Interview Survey (NHIS), Current Population Survey (CPS), Survey of Income and Program Participation (SIPP), National Survey of SSI Children and Families (NSCF), and American Community Survey (ACS) for their content coverage of disability relative to each of the four ICF components (i.e., body functions, body structures, activities and participation, and environment). We used second-level ICF three-digit codes to classify question content into categories within each ICF component and computed the proportion of categories within each ICF component that was represented in the questions selected from these five surveys.ResultsThe disability measures varied across surveys and years. The NHIS captured a greater proportion of the ICF body functions and body structures components than did other surveys. The SIPP captured the most content of the ICF activities and participation component, and the NSCF contained the most content of the ICF environmental factors component.ConclusionsThis research successfully illustrated demonstrated the utility of the ICF in examining the content of disability measures in five national surveys and over time.

Work-related measures of physical and behavioral health function: Test-retest reliability

BACKGROUND The Work Disability Functional Assessment Battery (WD-FAB), developed for potential use by the US Social Security Administration to assess work-related function, currently consists of five multi-item scales assessing physical function and four multi-item scales assessing behavioral health function; the WD-FAB scales are administered as Computerized Adaptive Tests (CATs). OBJECTIVE The goal of this study was to evaluate the test-retest reliability of the WD-FAB Physical Function and Behavioral Health CATs. METHODS We administered the WD-FAB scales twice, 7-10 days apart, to a sample of 376 working age adults and 316 adults with work-disability. Intraclass correlation coefficients were calculated to measure the consistency of the scores between the two administrations. Standard error of measurement (SEM) and minimal detectable change (MDC90) were also calculated to measure the scales precision and sensitivity. RESULTS For the Physical Function CAT scales, the ICCs ranged from 0.76 to 0.89 in the working age adult sample, and 0.77-0.86 in the sample of adults with work-disability. ICCs for the Behavioral Health CAT scales ranged from 0.66 to 0.70 in the working age adult sample, and 0.77-0.80 in the adults with work-disability. The SEM ranged from 3.25 to 4.55 for the Physical Function scales and 5.27-6.97 for the Behavioral Health function scales. For all scales in both samples, the MDC90 ranged from 7.58 to 16.27. CONCLUSION Both the Physical Function and Behavioral Health CATs of the WD-FAB demonstrated good test-retest reliability in adults with work-disability and general adult samples, a critical requirement for assessing work related functioning in disability applicants and in other contexts.

Improving measures of work-related physical functioning

Purpose To expand content of the physical function domain of the Work Disability Functional Assessment Battery (WD-FAB), developed for the US Social Security Administration’s (SSA) disability determination process.MethodsNewly developed questions were administered to 3532 recent SSA applicants for work disability benefits and 2025 US adults. Factor analyses and item response theory (IRT) methods were used to calibrate and link the new items to the existing WD-FAB, and computer-adaptive test simulations were conducted.ResultsFactor and IRT analyses supported integration of 44 new items into three existing WD-FAB scales and the addition of a new 11-item scale (Community Mobility). The final physical function domain consisting of: Basic Mobility (56 items), Upper Body Function (34 items), Fine Motor Function (45 items), and Community Mobility (11 items) demonstrated acceptable psychometric properties.ConclusionsThe WD-FAB offers an important tool for enhancement of work disability determination. The FAB could provide relevant information about work-related functioning for initial assessment of claimants; identifying denied applicants who may benefit from interventions to improve work and health outcomes; enhancing periodic review of work disability beneficiaries; and assessing outcomes for policies, programs and services targeting people with work disability.

INTERPRETING PHYSICAL AND BEHAVIORAL HEALTH SCORES FROM NEW WORK DISABILITY INSTRUMENTS

OBJECTIVE To develop a system to guide interpretation of scores generated from 2 new instruments measuring work-related physical and behavioral health functioning (Work Disability - Physical Function (WD-PF) and WD - Behavioral Function (WD-BH)). DESIGN Cross-sectional, secondary data from 3 independent samples to develop and validate the functional levels for physical and behavioral health functioning. SUBJECTS Physical group: 999 general adult subjects, 1,017 disability applicants and 497 work-disabled subjects. Behavioral health group: 1,000 general adult subjects, 1,015 disability applicants and 476 work-disabled subjects. METHODS Three-phase analytic approach including item mapping, a modified-Delphi technique, and known-groups validation analysis were used to develop and validate cut-points for functional levels within each of the WD-PF and WD-BH instruments scales. RESULTS Four and 5 functional levels were developed for each of the scales in the WD-PF and WD-BH instruments. Distribution of the comparative samples was in the expected direction: the general adult samples consistently demonstrated scores at higher functional levels compared with the claimant and work-disabled samples. CONCLUSION Using an item-response theory-based methodology paired with a qualitative process appears to be a feasible and valid approach for translating the WD-BH and WD-PF scores into meaningful levels useful for interpreting a persons work-related physical and behavioral health functioning.