Brian D. Stucky

Direct and indirect aggression during childhood and adolescence: a meta-analytic review of gender differences, intercorrelations, and relations to maladjustment.

This meta-analytic review of 148 studies on child and adolescent direct and indirect aggression examined the magnitude of gender differences, intercorrelations between forms, and associations with maladjustment. Results confirmed prior findings of gender differences (favoring boys) in direct aggression and trivial gender differences in indirect aggression. Results also indicated a substantial intercorrelation (r = .76) between these forms. Despite this high intercorrelation, the 2 forms showed unique associations with maladjustment: Direct aggression is more strongly related to externalizing problems, poor peer relations, and low prosocial behavior, and indirect aggression is related to internalizing problems and higher prosocial behavior. Moderation of these effect sizes by method of assessment, age, gender, and several additional variables were systematically investigated.

PROMIS Pediatric Pain Interference Scale: An Item Response Theory Analysis of the Pediatric Pain Item Bank

UNLABELLED An aim of the National Institutes of Health (NIH) Patient Reported Outcomes Measurement Information System (PROMIS) initiative is to develop item banks and computerized adaptive tests (CAT) that are applicable across a wide variety of chronic disorders. The PROMIS Pediatric Cooperative Group has concentrated on the development of pediatric self-report item banks for ages 8 through 17 years. The objective of the present study is to describe the Item Response Theory (IRT) analysis of the NIH PROMIS pediatric pain item bank and the measurement properties of the new unidimensional PROMIS Pediatric Pain Interference Scale. Test forms containing pediatric pain items were completed by a total of 3048 respondents. IRT analyses regarding scale dimensionality, item local dependence, and differential item functioning were conducted. A pain item pool was developed to yield scores on a T-score scale with a mean of 50 and standard deviation of 10. The recommended 8-item unidimensional short form for the PROMIS Pediatric Pain Interference Scale contains the item set which provides the maximum test information at the mean (50) on the T-score metric. A simulated CAT was computed that provides the most information at 5 possible score locations (30, 40, 50, 60, and 70 on the T-score metric). PERSPECTIVE The present study provides initial calibrations of the NIH PROMIS pediatric pain item bank and the creation of the PROMIS Pediatric Pain Interference Scale. It is anticipated that this new scale will have application in pediatric chronic and recurrent pain.

Construction of the eight-item patient-reported outcomes measurement information system pediatric physical function scales: built using item response theory.

OBJECTIVE To create self-report physical function (PF) measures for children using modern psychometric methods for item analysis as part of patient-reported outcomes measurement information system (PROMIS). STUDY DESIGN AND SETTING PROMIS qualitative methodology was applied to develop two PF item pools that comprised 32 mobility and 38 upper extremity items. Items were computer administered to subjects aged 8-17 years. Scale dimensionality and sources of local dependence (LD) were evaluated with factor analysis. Items were analyzed for differential item functioning (DIF) between genders. Items with LD, DIF, or low discrimination were considered for removal. Computerized adaptive testing performance was simulated, and short forms were constructed. RESULTS Three thousand forty-eight children (51.8% female, 40% nonwhite, and 22.7% chronically ill) participated. At least 754 respondents answered each item. Factor analytical results confirmed two dimensions of PF. Fifty-two of 70 items tested were retained. A 23-item mobility bank and a 29-item upper extremity bank resulted, and an eight-item short forms were created. The item banks have high information from the population mean to three standard deviations below. CONCLUSIONS PROMIS pediatric PF item banks and eight-item short forms assess two dimensions, mobility, and upper extremity function and show good psychometric characteristics after large-scale testing.

Construction of the Pediatric Asthma Impact Scale (PAIS) for the Patient-Reported Outcomes Measurement Information System (PROMIS)

Background. Recently, the National Institutes of Health Roadmap for Medical Research initiative led a large-scale effort to develop the Patient-Reported Outcomes Measurement Information System (PROMIS). PROMISs main goal was to develop a set of item banks and computerized adaptive tests for the clinical research community. Asthma, as the most common chronic childhood disease, was chosen for a disease-specific pediatric item bank. Objectives. The primary objective of this research is to present the details of the psychometric analyses of the asthma domain items. Methods. Item response theory (IRT) analyses were conducted on a 34–asthma item bank. Test forms containing PROMIS Pediatric Asthma domain items were completed by 622 children ages 8 to 12. Items were subsequently evaluated for local dependence, scale dimensionality, and differential item functioning. Results. A 17-item pool and an 8-item short form for the new PROMIS Pediatric Asthma Impact Scale (PAIS) were generated using IRT. The recommended 8-item short form contains the item set that provides the maximum test information at the mean (50) on the T-score metric. If more score precision is required, the complete 17-item pool is recommended and may be used in toto or as the basis of a computerized adaptive test (CAT). A shorter test form can also be created and scored on the same scale. Conclusions. The present study presents the PROMIS Pediatric Asthma Impact Scale (PAIS) developed with IRT, and provides the initial calibration data for the items.

Development of six PROMIS pediatrics proxy-report item banks

BackgroundPediatric self-report should be considered the standard for measuring patient reported outcomes (PRO) among children. However, circumstances exist when the child is too young, cognitively impaired, or too ill to complete a PRO instrument and a proxy-report is needed. This paper describes the development process including the proxy cognitive interviews and large-field-test survey methods and sample characteristics employed to produce item parameters for the Patient Reported Outcomes Measurement Information System (PROMIS) pediatric proxy-report item banks.MethodsThe PROMIS pediatric self-report items were converted into proxy-report items before undergoing cognitive interviews. These items covered six domains (physical function, emotional distress, social peer relationships, fatigue, pain interference, and asthma impact). Caregivers (n = 25) of children ages of 5 and 17 years provided qualitative feedback on proxy-report items to assess any major issues with these items. From May 2008 to March 2009, the large-scale survey enrolled children ages 8-17 years to complete the self-report version and caregivers to complete the proxy-report version of the survey (n = 1548 dyads). Caregivers of children ages 5 to 7 years completed the proxy report survey (n = 432). In addition, caregivers completed other proxy instruments, PedsQL™ 4.0 Generic Core Scales Parent Proxy-Report version, PedsQL™ Asthma Module Parent Proxy-Report version, and KIDSCREEN Parent-Proxy-52.ResultsItem content was well understood by proxies and did not require item revisions but some proxies clearly noted that determining an answer on behalf of their child was difficult for some items. Dyads and caregivers of children ages 5-17 years old were enrolled in the large-scale testing. The majority were female (85%), married (70%), Caucasian (64%) and had at least a high school education (94%). Approximately 50% had children with a chronic health condition, primarily asthma, which was diagnosed or treated within 6 months prior to theinterview. The PROMIS proxy sample scored similar or better on the other proxy instruments compared to normative samples.ConclusionsThe initial calibration data was provided by a diverse set of caregivers of children with a variety of common chronic illnesses and racial/ethnic backgrounds. The PROMIS pediatric proxy-report item banks include physical function (mobility n = 23; upper extremity n = 29), emotional distress (anxiety n = 15; depressive symptoms n = 14; anger n = 5), social peer relationships (n = 15), fatigue (n = 34), pain interference (n = 13), and asthma impact (n = 17).

Short Assessment of Health Literacy—Spanish and English: A Comparable Test of Health Literacy for Spanish and English Speakers

OBJECTIVE The intent of the study was to develop and validate a comparable health literacy test for Spanish-speaking and English-speaking populations. STUDY DESIGN The design of the instrument, named the Short Assessment of Health Literacy-Spanish and English (SAHL-S&E), combined a word recognition test, as appearing in the Rapid Estimate of Adult Literacy in Medicine (REALM), and a comprehension test using multiple-choice questions designed by an expert panel. We used the item response theory (IRT) in developing and validating the instrument. DATA COLLECTION Validation of SAHL-S&E involved testing and comparing the instrument with other health literacy instruments in a sample of 201 Spanish-speaking and 202 English-speaking subjects recruited from the Ambulatory Care Center at the University of North Carolina Healthcare System. PRINCIPAL FINDINGS Based on IRT analysis, 18 items were retained in the comparable test. The Spanish version of the test, SAHL-S, was highly correlated with other Spanish health literacy instruments, Short Assessment of Health Literacy for Spanish-Speaking Adults (r=0.88, p<.05) and the Spanish Test of Functional Health Literacy in Adults (TOFHLA) (r=0.62, p<.05). The English version, SAHL-E, had high correlations with REALM (r=0.94, p<.05) and the English TOFHLA (r=0.68, p<.05). Significant correlations were found between SAHL-S&E and years of schooling in both Spanish- and English-speaking samples (r=0.15 and 0.39, respectively). SAHL-S&E displayed satisfactory reliability of 0.80 and 0.89 in the Spanish- and English-speaking samples, respectively. IRT analysis indicated that the SAHL-S&E score was highly reliable for individuals with a low level of health literacy. CONCLUSIONS The new instrument, SAHL-S&E, has good reliability and validity. It is particularly useful for identifying individuals with low health literacy and could be used to screen for low health literacy among Spanish and English speakers.It is hardly news anymore that a significant proportion of adults in the United States have difficulty navigating the health care system and managing personal health issues because of inadequate health literacy or limited “capacity to obtain, process, and understand health information and services needed to make appropriate health decisions” (Seldon, Zorn, Ratzan, & Parker, 2000). Inadequate health literacy, as a growing body of research has shown, is a risk factor for patients’ difficulties in understand health information and following medical instructions (Cho, Lee, Arozullah, & Crittenden, 2008; Davis et al., 2006; Gazmararian, Williams, Peel, & Baker, 2003; Parker, Ratzan, & Lurie, 2003), poor disease/self-management knowledge (Gazmararian et al., 2003), underuse of preventive services and routine physician and dental visits (Baker et al., 2004; Jones, Lee, & Rozier, 2007; Lindau, Basu, & Leitsch, 2006; Lindau et al., 2002; Rogers, Wallace, & Weiss, 2006; Scott, Gazmararian, Williams, & Baker, 2002), increased hospitalizations and medical costs (Baker et al., 2002; Howard, Gazmararian, & Parker, 2005), and high mortality rates (Sudore et al., 2006). Identifying individuals with inadequate health literacy is difficult because information such as age, educational attainment (i.e., years of schooling), and self-reported literacy skills do not reliably reflect an individual’s health literacy level (Bass, Wilson, Griffith, & Barnett, 2002; Davis, Jackson, George, et al., 1993; Davis, Arnold, Berkel, et al., 1996; Nurss, el-Kebbi, Gallina, et al., 1997). Over the years, several instruments, including the Test of Functional Health Literacy in Adults (TOFHLA), the Rapid Estimate of Adult Literacy in Medicine (REALM), and the Newest Vital Sign (NVS), have been developed to assess health literacy in the U.S. (Davis et al., 1993; Murphy, Davis, Long, Jackson, & Decker, 1993; Parker, Baker, Williams, & Nurss, 1995; Weiss et al., 2005). Most of the instruments, however, have a strong focus on the English-speaking populations and are inappropriate for assessing the health literacy level of Spanish-speakers. In the case of REALM, an attempt to develop a Spanish version failed because of the phonetic structure of the Spanish language (Nurss, Baker, David, Parker, & Williams, 1995).1 Where a Spanish version is available, e.g., TOFHLA-Spanish, the Spanish instrument is usually developed using a rudimentary translation-and-back-translation technique and is not validated psychometrically. A recent study comparing the psychometric properties of the English and Spanish versions of shortened TOFHLA raised a significant concern about their comparability (Aguirre, Ebrahim, & Shea, 2005). Our research team developed an easy-to-use health literacy test, the Short Assessment of Health Literacy for Spanish-speaking Adults (SAHLSA), for Spanish-speakers (Lee, Bender, Ruiz, & Cho, 2006). The SAHLSA contains 50 test items and has good psychometric qualities. It has been adopted in research and clinical practice in the U.S. (Keselman et al., 2007; Rosembla & Tse, 2006) and is being validated for use in Latin American countries (Huaman-Calderon, Quiliano-Terreros, & Vilchez-Roman, 2009). Since the publication of SAHLSA, many users have expressed the need for an English version to allow comparisons of health literacy level between Spanish and English speakers for research and clinical purposes. In this paper, we report our subsequent effort to develop a comparable test for Spanish and English-speakers, named Short Assessment of Health Literacy-Spanish & English or SAHL-S&E, based on the same methods used in developing SAHLSA. The test contains 18 items and is easy to administer. In taking the test, examinees are asked to read aloud each of the 18 medical terms and then associate each term to another word similar in meaning to demonstrate comprehension. The following sections describe the development of the SAHL-S&E, the methods employed to validate the instrument, results of the validation, and recommendations for use of the instrument.

PROMIS Pediatric Peer Relationships Scale: Development of a Peer Relationships Item Bank as Part of Social Health Measurement

OBJECTIVE This studys objective was to develop a measure of social health using item response theory as part of the Patient Reported Outcomes Measurement Information System (PROMIS). METHODS After candidate items were generated from review of prior literature, focus groups, expert input, and cognitive interviews, items were administered to youth aged 8-17 as part of the PROMIS pediatric large scale testing. Exploratory and confirmatory factor analyses were used to assess dimensionality and to identify instances of local dependence. Items that met the unidimensionality criteria were subsequently calibrated using Samejimas Graded Response Model. Differential item functioning was examined by gender and age. RESULTS The sample included 3,048 youth who completed the questionnaire (51.8% female, 60% white, and 22.7% with chronic illness). The initial conceptualization of social function and sociability did not yield unidimensional item banks. Rather, factor analysis revealed dimensions contrasting peer relationships and adult relationships. The analysis also identified dimensions formed by responses to positively versus negatively worded items. The resulting 15-item bank measures quality of peer relationships and has strong psychometric characteristics as a full bank or an 8-item short form. CONCLUSIONS The PROMIS pediatric peer relationships scale demonstrates good psychometric characteristics and addresses an important aspect of child health.

Using the PedsQL™ 3.0 asthma module to obtain scores comparable with those of the PROMIS pediatric asthma impact scale (PAIS)

BackgroundThe National Institutes of Health’s Patient-Reported Outcomes Measurement Information System (PROMIS) has developed several scales measuring symptoms and function for use by the clinical research community. One advantage of PROMIS is the ability to link other scales to the PROMIS metric.ObjectivesThe objectives of this research are to provide evidence of validity for one of the PROMIS measures, the Pediatric Asthma Impact Scale (PAIS), and to link the PedsQL™ Asthma Symptoms Scale with the metric of the PAIS.MethodsDescriptive statistics were computed describing the relationships among scores on the PAIS, the PedsQL™ Asthma Symptoms, Treatment, Worry, and Communication Scales, and the DISABKIDS Asthma Impact and Worry Scales for approximately 300 children ages 8–17. A novel linkage method based on item response theory (IRT), calibrated projection, was used to link scores on the PedsQL™ Asthma Symptoms Scale with the metric of the PAIS.ResultsThe PAIS exhibited strong convergent validity with the PedsQL™ Asthma Symptoms Scale, and less strong relations with the other five scales. The linkage system uses scores on the PedsQL™ Asthma Symptoms Scale to produce relatively precise score estimates on the metric of the PAIS.ConclusionsResults of this study provide evidence for the validity of the PAIS, and a method to use scores on the PedsQL™ Asthma Symptoms Scale to estimate scores on the metric of the PAIS, in partial fulfillment of the PROMIS goal to provide a lingua franca for health-related quality of life.

Using Logistic Approximations of Marginal Trace Lines to Develop Short Assessments.

Test developers often need to create unidimensional scales from multidimensional data. For item analysis, marginal trace lines capture the relation with the general dimension while accounting for nuisance dimensions and may prove to be a useful technique for creating short-form tests. This article describes the computations needed to obtain logistic approximations of marginal trace lines for graded response items derived from multidimensional bifactor item response theory (IRT) models. Next, the properties of marginal-trace-line-based likelihoods are evaluated and compared with other bifactor IRT methods. It is noted that for the dimension of interest, the likelihoods computed from marginal item response functions are not equivalent to the conditional likelihoods from the multidimensional IRT model. The authors then propose a method that evaluates the degree of item-level dimensionality and allows for the selection of subsets of items (i.e., short form) that result in scaled scores and standard errors that are equivalent to other multidimensional IRT-based scoring procedures. Finally, a real-data application is provided, which illustrates the utility of logistic approximations of marginal trace lines in the creation of a content-diverse short form.

Methodology for Developing and Evaluating the PROMIS® Smoking Item Banks

INTRODUCTION This article describes the procedures used in the PROMIS Smoking Initiative for the development and evaluation of item banks, short forms (SFs), and computerized adaptive tests (CATs) for the assessment of 6 constructs related to cigarette smoking: nicotine dependence, coping expectancies, emotional and sensory expectancies, health expectancies, psychosocial expectancies, and social motivations for smoking. METHODS Analyses were conducted using response data from a large national sample of smokers. Items related to each construct were subjected to extensive item factor analyses and evaluation of differential item functioning (DIF). Final item banks were calibrated, and SF assessments were developed for each construct. The performance of the SFs and the potential use of the item banks for CAT administration were examined through simulation study. RESULTS Item selection based on dimensionality assessment and DIF analyses produced item banks that were essentially unidimensional in structure and free of bias. Simulation studies demonstrated that the constructs could be accurately measured with a relatively small number of carefully selected items, either through fixed SFs or CAT-based assessment. Illustrative results are presented, and subsequent articles provide detailed discussion of each item bank in turn. CONCLUSIONS The development of the PROMIS smoking item banks provides researchers with new tools for measuring smoking-related constructs. The use of the calibrated item banks and suggested SF assessments will enhance the quality of score estimates, thus advancing smoking research. Moreover, the methods used in the current study, including innovative approaches to item selection and SF construction, may have general relevance to item bank development and evaluation.