Katie A. Devine

Editorial: An Author's Checklist for Measure Development and Validation Manuscripts

A recent special issue of the Journal of Pediatric Psychology included papers focused on evidence-based assessment across several broad domains of assessment in pediatric psychology (e.g., adherence, pediatric pain, and quality of life). In one of these papers, Holmbeck et al. (2008) reviewed strengths and limitations of existing measures of psychosocial adjustment and psychopathology, concluding that many measures lacked supporting psychometric data (e.g., basic indices of reliability and validity) that would permit a complete evaluation of these measures. Given that measure development and validation papers are frequently published in JPP (Brown, 2007), it is important that we attend to guiding psychometric principles when developing and disseminating data on new measures to be employed with pediatric populations (Nunnally & Bernstein, 1994). Thus, the purpose of this paper is to present and describe a checklist for authors to use when submitting measure development papers to JPP. This checklist is included in the Appendix and is also included at the following link on the JPP website: http://www.oxfordjournals.org/our_journals/jpepsy/for_authors/measure%20development%20checklist.pdf Findings presented by Holmbeck et al. (2008) indicated that 34 of the 37 measures reviewed met pre-established “evidence-based assessment” (EBA) criteria for “well-established” measures (Cohen et al., 2008). To be considered “well-established,” a measure had to have been presented in at least two peer-reviewed journal articles by different investigatory teams, have demonstrated adequate levels of reliability and validity, and be accompanied by supporting information (e.g., a measure manual). Although most measures that we reviewed met these criteria, we also found that most of the 34 “well-established” measures were hampered by at least one major psychometric flaw and/or lacked important psychometric data. We concluded that a more fine-grained EBA classification system is needed. One important distinction in this literature relates to differences between empirically supported assessment and evidence-based assessment. This type of distinction was first discussed in the literature on clinical interventions (e.g., Spring, 2007). An empirically supported intervention is one that has demonstrated efficacy in randomized clinical trials or clinic-based effectiveness trials. An evidence-based intervention has empirical support in the manner just described, but also “integrates research evidence, clinical expertise, and patient preferences and characteristics … empirically-supported treatments (ESTs) are an important component of evidence-based practice (EBP), but EBP cannot be reduced to ESTs” (Spring, 2007, p.611). Applying these terms to the field of assessment and measure development efforts, an empirically supported assessment measure would be one that demonstrates satisfactory psychometric characteristics, broadly defined. To be evidence based, the measure should also demonstrate utility in clinical settings, be useful in making diagnoses, be sensitive to treatment effects, and/or provide incremental validity above and beyond other similar measures. Although papers in the special issue of JPP frequently referred to “evidence-based assessment” (Cohen et al., 2009), the articles included in the issue tended to evaluate the degree to which the measures were empirically-supported rather than evidence based. To be “evidence-based,” our reviews would have needed to integrate an evaluation of clinical utility, diagnostic utility, and treatment sensitivity with the empirical psychometric data presented in each review. As noted, the published reviews were more likely to focus on the latter rather than on the former. As suggested by Mash and Hunsley (2005), detailed EBA profiles would provide a complete evaluation of evidence across each of several psychometric and clinically relevant dimensions, including: (a) internal consistency, (b) test–retest reliability, (c) the availability of normative data, (d) content validity, (e) construct validity, (f) convergent and discriminant validity, (g) criterion-related validity, (h) incremental validity, (i) clinical utility, (j) diagnostic utility, and (k) treatment sensitivity. The focus on incremental validity and clinical and diagnostic utility raises the bar from a focus on “empirical support” (i.e., where the focus would tend to be primarily on psychometric data) to a broad focus on the “evidence base” for a measure. In developing the checklist that is the focus of this article, we attempted to provide a list of criteria relevant to establishing the evidence base (and not just empirical support) for a measure. In addition to shifting the focus from providing “empirical support” for a measure to providing an “evidence base” for our instruments, a checklist for measure development papers would permit JPP reviewers to evaluate such papers in the same way that reviewers of randomized clinical trials make use of the Consolidated Standards of Reporting Trials (CONSORT) checklist and flowchart (Altman et al., 2001). The CONSORT checklist contains reporting standards with respect to methodological features of and the manner in which results are reported in clinical trials. Moreover, authors are required to provide a flowchart that describes details of sample recruitment and attrition during the course of the study. Thus, a checklist for measure development papers would serve two interrelated purposes: (a) it would provide guidance to authors as they embark on the measure development process and would provide a list of criteria authors can use as they develop an evidence base for their measures, and (b) it would begin to standardize the manner in which psychometric and other assessment-related data are presented in measure development papers for this journal. Before providing a more detailed overview of the checklist, it is important to note that this checklist is rather exhaustive (see Appendix). As such, it represents what would “ideally” be expected for a measure development or validation manuscript rather than minimal criteria for such papers. No one paper can provide a complete evaluation of all important psychometric and clinically relevant dimensions that will establish once-and-for-all the evidence base for a measure. Instrument refinement is part of a measure development process that gradually builds an evidence base for a scale (see Smith & McCarthy, 1995, for suggestions on measure refinement). Indeed, the validation of any measure is a cumulative process that occurs across many different types of research studies and across research programs.

Psychosocial and family functioning in spina bifida

A developmentally oriented bio-neuropsychosocial model is introduced to explain the variation in family functioning and psychosocial adjustment in youth and young adults with spina bifida (SB). Research on the family functioning and psychosocial adjustment of individuals with SB is reviewed. The findings of past research on families of youth with SB support a resilience-disruption view of family functioning. That is, the presence of a child with SB disrupts normative family functioning but many families adapt to such disruption and exhibit considerable resilience in the face of adversity. Parents of youth with SB, and particularly those from lower socio-economic status (SES) homes, are at-risk for psychosocial difficulties. Individuals with SB are at-risk for developing internalizing symptoms, attention problems, educational difficulties, social maladjustment, and delays in the development of independent functioning. Emerging adults are often delayed in achieving milestones related to this stage of development (e.g., vocational and educational achievements). Methodologically sound, longitudinal, and theory-driven studies of family and psychosocial functioning are needed, as are randomized family-based intervention trials, to promote adaptive functioning and better psychosocial outcomes in families of individuals with SB.

Exercise and Risk of Major Cardiovascular Events in Adult Survivors of Childhood Hodgkin Lymphoma: A Report From the Childhood Cancer Survivor Study

PURPOSE Survivors of Hodgkin lymphoma (HL) are at increased risk of treatment-related cardiovascular (CV) events; whether exercise modifies this risk is unknown. METHODS Survivors of HL (n = 1,187; median age, 31.2 years) completed a questionnaire evaluating vigorous-intensity exercise behavior. CV events were collected in follow-up questionnaires and graded according to Common Terminology Criteria for Adverse Events (version 4.03). The primary end point was incidence of any major (grade 3 to 5) CV event. Poisson regression analyses were used to estimate the association between exercise exposure (metabolic equivalent [MET] hours/week(-1)) and risk of major CV events after adjustment for clinical covariates and cancer treatment. RESULTS Median follow-up was 11.9 years (range, 1.7 to 14.3 years). Cumulative incidence of any CV event was 12.2% at 10 years for survivors reporting 0 MET hours/week(-1) compared with 5.2% for those reporting ≥ 9 MET hours/week(-1). In multivariable analyses, the incidence of any CV event decreased across increasing MET categories (Ptrend = .002). Compared with survivors reporting 0 MET hours/week(-1), the adjusted rate ratio for any CV event was 0.87 (95% CI, 0.56 to 1.34) for 3 to 6 MET hours/week(-1), 0.45 (95% CI, 0.26 to 0.80) for 9 to 12 MET hours/week(-1), and 0.47 (95% CI, 0.23 to 0.95) for 15 to 21 MET hours/week(-1). Adherence to national vigorous intensity exercise guidelines (ie, ≥ 9 MET hours/week(-1)) was associated with a 51% reduction in the risk of any CV event in comparison with not meeting the guidelines (P = .002). CONCLUSION Vigorous exercise was associated with a lower risk of CV events in a dose-dependent manner independent of CV risk profile and treatment in survivors of HL.

Medication Barriers Predict Adolescent Transplant Recipients’ Adherence and Clinical Outcomes at 18-Month Follow-up

OBJECTIVE To prospectively validate the Parent and Adolescent Medication Barriers Scales (PMBS and AMBS) for assessing perceived barriers to medication adherence in adolescent transplant recipients by examining the relations of perceived barriers to medication adherence and clinical outcomes at 18-month follow-up. METHODS Of the 82 adolescent recipients enrolled in the initial cohort, 66 families participated in the follow-up. Relations among barriers, adherence, and clinical outcomes were examined. RESULTS Reported barriers demonstrated temporal stability over an extended span of time. Adolescent-perceived barriers of Disease Frustration/Adolescent Issues and parent-perceived barriers of Regimen Adaptation/Cognitive Issues were associated with poorer adherence to medication taking at follow-up. Interestingly, medical complications and mortality were significantly associated with both parent and adolescent-perceived ingestion issues barriers. CONCLUSIONS Barriers to adherence are essential to address in an effort to ameliorate adherence difficulties and potentially reduce the incidence of medical complications.

Adolescent Predictors of Emerging Adulthood Milestones in Youth with Spina Bifida

OBJECTIVE To examine the predictive utility of demographic (illness status and SES), individual (neurocognitive functioning and intrinsic motivation), and family-based (parental intrusiveness) factors during adolescence on the achievement of emerging adulthood milestones in youth with and without spina bifida (SB). METHODS Questionnaire and observational data were collected from 14/15-year-old adolescents with SB, typically developing peers, mothers, and teachers. Emerging adulthood milestones (i.e., leaving home, attending college, employment, romantic relationship experience, and number of friendships) were assessed at ages 18/19 years in the full sample and subset of youth who graduated from high school. RESULTS Typically, developing youth were more likely to achieve milestones compared to youth with SB in the full sample but not when only high school graduates were compared. Executive function, SES, intrinsic motivation, and parental intrusiveness emerged as significant predictors for particular milestones. CONCLUSIONS Interventions targeting executive function, intrinsic motivation, and parenting behavior may facilitate achievement of emerging adulthood milestones.

Cancer-Related Stress and Complementary and Alternative Medicine: A Review

A cancer diagnosis elicits strong psychophysiological reactions that characterize stress. Stress is experienced by all patients but is usually not discussed during patient-healthcare professional interaction; thus underdiagnosed, very few are referred to support services. The prevalence of CAM use in patients with history of cancer is growing. The purpose of the paper is to review the aspects of cancer-related stress and interventions of commonly used complementary and alternative techniques/products for amelioration of cancer-related stress. Feasibility of intervention of several CAM techniques and products commonly used by cancer patients and survivors has been established in some cancer populations. Efficacy of some CAM techniques and products in reducing stress has been documented as well as stress-related symptoms in patients with cancer such as mindfulness-based stress reduction, yoga, Tai Chi Chuan, acupuncture, energy-based techniques, and physical activity. Much of the research limitations include small study samples and variety of intervention length and content. Efficacy and safety of many CAM techniques and some herbs and vitamin B and D supplements need to be confirmed in further studies using scientific methodology. Several complementary and alternative medicine therapies could be integrated into standard cancer care to ameliorate cancer-related stress.

Posttraumatic Growth in Young Adults Who Experienced Serious Childhood Illness: A Mixed-Methods Approach

Sixty young adult survivors of a serious childhood illness completed quantitative and qualitative measures assessing the relationship between specific disease and distress factors and posttraumatic growth (PTG). Individuals who had recovered from their illness reported greater growth than those who were currently experiencing their illness. The regression model accounted for 47% of the variance in PTG, with perceived severity, illness status, and posttraumatic stress symptoms emerging as significant predictors. Qualitative analyses identified salient positive and negative factors associated with having had an illness, such as a positive shift in perspective and frequent medical requirements. Being past the daily demands of illness management may allow for greater PTG. Realization of positive aspects of having had an illness may require prompting.

Predictors of Long-Term Health-Related Quality of Life in Adolescent Solid Organ Transplant Recipients

OBJECTIVES This study aimed to identify prospective predictors of health-related quality of life (HRQOL) for adolescent solid organ (kidney, liver, heart, lung) transplant recipients. METHODS Data regarding demographics, individual/transplant characteristics, and environmental characteristics were gathered from 66 adolescent transplant recipients and their families at baseline and used to predict the physical functioning, mental health, and general health perceptions domains of HRQOL 18 months later. RESULTS Baseline levels of HRQOL explained the greatest amount of variance in levels of HRQOL at follow-up; however, specific demographic (i.e., income), individual/transplant (i.e., adherence, frequency of rescheduled clinic appointments, and presence of a rejection episode), and environmental factors (i.e., family conflict) contributed to the variance in HRQOL domains beyond baseline levels. CONCLUSIONS This study identified certain modifiable individual and environmental factors and non-modifiable risk factors associated with lower future HRQOL. Transplant centers should begin screening and addressing these factors to potentially improve HRQOL.

Friendships of Children and Adolescents With Spina Bifida: Social Adjustment, Social Performance, and Social Skills

OBJECTIVE To characterize dyadic and general friendships of youth with spina bifida (SB). METHODS Families of youth with SB recruited a peer to participate; 106 dyads participated. Youth with SB and peers completed questionnaires and interviews regarding characteristics of the dyadic friendship and each individuals general friendships. RESULTS Youth with SB and their peers were similar in many ways. However, youth with SB rated the friendship as closer and were more likely to see peers as best friends rather than the reverse. Regarding general friendships, youth with SB spent fewer days with friends, reported lower levels of companionship, security, and closeness in their friendships, and reported lower levels of emotional support from peers and family. CONCLUSIONS Youth with SB experience significant differences in the quality and reciprocation of friendships. Targeted interventions may assist youth in developing high quality friendships.

Mother–Adolescent Agreement Regarding Decision-Making Autonomy: A Longitudinal Comparison of Families of Adolescents with and without Spina Bifida

OBJECTIVE Longitudinal comparison of mother and adolescent agreement regarding decision-making autonomy for adolescents with and without spina bifida (SB). METHODS Forty-two mother-adolescent dyads of adolescents with SB and 55 comparison dyads reported on who was responsible for decision-making across five waves of data collection, beginning at age 8 or 9 years through age 16 or 17 years. RESULTS The proportion of tasks that dyads agreed were decided by adolescents increased over time for both samples beginning at age 12 or 13 years, but appeared to be delayed by roughly two years for youth with SB and was lower for youth with SB from lower socioeconomic (SES) backgrounds. Mothers and adolescents with low SES demonstrated higher proportions of tasks that dyads agreed were decided by mothers. CONCLUSIONS SB and low SES are risk factors for lower levels of agreed-upon decision-making autonomy. Future studies should examine how parent-adolescent agreement regarding autonomy relates to psychosocial outcomes.