M. Michele Athay

The relationship between change in therapeutic alliance ratings and improvement in youth symptom severity: Whose ratings matter the most?

This paper presents the psychometric evaluation of brief measures of therapeutic alliance (TA) for youths, clinicians and caregivers and a longitudinal analysis of relationships between changes in TA and changes in youth symptom and functioning severity. Psychometric analyses using methods from Classical Test Theory, Item Response Theory, and Factor Analysis indicate that the measures of TA used in this study offer something new for both practice and research. The measures have variability, sensitivity to change over time, brevity and can be used with multiple parties through parallel forms. The longitudinal analyses, employing hierarchical linear modeling with time-varying covariates, found that TA ratings of the clinician correlated with symptom improvement as rated by the clinician, caregiver and youth. Additional analyses showed that decreases in clinician-rated youth TA was most important in predicting a lower rate of youth improvement. Implications for future research and clinical practice are discussed.

The symptoms and functioning severity scale (SFSS): psychometric evaluation and discrepancies among youth, caregiver, and clinician ratings over time.

This paper describes the development and psychometric evaluation of the symptoms and functioning severity scale (SFSS), which includes three parallel forms to systematically capture clinician, youth, and caregiver perspectives of youth symptoms on a frequent basis. While there is widespread consensus that different raters of youth psychopathology vary significantly in their assessment, this is the first paper that specifically investigated the discrepancies among clinician, youth, and caregiver ratings throughout the treatment process within a community mental health setting. Results for all three respondent versions indicated the SFSS is a psychometrically sound instrument for use in this population. Significant discrepancies in scores existed at baseline among the three respondents. Longitudinal analyses reveal the youth-clinician and caregiver-clinician score discrepancies decreased significantly over time. Differences by youth gender existed for caregiver-clinician discrepancies. The average youth-caregiver score discrepancy remained consistent throughout treatment. Implications for future research and clinical practice are discussed.

The Peabody Treatment Progress Battery: History and Methods for Developing a Comprehensive Measurement Battery for Youth Mental Health

There is increased need for comprehensive, flexible, and evidence-based approaches to measuring the process and outcomes of youth mental health treatment. This paper introduces a special issue dedicated to the Peabody Treatment Progress Battery (PTPB), a battery of measures created to meet this need. The PTPB is an integrated set of brief, reliable, and valid instruments that can be administered efficiently at low cost and can provide systematic feedback for use in treatment planning. It includes eleven measures completed by youth, caregivers, and/or clinicians that assess clinically-relevant constructs such as symptom severity, therapeutic alliance, life satisfaction, motivation for treatment, hope, treatment expectations, caregiver strain, and service satisfaction. This introductory article describes the rationale for the PTPB and its development and evaluation, detailing the specific analytic approaches utilized by the different papers in the special issue and a description of the study and samples from which the participants were taken.

Measuring change for a multidimensional test using a generalized explanatory longitudinal item response model

Even though many educational and psychological tests are known to be multidimensional, little research has been done to address how to measure individual differences in change within an item response theory framework. In this paper, we suggest a generalized explanatory longitudinal item response model to measure individual differences in change. New longitudinal models for multidimensional tests and existing models for unidimensional tests are presented within this framework and implemented with software developed for generalized linear models. In addition to the measurement of change, the longitudinal models we present can also be used to explain individual differences in change scores for person groups (e.g., learning disabled students versus non-learning disabled students) and to model differences in item difficulties across item groups (e.g., number operation, measurement, and representation item groups in a mathematics test). An empirical example illustrates the use of the various models for measuring individual differences in change when there are person groups and multiple skill domains which lead to multidimensionality at a time point.

Measuring Youths’ Perceptions of Counseling Impact: Description, Psychometric Evaluation, and Longitudinal Examination of the Youth Counseling Impact Scale v.2

The Youth Counseling Impact Scale (YCIS) is an empirically validated treatment progress measure that assesses youths’ perceptions of the short term effectiveness of therapy. Since its initial publication, the original 10-item measure has been shortened to ease measurement burden and revised to include a question about a youth’s insight into his or her strengths. The current study describes the development of the revised YCIS (v.2) and evaluates its psychometric properties. Additionally, this study examines whether the YCIS (v.2) total score or subscale scores change over time and investigates whether there are gender or age differences for youths’ perceptions of the impact of therapy. Results found the revised version obtained comparable information to that of the original measure, and that the revised version retained the factor structure of the original model with one primary general factor of Counseling Impact and two secondary factors (Insight and Change). Results also suggested that while the YCIS (v.2) total score and Change subscale score did not change linearly over the course of treatment, the Insight subscale score showed a small but significant linear increase over time. No significant differences in YCIS scores based on youth age or gender were found. The implication of these findings, the clinical and empirical utility of this measure, and its limitations are discussed.

Caregiver life satisfaction: relationship to youth symptom severity through treatment.

This study utilized the Satisfaction with Life Scale to investigate the life satisfaction of caregivers for youth receiving mental health services (N = 383). Specifically, this study assessed how caregiver life satisfaction relates to youth symptom severity throughout treatment. Hierarchical linear modeling with a time-varying covariate was used to estimate the linear trajectory of caregiver life satisfaction and how it relates to youth symptom severity as rated by caregivers, youth, and clinicians. Results found initial caregiver life satisfaction was inversely related to caregiver and clinician rated youth symptom severity. In addition, subsequent caregiver life satisfaction demonstrated a small but significant relationship to changes in youth symptom severity during treatment where a decrease in youth symptoms corresponded to an increase in caregiver life satisfaction, and vice versa. Caregiver background characteristics related to higher life satisfaction included being (a) married, a birth parent, and younger than 40 years old, and (b) having the absence of previous diagnoses of an emotional, behavioral, or substance use disorder. Despite significant change over time, caregivers of clinically referred youth demonstrated low levels of life satisfaction throughout youth treatment. Given the bidirectional influences on one another, tending to the well-being of caregivers may positively influence both caregivers and youths.