Jan R. Böhnke

Feedback and therapist effects in the context of treatment outcome and treatment length

Abstract Objective: This study estimates feedback and therapist effects and tests the predictive value of therapists’ and patient attitudes toward psychometric feedback for treatment outcome and length. Methods: Data of 349 outpatients and 44 therapists in private practices were used. Separate multilevel analyses were conducted to estimate predictors and feedback and therapist effects. Results: Around 5.88% of the variability in treatment outcome and 8.89% in treatment length were attributed to therapists. There was no relationship between the average effectiveness of therapists and the average length of their treatments. Initial impairment, early alliance, number of diagnoses, feedback as well as therapists’ and patients’ attitudes toward feedback were significant predictors of treatment outcome. Treatments tended to be longer for patients with a higher number of approved sessions by the insurance company, with higher levels of interpersonal distress at intake, and for those who developed negatively (negative feedback) over the course of their treatment. Conclusions: Therapist effects on treatment outcome and treatment length in routine care seem to be relevant predictors in the context of feedback studies. Therapists’ attitudes toward and use of feedback as well as patients’ attitudes toward feedback should be further investigated in future research on psychometric feedback.

Factors of psychological distress: clinical value, measurement substance, and methodological artefacts

PurposePsychometric models and statistical techniques are cornerstones of research into latent structures of specific psychopathology and general mental health. We discuss “pivot points” for future research efforts from a psychometric epidemiology perspective, emphasising sampling and selection processes of both indicators that guide data collection as well as samples that are confronted with them.MethodFirst, we discuss how a theoretical model of psychopathology determines which empirical indicators (questions, diagnoses, etc.) and modelling methods are appropriate to test its implications. Second, we deal with how different research designs introduce different (co-)variances between indicators, potentially leading to a different understanding of latent structures. Third, we discuss widening the range of statistical models available within the “psychometrics class”: the inclusion of categorical approaches can help to enlighten the debate on the structure of psychopathology and agreement on a minimal set of models might lead to greater convergence between studies. Fourth, we deal with aspects of methodology that introduce spurious (co-)variance in latent structure analysis (response styles, clustered data) and differential item functioning to gather more detailed information and to guard against over-generalisation of results, which renders assessments unfair.ConclusionsBuilding on established insights, future research efforts should be more explicit about their theoretical understanding of psychopathology and how the analysis of a given indicator–respondent set informs this theoretical model. A coherent treatment of theoretical assumptions, indicators, and samples holds the key to building a comprehensive account of the latent structures of different types of psychopathology and mental health in general.

Entwicklung und psychometrische Kennwerte des Fragebogens zur Evaluation von Psychotherapieverläufen (FEP)

Zusammenfassung. Die kontinuierliche Messung des therapeutischen Fortschritts ist im Rahmen der Qualitatssicherung in Psychotherapie und Psychiatrie ein zentrales Element. In dieser Arbeit wird ein...

Reliability of Therapist Effects in Practice-Based Psychotherapy Research: A Guide for the Planning of Future Studies

This paper aims to provide researchers with practical information on sample sizes for accurate estimations of therapist effects (TEs). The investigations are based on an integrated sample of 48,648 patients treated by 1800 therapists. Multilevel modeling and resampling were used to realize varying sample size conditions to generate empirical estimates of TEs. Sample size tables, including varying sample size conditions, were constructed and study examples given. This study gives an insight into the potential size of the TE and provides researchers with a practical guide to aid the planning of future studies in this field.

Similarities and differences of emotions in human–machine and human–human interactions: what kind of emotions are relevant for future companion systems?

Cognitive-technical intelligence is envisioned to be constantly available and capable of adapting to the users emotions. However, the question is: what specific emotions should be reliably recognised by intelligent systems? Hence, in this study, we have attempted to identify similarities and differences of emotions between human–human (HHI) and human–machine interactions (HMI). We focused on what emotions in the experienced scenarios of HMI are retroactively reflected as compared with HHI. The sample consisted of N = 145 participants, who were divided into two groups. Positive and negative scenario descriptions of HMI and HHI were given by the first and second groups, respectively. Subsequently, the participants evaluated their respective scenarios with the help of 94 adjectives relating to emotions. The correlations between the occurrences of emotions in the HMI versus HHI were very high. The results do not support the statement that only a few emotions in HMI are relevant. Practitioner Summary: This study sought to identify the relevant emotions in different technical domains their companion systems tend to use. Overall, the 20 essential emotions found as highly relevant for HMI were as follows: (i) positive, i.e. satisfied, pleased, happy, relieved, pleasant, well, serene, optimistic, confident and self-confident and (ii) negative, i.e. annoyed, aggravated, impatient, angry, unsatisfied, displeased, irritable, frustrated, enraged and tense.

The Stability of Extreme Response Style and Acquiescence Over 8 Years

This study investigated the stability of extreme response style (ERS) and acquiescence response style (ARS) over a period of 8 years. ERS and ARS were measured with item sets drawn randomly from a large pool of items used in an ongoing German panel study. Latent-trait-state-occasion and latent-state models were applied to test the relationship between time-specific (state) response style behaviors and time-invariant trait components of response styles. The results show that across different random item samples, on average between 49% and 59% of the variance in the state response style factors was explained by the trait response style factors. This indicates that the systematic differences respondents show in their preferences for certain response categories are remarkably stable over a period of 8 years. The stability of ERS and ARS implies that it is important to consider response styles in the analysis of self-report data from polytomous rating scales, especially in longitudinal studies aimed at investigating stability in substantive traits. Furthermore, the stability of response styles raises the question in how far they might be considered trait-like latent variables themselves that could be of substantive interest.

Negative affectivity as a transdiagnostic factor in patients with common mental disorders.

BACKGROUND Screening and monitoring systems are increasingly used in psychotherapy, but it has been questioned whether outcome measurement using multiple questionnaires is warranted. Arguably, type and number of assessment instruments should be determined by empirical research. This study investigated the latent factor structure of a multi-dimensional outcome measurement strategy used in English services aligned to the Improving Access to Psychological Therapies (IAPT) programme. METHODS Factor analyses and structural equation models were performed on 11,939 intake assessments of outpatients accessing an IAPT service between 2008 and 2010. We examined whether three routinely employed instruments (PHQ-9 for depression, GAD-7 for anxiety, WSAS for functional impairment) assess empirically different dimensions. RESULTS The instruments were found to assess mainly one general dimension and only some items of the GAD-7 and WSAS assess unique variance beyond this general dimension. In a structural equation model the disorder-specific factor scores were predicted by patients׳ diagnostic categories. LIMITATIONS Since a large naturalistic data base was used, missing data for diagnoses and scale items were encountered. Diagnoses were obtained with brief case-finding measures rather than structured diagnostic interviews. CONCLUSION Although the items seem to address mostly one dimension, some variance is due to differences between individuals in anxiety and impairment. While this generally supports multi-dimensional assessment in a primary care population, the clinical upshot of the study is to concentrate attention on transdiagnostic factors as a target for treatment.

Calibrating well-being, quality of life and common mental disorder items: psychometric epidemiology in public mental health research.

Background The assessment of ‘general health and well-being’ in public mental health research stimulates debates around relative merits of questionnaire instruments and their items. Little evidence regarding alignment or differential advantages of instruments or items has appeared to date. Aims Population-based psychometric study of items employed in public mental health narratives. Method Multidimensional item response theory was applied to General Health Questionnaire (GHQ-12), Warwick-Edinburgh Mental Well-being Scale (WEMWBS) and EQ-5D items (Health Survey for England, 2010–2012; n = 19 290). Results A bifactor model provided the best account of the data and showed that the GHQ-12 and WEMWBS items assess mainly the same construct. Only one item of the EQ-5D showed relevant overlap with this dimension (anxiety/depression). Findings were corroborated by comparisons with alternative models and cross-validation analyses. Conclusions The consequences of this lack of differentiation (GHQ-12 v. WEMWBS) for mental health and well-being narratives deserves discussion to enrich debates on priorities in public mental health and its assessment.

Using Item and Test Information to Optimize Targeted Assessments of Psychological Distress

The need for efficient clinical assessment instruments has been growing during the past years. In the current application, the item information (item response theory) is used to evaluate and build fixed short versions. The method was applied to a questionnaire measuring psychological distress and data were collected from two mixed outpatient and general population samples. After fitting the partial credit model, two short versions were built: one to increase efficiency in screening applications; the other for the monitoring of high distress patients. A cross-validation bootstrap procedure is proposed to check whether the short versions are more efficient than alternative item selections. Using the partial credit model, the results from short and full versions can be compared on score level, which improves the flexibility of the assessment. The discussion focuses on the model selection and on how many items are realistically needed in routine assessments of psychological distress.

Item bank measurement of depression: will one dimension work?

Wahl et al. [1] have provided an elegant exemplar of the psychometric calibration approach known as item banking, applying modern item response theory (IRT) to multiinstrument depression severity measurement. Their inclusive approach, integrative data analysis, and identification of commensurate items from a large number of common scales are enriched by test information functions for each item set, instrument-to-common-metric mapping functions and threshold locations. The authors are to be congratulated on offering a comprehensive standardized view of what might otherwise be 11 unrelated depression metrics. Applied research could unite behind their unidimensional measures.