Daniel Almirall

A Tutorial on Propensity Score Estimation for Multiple Treatments Using Generalized Boosted Models

The use of propensity scores to control for pretreatment imbalances on observed variables in non-randomized or observational studies examining the causal effects of treatments or interventions has become widespread over the past decade. For settings with two conditions of interest such as a treatment and a control, inverse probability of treatment weighted estimation with propensity scores estimated via boosted models has been shown in simulation studies to yield causal effect estimates with desirable properties. There are tools (e.g., the twang package in R) and guidance for implementing this method with two treatments. However, there is not such guidance for analyses of three or more treatments. The goals of this paper are twofold: (1) to provide step-by-step guidance for researchers who want to implement propensity score weighting for multiple treatments and (2) to propose the use of generalized boosted models (GBM) for estimation of the necessary propensity score weights. We define the causal quantities that may be of interest to studies of multiple treatments and derive weighted estimators of those quantities. We present a detailed plan for using GBM to estimate propensity scores and using those scores to estimate weights and causal effects. We also provide tools for assessing balance and overlap of pretreatment variables among treatment groups in the context of multiple treatments. A case study examining the effects of three treatment programs for adolescent substance abuse demonstrates the methods.

Cognitive Behavior Therapy Augmentation of Pharmacotherapy in Pediatric Obsessive-Compulsive Disorder The Pediatric OCD Treatment Study II (POTS II) Randomized Controlled Trial

CONTEXT The extant literature on the treatment of pediatric obsessive-compulsive disorder (OCD) indicates that partial response to serotonin reuptake inhibitors (SRIs) is the norm and that augmentation with short-term OCD-specific cognitive behavior therapy (CBT) may provide additional benefit. OBJECTIVE To examine the effects of augmenting SRIs with CBT or a brief form of CBT, instructions in CBT delivered in the context of medication management. DESIGN, SETTING, AND PARTICIPANTS A 12-week randomized controlled trial conducted at 3 academic medical centers between 2004 and 2009, involving 124 pediatric outpatients between the ages of 7 and 17 years with OCD as a primary diagnosis and a Childrens Yale-Brown Obsessive Compulsive Scale score of 16 or higher despite an adequate SRI trial. INTERVENTIONS Participants were randomly assigned to 1 of 3 treatment strategies that included 7 sessions over 12 weeks: 42 in the medication management only, 42 in the medication management plus instructions in CBT, and 42 in the medication management plus CBT; the last included 14 concurrent CBT sessions. MAIN OUTCOME MEASURES Whether patients responded positively to treatment by improving their baseline obsessive-compulsive scale score by 30% or more and demonstrating a change in their continuous scores over 12 weeks. RESULTS The medication management plus CBT strategy was superior to the other 2 strategies on all outcome measures. In the primary intention-to-treat analysis, 68.6% (95% CI, 53.9%-83.3%) in the plus CBT group were considered responders, which was significantly better than the 34.0% (95% CI, 18.0%-50.0%) in the plus instructions in CBT group, and 30.0% (95% CI, 14.9%-45.1%) in the medication management only group. The results were similar in pairwise comparisons with the plus CBT strategy being superior to the other 2 strategies (P < .01 for both). The plus instructions in CBT strategy was not statistically superior to medication management only (P = .72). The number needed-to-treat analysis with the plus CBT vs medication management only in order to see 1 additional patient at week 12, on average, was estimated as 3; for the plus CBT vs the plus instructions in CBT strategy, the number needed to treat was also estimated as 3; for the plus instructions in CBT vs medication management only the number needed to treat was estimated as 25. CONCLUSIONS Among patients aged 7 to 17 years with OCD and partial response to SRI use, the addition of CBT to medication management compared with medication management alone resulted in a significantly greater response rate, whereas augmentation of medication management with the addition of instructions in CBT did not. TRIAL REGISTRATION clinicaltrials.gov Identifier: NCT00074815.

Communication interventions for minimally verbal children with autism: a sequential multiple assignment randomized trial.

OBJECTIVE This study tested the effect of beginning treatment with a speech-generating device (SGD) in the context of a blended, adaptive treatment design for improving spontaneous, communicative utterances in school-aged, minimally verbal children with autism. METHOD A total of 61 minimally verbal children with autism, aged 5 to 8 years, were randomized to a blended developmental/behavioral intervention (JASP+EMT) with or without the augmentation of a SGD for 6 months with a 3-month follow-up. The intervention consisted of 2 stages. In stage 1, all children received 2 sessions per week for 3 months. Stage 2 intervention was adapted (by increased sessions or adding the SGD) based on the childs early response. The primary outcome was the total number of spontaneous communicative utterances; secondary measures were the total number of novel words and total comments from a natural language sample. RESULTS Primary aim results found improvements in spontaneous communicative utterances, novel words, and comments that all favored the blended behavioral intervention that began by including an SGD (JASP+EMT+SGD) as opposed to spoken words alone (JASP+EMT). Secondary aim results suggest that the adaptive intervention beginning with JASP+EMT+SGD and intensifying JASP+EMT+SGD for children who were slow responders led to better posttreatment outcomes. CONCLUSION Minimally verbal school-aged children can make significant and rapid gains in spoken spontaneous language with a novel, blended intervention that focuses on joint engagement and play skills and incorporates an SGD. Future studies should further explore the tailoring design used in this study to better understand childrens response to treatment. Clinical trial registration information-Developmental and Augmented Intervention for Facilitating Expressive Language (CCNIA); http://clinicaltrials.gov/; NCT01013545.

Experimental Design and Primary Data Analysis Methods for Comparing Adaptive Interventions

In recent years, research in the area of intervention development has been shifting from the traditional fixed-intervention approach to adaptive interventions, which allow greater individualization and adaptation of intervention options (i.e., intervention type and/or dosage) over time. Adaptive interventions are operationalized via a sequence of decision rules that specify how intervention options should be adapted to an individuals characteristics and changing needs, with the general aim to optimize the long-term effectiveness of the intervention. Here, we review adaptive interventions, discussing the potential contribution of this concept to research in the behavioral and social sciences. We then propose the sequential multiple assignment randomized trial (SMART), an experimental design useful for addressing research questions that inform the construction of high-quality adaptive interventions. To clarify the SMART approach and its advantages, we compare SMART with other experimental approaches. We also provide methods for analyzing data from SMART to address primary research questions that inform the construction of a high-quality adaptive intervention.

Introduction to SMART designs for the development of adaptive interventions: with application to weight loss research

The management of many health disorders often entails a sequential, individualized approach whereby treatment is adapted and readapted over time in response to the specific needs and evolving status of the individual. Adaptive interventions provide one way to operationalize the strategies (e.g., continue, augment, switch, step-down) leading to individualized sequences of treatment. Often, a wide variety of critical questions must be answered when developing a high-quality adaptive intervention. Yet, there is often insufficient empirical evidence or theoretical basis to address these questions. The Sequential Multiple Assignment Randomized Trial (SMART)—a type of research design—was developed explicitly for the purpose of building optimal adaptive interventions by providing answers to such questions. Despite increasing popularity, SMARTs remain relatively new to intervention scientists. This manuscript provides an introduction to adaptive interventions and SMARTs. We discuss SMART design considerations, including common primary and secondary aims. For illustration, we discuss the development of an adaptive intervention for optimizing weight loss among adult individuals who are overweight.

Designing a pilot sequential multiple assignment randomized trial for developing an adaptive treatment strategy

There is growing interest in how best to adapt and readapt treatments to individuals to maximize clinical benefit. In response, adaptive treatment strategies (ATS), which operationalize adaptive, sequential clinical decision making, have been developed. From a patients perspective an ATS is a sequence of treatments, each individualized to the patients evolving health status. From a clinicians perspective, an ATS is a sequence of decision rules that input the patients current health status and output the next recommended treatment. Sequential multiple assignment randomized trials (SMART) have been developed to address the sequencing questions that arise in the development of ATSs, but SMARTs are relatively new in clinical research. This article provides an introduction to ATSs and SMART designs. This article also discusses the design of SMART pilot studies to address feasibility concerns, and to prepare investigators for a full-scale SMART. We consider an example SMART for the development of an ATS in the treatment of pediatric generalized anxiety disorders. Using the example SMART, we identify and discuss design issues unique to SMARTs that are best addressed in an external pilot study prior to the full-scale SMART. We also address the question of how many participants are needed in a SMART pilot study. A properly executed pilot study can be used to effectively address concerns about acceptability and feasibility in preparation for (that is, prior to) executing a full-scale SMART.

Q-Learning: A Data Analysis Method for Constructing Adaptive Interventions

Increasing interest in individualizing and adapting intervention services over time has led to the development of adaptive interventions. Adaptive interventions operationalize the individualization of a sequence of intervention options over time via the use of decision rules that input participant information and output intervention recommendations. We introduce Q-learning, which is a generalization of regression analysis to settings in which a sequence of decisions regarding intervention options or services is made. The use of Q is to indicate that this method is used to assess the relative quality of the intervention options. In particular, we use Q-learning with linear regression to estimate the optimal (i.e., most effective) sequence of decision rules. We illustrate how Q-learning can be used with data from sequential multiple assignment randomized trials (SMARTs; Murphy, 2005) to inform the construction of a more deeply tailored sequence of decision rules than those embedded in the SMART design. We also discuss the advantages of Q-learning compared to other data analysis approaches. Finally, we use the Adaptive Interventions for Children With ADHD SMART study (Center for Children and Families, University at Buffalo, State University of New York, William E. Pelham as principal investigator) for illustration.

Optimization of behavioral dynamic treatment regimens based on the sequential, multiple assignment, randomized trial (SMART)

Background and purpose A behavioral intervention is a program aimed at modifying behavior for the purpose of treating or preventing disease, promoting health, and/or enhancing well-being. Many behavioral interventions are dynamic treatment regimens, that is, sequential, individualized multicomponent interventions in which the intensity and/or type of treatment is varied in response to the needs and progress of the individual participant. The multiphase optimization strategy (MOST) is a comprehensive framework for development, optimization, and evaluation of behavioral interventions, including dynamic treatment regimens. The objective of optimization is to make dynamic treatment regimens more effective, efficient, scalable, and sustainable. An important tool for optimization of dynamic treatment regimens is the sequential, multiple assignment, randomized trial (SMART). The purpose of this article is to discuss how to develop optimized dynamic treatment regimens within the MOST framework. Methods and results The article discusses the preparation, optimization, and evaluation phases of MOST. It is shown how MOST can be used to develop a dynamic treatment regimen to meet a prespecified optimization criterion. The SMART is an efficient experimental design for gathering the information needed to optimize a dynamic treatment regimen within MOST. One signature feature of the SMART is that randomization takes place at more than one point in time. Conclusion MOST and SMART can be used to develop optimized dynamic treatment regimens that will have a greater public health impact.

Trajectories of Change in Youth Anxiety during Cognitive Behavior Therapy

OBJECTIVE To evaluate changes in the trajectory of youth anxiety following the introduction of specific cognitive-behavior therapy (CBT) components: relaxation training, cognitive restructuring, and exposure tasks. METHOD Four hundred eighty-eight youths ages 7-17 years (50% female; 74% ≤ 12 years) were randomly assigned to receive either CBT, sertraline (SRT), their combination (COMB), or pill placebo (PBO) as part of their participation in the Child/Adolescent Anxiety Multimodal Study (CAMS). Youths in the CBT conditions were evaluated weekly by therapists using the Clinical Global Impression Scale-Severity (CGI-S; Guy, 1976) and the Childrens Global Assessment Scale (CGAS; Shaffer et al., 1983) and every 4 weeks by blind independent evaluators (IEs) using the Pediatric Anxiety Ratings Scale (PARS; RUPP Anxiety Study Group, 2002). Youths in SRT and PBO were included as controls. RESULTS Longitudinal discontinuity analyses indicated that the introduction of both cognitive restructuring (e.g., changing self-talk) and exposure tasks significantly accelerated the rate of progress on measures of symptom severity and global functioning moving forward in treatment; the introduction of relaxation training had limited impact. Counter to expectations, no strategy altered the rate of progress in the specific domain of anxiety that it was intended to target (i.e., somatic symptoms, anxious self-talk, avoidance behavior). CONCLUSIONS Findings support CBT theory and suggest that cognitive restructuring and exposure tasks each make substantial contributions to improvement in youth anxiety. Implications for future research are discussed. (PsycINFO Database Record

Do CRM Systems Cause One-to-One Marketing Effectiveness?

This article provides an assessment of the causal effect of customer relationship management (CRM) applications on one-to-one marketing effectiveness. We use a potential outcomes based propensity score approach to assess this causal effect. We find that firms using CRM systems have greater levels of one-to-one marketing effectiveness. We discuss the strengths and challenges of using the propensity score approach to design and execute CRM related observational studies. We also discuss the applicability of the framework in this paper to study typical causal questions in business and electronic commerce research at the firm, individual and economy levels, and to clarify the assumptions that researchers must make to infer causality from observational data.