Christopher J. Whitaker

Measures of Diversity in Classifier Ensembles and Their Relationship with the Ensemble Accuracy

Diversity among the members of a team of classifiers is deemed to be a key issue in classifier combination. However, measuring diversity is not straightforward because there is no generally accepted formal definition. We have found and studied ten statistics which can measure diversity among binary classifier outputs (correct or incorrect vote for the class label): four averaged pairwise measures (the Q statistic, the correlation, the disagreement and the double fault) and six non-pairwise measures (the entropy of the votes, the difficulty index, the Kohavi-Wolpert variance, the interrater agreement, the generalized diversity, and the coincident failure diversity). Four experiments have been designed to examine the relationship between the accuracy of the team and the measures of diversity, and among the measures themselves. Although there are proven connections between diversity and accuracy in some special cases, our results raise some doubts about the usefulness of diversity measures in building classifier ensembles in real-life pattern recognition problems.

Limits on the majority vote accuracy in classifier fusion

Abstract We derive upper and lower limits on the majority vote accuracy with respect to individual accuracy p, the number of classifiers in the pool (L), and the pairwise dependence between classifiers, measured by Yule’s Q statistic. Independence between individual classifiers is typically viewed as an asset in classifier fusion. We show that the majority vote with dependent classifiers can potentially offer a dramatic improvement both over independent classifiers and over an individual classifier with accuracy p. A functional relationship between the limits and the pairwise dependence Q is derived. Two patterns of the joint distribution for classifier outputs (correct/incorrect) are identified to derive the limits: the pattern of success and the pattern of failure. The results support the intuition that negative pairwise dependence is beneficial although not straightforwardly related to the accuracy. The pattern of success showed that for the highest improvement over p, all pairs of classifiers in the pool should have the same negative dependence.

Who benefits and how does it work? Moderators and mediators of outcome in an effectiveness trial of a parenting intervention.

We examined mediators and moderators of change in conduct problems, in a multiagency randomized trial of the Incredible Years parenting program. Preschoolers (n = 153) at risk for conduct problems were randomly assigned to intervention (n = 104) and wait-list (n = 49) groups. Boys and younger children, and those with more depressed mothers, tended to show greater improvement in conduct problems post-intervention. Other risk factors (i.e., teen or single parenthood, very low income, high initial levels of problem behavior) showed no predictive effects, implying intervention was at least as successful at helping the most disadvantaged families, compared to more advantaged. Mediator analyses found change in positive parenting skill predicted change in conduct problems.

Is independence good for combining classifiers

Independence between individual classifiers is typically viewed as an asset in classifier fusion. We study the limits on the majority vote accuracy when combining dependent classifiers. Q statistics are used to measure the dependence between classifiers. We show that dependent classifiers could offer a dramatic improvement over the individual accuracy. However, the relationship between dependency and accuracy of the pool is ambivalent. A synthetic experiment demonstrates the intuitive result that, in general, negative dependence is preferable.

Mindfulness-Based Cognitive Therapy for Preventing Relapse in Recurrent Depression: A Randomized Dismantling Trial.

Objective: We compared mindfulness-based cognitive therapy (MBCT) with both cognitive psychological education (CPE) and treatment as usual (TAU) in preventing relapse to major depressive disorder (MDD) in people currently in remission following at least 3 previous episodes. Method: A randomized controlled trial in which 274 participants were allocated in the ratio 2:2:1 to MBCT plus TAU, CPE plus TAU, and TAU alone, and data were analyzed for the 255 (93%; MBCT = 99, CPE = 103, TAU = 53) retained to follow-up. MBCT was delivered in accordance with its published manual, modified to address suicidal cognitions; CPE was modeled on MBCT, but without training in meditation. Both treatments were delivered through 8 weekly classes. Results: Allocated treatment had no significant effect on risk of relapse to MDD over 12 months follow-up, hazard ratio for MBCT vs. CPE = 0.88, 95% CI [0.58, 1.35]; for MBCT vs. TAU = 0.69, 95% CI [0.42, 1.12]. However, severity of childhood trauma affected relapse, hazard ratio for increase of 1 standard deviation = 1.26 (95% CI [1.05, 1.50]), and significantly interacted with allocated treatment. Among participants above median severity, the hazard ratio was 0.61, 95% CI [0.34, 1.09], for MBCT vs. CPE, and 0.43, 95% CI [0.22, 0.87], for MBCT vs. TAU. For those below median severity, there were no such differences between treatment groups. Conclusion: MBCT provided significant protection against relapse for participants with increased vulnerability due to history of childhood trauma, but showed no significant advantage in comparison to an active control treatment and usual care over the whole group of patients with recurrent depression.

Long-term effectiveness of a parenting intervention for children at risk of developing conduct disorder

BACKGROUND The typical pattern for intervention outcome studies for conduct problems has been for effect sizes to dissipate over time with decreasing effects across subsequent follow-ups. AIMS To establish whether the short-term positive effects of a parenting programme are sustained longer term. To observe trends, and costs, in health and social service use after intervention. METHOD Parents with children aged 36-59 months at risk of developing conduct disorder (n = 104) received intervention between baseline and first follow-up (6 months after baseline n = 86) in 11 Sure Start areas in North Wales. Follow-ups two (n = 82) and three (n = 79) occurred 12 and 18 months after baseline. Child problem behaviour and parenting skills were assessed via parent self-report and direct observation in the home. RESULTS The significant parent-reported improvements in primary measures of child behaviour, parent behaviour, parental stress and depression gained at follow-up one were maintained to follow-up three, as were improved observed child and parent behaviours. Overall, 63% of children made a minimum significant change (0.3 standard deviations) on the Eyberg Child Behavior Inventory problem scale between baseline and follow-up (using intention-to-treat data), 54% made a large change (0.8 standard deviations) and 39% made a very large change (1.5 standard deviations). Child contact with health and social services had reduced at follow-up three. CONCLUSIONS Early parent-based intervention reduced child antisocial behaviour and benefits were maintained, with reduced reliance on health and social service provision, over time.

The influence of psychological, social and contextual factors on the expression and measurement of awareness in early-stage dementia: testing a biopsychosocial model

Insufficient attention has been paid to the influence of psychological and social factors on discrepancy‐based measures of awareness.

Generalized method for adaptive randomization in clinical trials

A flexible, generalized method of treatment allocation is proposed. The method uses a set of controlling parameters that enables the generic algorithm to produce a family of possible outcomes ranging from simple randomization to deterministic allocation. The method controls balance at stratum level, stratification level and overall without detriment to the predictability of the method. The paper lists the desirable characteristics of allocation methods and shows that the proposed method fulfils the majority and is easy to use in the clinical context, once the coding has been established. An explanation of the method for 2, 3 and 4 treatment group allocations is given. Simulations demonstrate the flexibility of the method.

Treatment fidelity as a predictor of behaviour change in parents attending group-based parent training

BACKGROUND Change in parenting skills, particularly increased positive parenting, has been identified as the key component of successful evidence-based parent training (PT), playing a causal role in subsequent child behaviour change for both prevention and treatment of Conduct Disorder. The amount of change in parenting skills observed after PT varies and may be accounted for by both the content of the programme and by the level of PT implementer process skills. Such variation in implementer skills is an important component in the assessment of treatment fidelity, itself an essential factor in successful intervention outcome. AIMS To establish whether the Leader Observation Tool, a reliable and valid process skills fidelity measure, can predict change in parenting skills after attendance on the Incredible Years PT programme. RESULTS Positive leader skills categories of the Leader Observation Tool significantly predicted change in both parent-reported and independently observed parenting skills behaviour, which in turn, predicted change in child behaviour outcome. CONCLUSIONS Delivering an intervention with a high level of treatment fidelity not only preserves the behaviour change mechanisms of the intervention, but can also predict parental behaviour change, which itself predicts child behaviour change as a result of treatment.

Maintenance cognitive stimulation therapy for dementia: single-blind, multicentre, pragmatic randomised controlled trial

BACKGROUND There is good evidence for the benefits of short-term cognitive stimulation therapy for dementia but little is known about possible long-term effects. AIMS To evaluate the effectiveness of maintenance cognitive stimulation therapy (CST) for people with dementia in a single-blind, pragmatic randomised controlled trial including a substudy with participants taking acetylcholinesterase inhibitors (AChEIs). METHOD The participants were 236 people with dementia from 9 care homes and 9 community services. Prior to randomisation all participants received the 7-week, 14-session CST programme. The intervention group received the weekly maintenance CST group programme for 24 weeks. The control group received usual care. Primary outcomes were cognition and quality of life (clinical trial registration: ISRCTN26286067). RESULTS For the intervention group at the 6-month primary end-point there were significant benefits for self-rated quality of life (Quality of Life in Alzheimers Disease (QoL-AD) P = 0.03). At 3 months there were improvements for proxy-rated quality of life (QoL-AD P = 0.01, Dementia Quality of Life scale (DEMQOL) P = 0.03) and activities of daily living (P = 0.04). The intervention subgroup taking AChEIs showed cognitive benefits (on the Mini-Mental State Examination) at 3 (P = 0.03) and 6 months (P = 0.03). CONCLUSIONS Continuing CST improves quality of life; and improves cognition for those taking AChEIs.