C. Bartelink

Deciding on child maltreatment : A literature review on methods that improve decision-making

Assessment and decision-making in child maltreatment cases is difficult. Practitioners face many uncertainties and obstacles during their assessment and decision-making process. Research exhibits shortcomings in this decision-making process. The purpose of this literature review is to identify and discuss methods to overcome these shortcomings. We conducted a systematic review of the published literature on decision-making using PsychINFO and MEDLINE from 2000 through May 2014. We included reviews and quantitative research studies that investigated methods aimed at improving professional decision-making on child abuse and neglect in child welfare and child protection. Although many researchers have published articles on decision-making including ideas and theories to improve professional decision-making, empirical research on these improvements is scarce. Available studies have shown promising results. Structured decision-making has created a greater child-centred and holistic approach that takes the childs family and environment into account, which has made practitioners work more systematically and improved the analysis of complex situations. However, this approach has not improved inter-rater agreement on decisions made. Shared decision-making may improve the participation of parents and children and the quality of decisions by taking client treatment preferences into account in addition to scientific evidence and clinical experience. A number of interesting developments appear in recent research literature; however, child welfare and child protection must find additional inspiration from other areas, e.g., mental health services, because research on decision-making processes in child welfare and child protection is still rare.

Reply to the Letter to the Editor of Van der Put, Assink, & Stams about "Deciding on child maltreatment: A literature review on methods that improve decision-making"

We thank Van der Put, Assink and Stams for their interest in our literature review on methods that improve decisionmaking in child maltreatment cases. They stir up a discussion on actuarial versus consensus-based instruments that has been going on for years and has previously been called “the war on risk assessment” (Johnson, 2006; White & Walsh, 2006). It is important to carefully assess the evidence on the value of risk assessment instruments, and of instruments versus clinical judgment. The purpose is to assist workers in using the best methods available for making judgments and decisions on vulnerable children and families. Van der Put et al. express concerns about the comprehensiveness of our literature review. Specifically, they miss the articles of Baird and Wagner (2000) and D’Andrade, Benton, and Austin (2005). We did not miss these articles. However, we did not describe the studies they mention separately, because Barlow, Fisher, and Jones (2012) included these in their review. Inclusion of all three articles would have led to an extensive overlap and might make the reader think that there is a lot of evidence on risk assessment instruments, while actually the evidence is scarce and the articles presented information on the same studies. Barlow et al. (2012) included 17 studies that D’Andrade et al. (2005) also included. Five studies in the review of D’Andrade et al. (2005) were not included by Barlow et al. (2012). These studies investigated the reliability and validity of consensus-based risk assessment instruments. They add to the evidence that consensus-based instruments have low reliability and validity. Although this finding would not have changed our conclusions, we acknowledge that – for the sake of clarity – it would have been best if we had mentioned this non-overlapping part of D’Andrade et al.’s review. The second concern of Van der Put et al. is the exclusion of studies reporting on individual instruments. We agree with Van der Put et al. that it is quite usual to study the reliability and validity of individual risk assessment instruments without comparison to a control group (i.e. another instrument or no instrument). It might seem too restrictive to exclude studies on the performance of individual instruments when this is the usual research approach. However, based on studies on individual instruments, it is not possible to conclude whether these instruments will improve decisions made in child maltreatment cases, which was the purpose of our review. Therefore we excluded studies on individual instruments that made no comparison with other instruments or conditions, as is usual in meta-analyses and systematic reviews on the effectiveness of interventions. Further, Van der Put et al. state that the study of Barlow et al. (2012) should not be included because this study reported on individual instruments. However, we excluded articles reporting on a single instrument without a control condition. For the purpose of our study we were interested in including reviews that compared and discussed different (types) of instruments, or the use of an instrument versus no instrument. The study of Barlow et al. (2012) offered such a comparison. A drawback of reviews like that of Barlow et al.’s is that it cannot be assured that the comparison made between instruments pertains to comparable cases, because they do not describe or analyze statistically the research results in relation to the purpose and target group of the included instruments. Barlow et al. (2012) describe and compare the instruments systematically; they did not make clear whether the cases in the included studies are comparable. Therefore, in the future these reviews should also comprise a careful comparison of the cases that have been subject in each of the included studies. Finally, Van der Put et al. find it remarkable that study findings obtained in other disciplines were not included (for example Aegisdóttir et al., 2006; Dawes, Faust, & Meehl, 1989; Grove & Meehl, 1996; Grove, Zald, Lebow, Snitz, & Nelson, 2000; Hanson & Morton-Bourgon, 2009; Hilton, Harris, & Rice, 2006; Leschied, Chiodo, Whitehead, Hurley, & Marshall, 2003; Meehl, 1954, 1986). Evidence from other disciplines makes clear that actuarial prediction methods are about 10