James Leslie Herbert

Researching Evaluation Influence: A Review of the Literature

Background: The impact of an evaluation is an important consideration in designing and carrying out evaluations. Evaluation influence is a way of thinking about the effect that an evaluation can have in the broadest possible terms, which its proponents argue will lead to a systematic body of evidence about influential evaluation practices. Method: This literature review sets out to address three research questions: How have researchers defined evaluation influence; how is this reflected in the research; and what does the research suggest about the utility of evaluation influence as a conceptual framework. Drawing on studies that had cited one of the key evaluation influence articles and conducted original research on some aspect of influence this article reviewed the current state of the literature toward the goal of developing a body of evidence about how to practice influential evaluation. Results: Twenty-eight studies were found that have drawn on evaluation influence, which were categorized into (a) descriptive studies, (b) analytical studies, and (c) hypothesis testing. Conclusion: Despite the prominence of evaluation influence in the literature, there is slow progress toward a persuasive body of literature. Many of the studies reviewed offered vague and inconsistent definitions and have applied influence in an unspecified way in the research. It is hoped that this article will stimulate interest in the systematic study of influence mechanisms, leading to improvements in the potential for evaluation to affect positive social change.

Evidence for the Efficacy of the Child Advocacy Center Model A Systematic Review

The Child Advocacy Center (CAC) model has been presented as the solution to many of the problems inherent in responses by authorities to child sexual abuse. The lack of referral to therapeutic services and support, procedurally flawed and potentially traumatic investigation practices, and conflict between the different statutory agencies involved are all thought to contribute to low conviction rates for abuse and poor outcomes for children. The CAC model aims to address these problems through a combination of multidisciplinary teams, joint investigations, and services, all provided in a single child friendly environment. Using a systematic search strategy, this research aimed to identify and review all studies that have evaluated the effectiveness of the approach as a whole, recognizing that a separate evidence base exists for parts of the approach (e.g., victim advocacy and therapeutic responses). The review found that while the criminal justice outcomes of the model have been well studied, there was a lack of research on the effect of the model on child and family outcomes. Although some modest outcomes were clear, the lack of empirical research, and overreliance on measuring program outputs, rather than outcomes, suggests that some clarification of the goals of the CAC model is needed.

The impacts of institutional child sexual abuse: A rapid review of the evidence

While awareness of institutional child sexual abuse has grown in recent years, there remains limited understanding of its occurrence and outcomes as a distinct form of abuse. Drawing on research commissioned by the Australian Royal Commission into Institutional Responses to Child Sexual Abuse, this article presents a rapid review of available evidence on the impacts of institutional abuse on victim/survivors. Literature searches identified 75 sources spanning international peer reviewed work and reports to Government that document or quantify the impacts of mostly historical child sexual abuse occurring in religious, educational, sporting and residential or out-of-home care settings. Consistent with child sexual abuse in other contexts, institutional child sexual abuse is found to be associated with numerous, pervasive and connected impacts upon the psychological, physical, social, educative and economic wellbeing of victims/survivors. Further, institutional child sexual abuse is associated with vicarious trauma at the individual, family and community level, and with impacts to the spiritual wellbeing of victims/survivors of abuse that occurs in religious settings. The identified literature suggests the trauma of institutional child sexual abuse may be exacerbated by the interplay of abuse dynamics in institutional settings, which may reduce or impede circumstances supporting disclosure, belief, support and protection from future harm. Acknowledging the limitations of the present study and the available evidence, this narrative synthesis provides insights into the complex impacts of institutional child sexual abuse.

Better Together? A Review of Evidence for Multi-Disciplinary Teams Responding to Physical and Sexual Child Abuse:

Multi-Disciplinary teams (MDTs) have often been presented as the key to dealing with a number of intractable problems associated with responding to allegations of physical and sexual child abuse. While these approaches have proliferated internationally, researchers have complained of the lack of a specific evidence base identifying the processes and structures supporting multi-disciplinary work and how these contribute to high-level outcomes. This systematic search of the literature aims to synthesize the existing state of knowledge on the effectiveness of MDTs. This review found that overall there is reasonable evidence to support the idea that MDTs are effective in improving criminal justice and mental health responses compared to standard agency practices. The next step toward developing a viable evidence base to inform these types of approaches seems to be to more clearly identify the mechanisms associated with effective MDTs in order to better inform how they are planned and implemented.

Towards Program Evaluation for Practitioner Learning: Human Service Practitioners' Perceptions of Evaluation

Abstract While human service practice improvement is acknowledged as an important outcome of program evaluation, much evaluation is narrowly focused on accountability requirements. This type of evaluation often has limited use and relevance to human service practitioners. In exploring forms of evaluation that do have practice relevance, this article presents the experiences and perceptions of human service practitioners subject to evaluations that set out to foster learning. Across the cases participants expressed concerns about data quality and credibility, the relevance of evaluation data to the program context, the invasiveness of the evaluation, management of the process (particularly limited consultation with practitioners), and the lack of effective dissemination. These cases illustrate the challenges for evaluation to be useful to practice, and to foster meaningful improvements to services.

A national survey of characteristics of child advocacy centers in the United States: Do the flagship models match those in broader practice?

Child Advocacy Centers (CAC) emphasize developing effective cross-agency collaborations between workers involved in serious abuse investigations to foster improvements in agency outcomes, and to minimize distress, confusion and uncertainty for children and families. This study examined the characteristics of CACs, whether models in practice match the predominant model presented in the research literature. Directors of CACs in the United States that were members of the National Childrens Alliance (NCA) mailing list (n=361) completed an online survey in 2016. While some core characteristics were ubiquitous across CACs, the data suggests that different types of CACs exist defined by characteristics that are not prescribed under NCA principles, but which are arguably relevant to the quality of the response. From the results of a cluster analysis, the researchers propose a typology of CACs that reflects the development and integration of centers: (a) core CAC services (i.e. interviewing & cross-agency case review); (b) an aggregator of external services, and (c) a more centralized full-service CAC. Further research is needed to understand how these variations may impact practice and outcomes; this is particularly important considering many CACs do not match the full-service models most commonly examined in the research literature, which limits the degree to which these findings apply to CACs generally. This article proposes further research framed by the need to better understand how different parts of the response impact on outcomes for children and families affected by abuse.

Leaving a licensed boarding house: transcending trans-institutionalisation with person-centred transitions

While de-institutionalisation set out with the laudable aim to improve the quality of life of people with disability, the untethering of accommodation from support services has in some cases led to less than desirable outcomes. Poor regulation of for-profit boarding houses licensed to care for people with disability has led to the exploitation and abuse of some of Australia’s most vulnerable and marginalised people. Recognition of this, and a number of other factors, has led to the closure of many of New South Wales’ licensed boarding houses. This research presents the experiences of 15 former residents of licensed boarding houses and their supporters spanning from their time as a resident, the transition to other accommodation and their quality of life post transition. While finding the transition challenging, almost all of the former residents identified improvements in their lives since leaving the boarding house.