John M. Kelley

Evaluating the Effectiveness of a School Drug and Alcohol Prevention Curriculum: A New Look at "Here's Looking at You, Two.".

The article reports on the results of an evaluation of a school drug and alcohol prevention curriculum marketed under the title, “Heres Looking at You, Two.” Previous evaluations, unreported in the literature and having unresolved methodological problems, have found that while the program appears effective at transmitting information regarding drug and alcohol abuse, it has not been effective at changing the underlying attitudes and behaviors that, in part, explain substance abuse. Employing a more rigorous methodology, our examination of a relatively large sample across five school districts provides support for previous findings. Over the short-term of one year, the program was particularly effective at transmitting substance information to primary and middle school students. The program, however, produced very little of the expected effect on the underlying attitudes that are critical to changing substance abuse behaviors.

Guest editors' introduction: Teaching evaluation by doing it: A multiple-utility approach

Abstract This paper maps a two-semester graduate course in planning, statistics, and research. The course utilizes program evaluation as the unifying theme to present statistics and planning. The course requires, as its core assignment, student teams to perform extramural evaluative research projects.

Designing a Short-Term Process Evaluation of a Wellness Center Serving Persons With HIV/AIDS1

In the summer of 1996, the Berks AIDS Network (BAN), a social services case management, support, and referral agency, was awarded a 6-month demonstration grant from the Pennsylvania Department of Health, which was distributed through AIDSNET, the regional nonprofit coordinating body. The purpose of the project was to develop a Wellness Center to enhance primary health care services for persons with HIV/AIDS living in Berks County, Pennsylvania. Specifically, BAN was adding a nurse practitioner to the array of services they already offered, with the goal of enhancing HIV-infected individuals’ care. Services offered through the Wellness Center included:

The Use of Client Surrogates in Determining the Reliability of a Standardized Assessment Instrument in Multiple, Diverse Settings.

Abstract Since its introduction in 1980, the widely-used Addiction Severity Index has been subjected to extensive validity and reliability studies. Reliability procedures have commonly entailed test–retest and multiple judge video reviews, all within controlled environments. The following study uses a method that differs substantially from conventional reliability assessment by employing trained, professional actors as client surrogates to examine the reliability of the instrument when administered in the field, in naturalistic, diverse agency settings. This article details this method, addressing the question, to what extent did this method produce consistent stimuli to enable the measurement of reliability?

On Consumer Self-Direction of Attendant Care Services: An Empirical Analysis of Survey Responses.

Abstract In the context of an attendant care program for severely disabled individuals (“consumers”), independent living is commonly thought to consist of increased involvement in community affairs and consumer self-direction of attendant care services. To assess the extent of and preference for self-direction in the Attendant Care Demonstration in the Commonwealth of Pennsylvania during its third program year, this study presents and analyzes survey responses by consumers. It is found that a variety of administrative designs for service delivery exist, ranging from the “agency model” to the “consumer model,” and a variety of administrative designs are preferred. It is also found that consumers tend to prefer the type of administrative design that they have experienced, whether consumer- or agency-oriented. One policy implication involves the establishment of a program design which allows for a continuum of choices and allows consumers to take control of more aspects of service delivery as they gain the experience and desire to do so.

Needs assessment and strategic planning: A study of the girl scouts

Abstract Using a needs assessment conducted for the Girl Scouts of Greater Philadelphia as a test case, this article analyzes the conditions facilitating the collection of information needed for strategic planning in a voluntary organization and the related conditions influencing whether or not the information would in fact be used. The aim is to assess how far the insights gained from experience in assembling information bases for strategic planning in the corporate world can be applied to similar processes in the voluntary sector. A description of the case study and selected findings are included.

Longer term impacts of pre-employment services on the earnings of disadvantaged youth☆

Abstract This article presents the results of long-term follow-up evaluations of two youth pre-employment programs—70,001 Ltd. and Jobs for Youth. Participant outcomes for these programs were analyzed using data from two follow-up waves. Participants in 70,001, still showing gains over comparisons in the second follow-up wave, were surveyed in a third wave—covering a period that ranged from 24 to 40 months from intake into the program. The results indicated that participants demonstrated initial earnings gains from these pre-employment programs, but that over time these initial positive effects decayed as comparisons caught up with participants.

Inspired to Stretch, to Bend a Little, to Grow: The Intersection of Theory and Practice at the Eleanor Chelimsky Forum.

The authors discuss both the genesis of the Eleanor Chelimsky Forum on Evaluation Theory and Practice and the 2015 Forum, which featured remarks by Abraham Wandersman.