Ralph Renger

Development and evaluation of the McKnight Risk Factor Survey for assessing potential risk and protective factors for disordered eating in preadolescent and adolescent girls.

OBJECTIVE To describe the development, test-retest reliability, internal consistency, and convergent validity of the McKnight Risk Factor Survey-III (MRFS-III). The MRFS-III was designed to assess a number of potential risk and protective factors for the development of disordered eating in preadolescent and adolescent girls. METHOD Several versions of the MRFS were pilot tested before the MRFS-III was administered to a sample of 651 4th through 12th- grade girls to establish its psychometric properties. RESULTS Most of the test-retest reliability coefficients of individual items on the MRFS-III were r > .40. Alpha coefficients for each risk and protective factor domain on the MRFS-III were also computed. The majority of these coefficients were r > .60. High convergent validity coefficients were obtained for specific items on the MRFS-III and measures of self-esteem (Rosenberg Self-Esteem Scale) and weight concerns (Weight Concerns Scale). CONCLUSIONS The test-retest reliability, internal consistency, and convergent validity of the MRFS-III suggest that it is a useful new instrument to assess potential risk and protective factors for the development of disordered eating in preadolescent and adolescent girls.

A Three-Step Approach to Teaching Logic Models

Developing a logic model is an essential first step in program evaluation. Our experience has been that there is little guidance to teach students how to develop a logic model that is true to its intended purpose of providing a clear visual representation of the underlying rationale that is not shrouded by including the elements of evaluation. We have developed a three-step approach that begins with developing the visual representation of the underlying rationale, central to which is the identification of Antecedent conditions. Step 2 ensures that program activities Target antecedent conditions, while Step 3 focuses on Measurement issues, depicting indicators and objectives for outcomes being included in the evaluation plan. We have coined this method of teaching the ATM approach. We hope that teachers of evaluation will find the ATM approach useful in the form presented here or at least stimulate thought as to how to adapt the approach to meet individual teaching needs.

Optimal Living Profile : An Inventory to Assess Health and Wellness

Objective: To develop an instrument to assess environmental, emotional, spiritual, physical, social, and intellectual health and wellness. Methods: Reliability was assessed using 102 respondents who completed the OLP on 2 occasions. Concurrent validity was assessed using a second sample of 34 respondents and a panel of 6 experts. Results: Most items had high test-retest correlation coefficients. Each dimension had high internal consistency (ie, high Cronbach Alphas). Independence of items in each dimension suggested good divergent and convergent validity. Conclusions: The OLP is a reliable and valid instrument for assessing dimensions of the Total Person Concept. Am J Health Behav 2000;24(6):403-412

Self-esteem and the prevention of eating disorders

Abstract This article begins by reviewing the literature linking self esteem to eating disorders (EDs) in girls. Prevention strategies used to enhance self-esteem and some of the limitations of these approaches are then discussed. The article concludes with a presentation of a model that addresses these shortcomings and that will help guide the design, implementation, and evaluation of future, prevention programs targeting EDs.

Strategies for Values Inquiry: An Exploratory Case Study

Values inquiry refers to systematic investigation of the values relevant to a program, including understanding those outcomes most prized by stakeholders. Although much attention has been given to stakeholder involvement, the literature on value inquiries per se as an approach is limited. This paper demonstrates how several of the steps used in the ATM approach to developing logic models [Am. J. Eval. 23 (2002) 493] are congruent with the tenets of values inquiry and can serve to advance values inquiry methodology. Using a case study approach, we describe how interview and deliberative approaches were combined to prioritize a large number of potential outcomes for a coalition of agencies working toward the same goal. Successes and challenges in understanding the application of the ATM approach to furthering the development of values inquiry methods are offered.

Consequences to Federal Programs When the Logic-Modeling Process Is Not Followed With Fidelity

The Office of Management and Budget has recommended the termination of numerous federal programs, citing a lack of program results as the primary reason for this decision. In response to this recommendation, several federal agencies have turned to logic modeling to demonstrate that programs are on the path to results accountability. However, approaches to logic modeling in some agencies have not followed the strategies that evaluators recommend as necessary to lead to a high quality logic model. Models of poor quality are unlikely to contribute to improved program accountability or better program results. In this article, the author assesses the quality of the logic-modeling approach taken by one agency to illustrate how a flawed approach to logic modeling may lead to incorrect conclusions about programs and about the benefits of logic models. As a result of this analysis, the author questions the conditions under which capacity building should be considered and whether the field of evaluation needs to be more judicious when mainstreaming methodologies.

Osteoporosis prevention: knowledge and behavior in a southwestern community.

Prior to developing an osteoporosis prevention education program and social marketing campaign, we sought to (a) establish current status of osteoporosis-related knowledge and behavior among women aged 25–55 years in Maricopa County, Arizona, and (b) assess factors that segment the population by age and ethnicity. Two-hundred women were surveyed by telephone using random-digit dialing selection. Data demonstrated knowledge of need to consume adequate calcium, but mixed understanding of forms of exercise that help prevent osteoporosis. Knowledge of osteoporosis prevention did not differ as a function of menopause status. Differences for Hispanic versus non-Hispanic womens knowledge showed fewer correct responses for Hispanics for dietary and physical activity questions, and more correct responses on the relationship between body weight and osteoporosis risk. Hispanic women and post-menopausal women generally fit the pattern of higher risk behaviors with Hispanic women exercising and using HRT less and postmenopausal women reporting lower calcium intake and physical activity and more tobacco use. Hispanic women appeared to have similar intake of dietary calcium despite lower levels of milk products. Social marketing campaigns for osteoporosis prevention should be segmented for cultural and age differences, especially considering differences in orientations toward exercise, milk consumption, and competing emphasis on other diseases.

Geographic Information Systems (GIS) as an Evaluation Tool

Evaluators must seek methods that convey the results of an evaluation so that those who intend on using the information easily understand them. The purpose of this article is to describe how Geographic Information Systems (GIS) can be used to assist evaluators to convey complex information simply, via a spatial representation. Although the utility of GIS in such disciplines as geography, planning, epidemiology and public health is well documented, a review of the literature suggests that its usefulness as a tool for evaluators has gone relatively unnoticed. The paper posits that evaluators may have not recognized the potential of GIS, because of two beliefs that GIS can only provide cross-sectional, snapshots of data, and hence cannot depict change and that many of the available databases that underlie GIS do not contain data relevant to the evaluation at hand. This article demonstrates how GIS can be used to plot change over time, including impact and outcome data gathered by primary data collection.

How Using a Logic Model Refined Our Program to Ensure Success

When a program does not produce expected changes in the outcomes of interest, program staff must understand why before making programmatic decisions. One mechanism for doing so is the logic model. This article describes how using one such logic model (the ATM approach) was used to improve areas of program planning and implementation. The key components include interviewing stakeholders to identify antecedent conditions to the problem of interest that are placed in a summary map and then prioritizing those conditions to address with program activities. Although the logic modeling process was helpful in making programmatic improvements, there were also several unintended benefits of engaging in the process including: providing a framework for planning and evaluation, helping the program operate more effectively and efficiently, and providing a common language among program staff and the evaluators.

Learning from a Challenging Fieldwork Evaluation Experience: Perspectives of a Student and an Instructor.

The use of fieldwork experiences has been widely accepted by teachers of evaluation as a method of preparing future practitioners for challenges encountered in the field. However, several challenges must be addressed to make fieldwork experiences a useful tool in preparing students for their professional careers. The authors present a case study in which a graduate student in an evaluation class approached an agency seeking a fieldwork project. The experience was not positive. Both a student’s and an instructor’s perspectives are offered in understanding the reasons for the negative experience. Suggestions are provided from both perspectives as to how the experience could have been improved. The article is intended as a teaching tool to be provided by instructors to students seeking applied experiences before approaching agencies. This article provides an opportunity for student evaluators to learn from the experiences of another student; likewise, the instructor in this case study provides insights that other instructors of evaluation may find helpful.