Nancy K. Franz

Non-formal educator use of evaluation results

Increasing demands for accountability in educational programming have resulted in increasing calls for program evaluation in educational organizations. Many organizations include conducting program evaluations as part of the job responsibilities of program staff. Cooperative Extension is a complex organization offering non-formal educational programs through land grant universities. Many Extension services require non-formal educational program evaluations be conducted by field-based Extension educators. Evaluation research has focused primarily on the efforts of professional, external evaluators. The work of program staff with many responsibilities including program evaluation has received little attention. This study examined how field based Extension educators (i.e. program staff) in four Extension services use the results of evaluations of programs that they have conducted themselves. Four types of evaluation use are measured and explored; instrumental use, conceptual use, persuasive use and process use. Results indicate that there are few programmatic changes as a result of evaluation findings among the non-formal educators surveyed in this study. Extension educators tend to use evaluation results to persuade others about the value of their programs and learn from the evaluation process. Evaluation use is driven by accountability measures with very little program improvement use as measured in this study. Practical implications include delineating accountability and program improvement tasks within complex organizations in order to align evaluation efforts and to improve the results of both. There is some evidence that evaluation capacity building efforts may be increasing instrumental use by educators evaluating their own programs.

Cost-Effectiveness Model for Youth EFNEP Programs: What Do We Measure and How Do We Do It?

The Youth Expanded Food and Nutrition Education Program (EFNEP) is one of the United States Department of Agricultures hallmark nutrition education programs for limited-resource youth. The objective of this study was to gather opinions from experts in EFNEP and related content areas to identify costs, effects (impacts), and related instruments to develop a cost-effectiveness model (instrument) for youth EFNEP, which does not exist. A cost-effectiveness model determines the economic or financial cost of producing an impact. The findings highlight several challenges in identifying inputs through consensus and provide a roadmap for the creation of a model that can be adopted by state EFNEP coordinators.

Strengthening Families: Exploring the Impacts of Family Camp Experiences on Family Functioning and Parenting

Research suggests that family camp experiences can enhance family relationships. Families often participate in family camp experiences for a vacation, as part of a therapeutic and/or intervention strategy, or to gain general enrichment or engagement. To better understand the impacts of family camp experiences on family functioning, a mixed-methods study was conducted with 60 families across 18 camps. Respondents shared that family camp experiences benefit families because of the positive impacts of the camp staff, parenting reinforcement, and enhancement of family relationships, with 60% of respondents indicating that family camp experiences reinforced good parenting and 86% of respondents indicating that the family camp experience reinforced family relationships. Recommendations for future research and practice are provided.

The Perceived Value of an Extension Leadership Network: Enhancing Personal and Organizational Effectiveness

Abstract This study was designed to determine the value of the Southern Region Cooperative Extension Program Leadership Network in the United States. Of particular interest was determining the value added by the network for individual extension administrators, their state organizations, and the southern region. The researchers conducted a series of focus groups and an electronic survey of the members of the Southern Region Cooperative Extension Program Leadership Network to determine the value of the network. The Program Leadership Network was found to provide great value for the vast majority of study participants at the individual, team, and organizational levels. This value included enhancing individual performance, encouraging critical thinking, catalyzing best practices, providing opportunities for leadership and skill development, enhancing the relationship between extension programmes, and serving as a think tank for regional and national initiatives. Extension and programme administrators in other regions and countries should examine leadership networks to determine if they are adding value to extension work. Development of a network similar to the Southern Region Cooperative Extension Program Leadership Network could improve individual, team, and organizational effectiveness. This is especially true for enhancing multistate, regional, and national programming. Extension systems and workers will find value in this study since the results indicate that networking of extension workers enhances personal, team, and organizational performance. A secondary value includes the increased number of multistate, regional, and national coordination of extension programming efforts.

The carry-over effects of school gardens on fruit and vegetable availability at home: A randomized controlled trial with low-income elementary schools

This group-randomized controlled trial examines the effects of a school garden intervention on availability of fruits and vegetables (FV) in elementary school childrens homes. Within each region, low income U.S. schools in Arkansas, Iowa, New York, and Washington State were randomly assigned to intervention group (n = 24) or waitlist control group (n = 22). Children were in grades 2, 4, and 5 at baseline (n = 2768). The garden intervention consisted of both raised-bed garden kits and a series of grade-appropriate lessons. FV availability at home was measured with a modified version of the GEMS FJV Availability Questionnaire. The instrument was administered at baseline (Fall 2011) and throughout the intervention (Spring 2012, Fall 2012, Spring 2013). Analyses were completed using general linear mixed models. The garden intervention led to an overall increase in availability of low-fat vegetables at home. Among younger children (2nd grade at baseline), the garden intervention led to greater home availability of vegetables, especially, low-fat vegetables. Moreover, for the younger group, garden intervention fidelity (GIF) or robustness predicted home availability of fruit, vegetables, and low-fat vegetables. School gardens have potential to affect FV availability in the home environment.

Book Review: Presenting Data Effectively: Communicating Your Findings for Maximum Impact

As program evaluators we aspire to conduct sterling evaluations. However, the value of these evaluations quickly declines if we fail to effectively communicate findings in meaningful ways to stakeholders. In my work presenting data to a wide variety of audiences from elected officials to university leaders, coworkers, academics, and partner organizations, I have put effort into being concise, accurate, and clear in my communications. I have been less intentional about designing the graphics and other elements of data presentation in ways that align with how our brains operate and how we best retain information. So I was very curious about what “Presenting data effectively: Communicating your findings for maximum impact,” by Stephanie Evergreen, has to offer to my work as a Cooperative Extension administrator, educator, program evaluator, and professor. Disciplines Educational Administration and Supervision | Educational Leadership | Instructional Media Design | Other Education Comments This manuscript is from the American Journal of Evaluation; December 35(2014):594-596. doi:10.1177/ 1098214014530800. Posted with permission. This book review is available at Iowa State University Digital Repository: http://lib.dr.iastate.edu/extension_research/15 or Peer Rview Presenting Data Effectively: Communicating Your Findings for Maximum Impact Journal: American Journal of Evaluation Manuscript ID: AJE-14-02-0002 Manuscript Type: Book Reviews http://mc.manuscriptcentral.com/ajeval American Journal of Evaluation This book review is from American Journal of Evaluation, 35(December 2014):594-596. doi: 10.1177/1098214014530800 Posted with permission. or Peer Rview Nancy Franz Presenting Data Effectively: Communicating Your Findings for Maximum Impact by Stephanie D. H. Evergreen Thousand Oaks, CA: Sage, 2014. 200 pp. Paperback

Book Review: Presenting Data Effectively: Communicating Your Findings for Maximum ImpactEvergreenS. D. H. (2014). Presenting Data Effectively: Communicating Your Findings for Maximum Impact. Thousand Oaks, CA: Sage. 200 pp. Paperback

40.00. ISBN: 9781452257365 Reviewed by: Nancy Franz, Iowa State University, Ames, IA, USA As program evaluators we aspire to conduct sterling evaluations. However, the value of these evaluations quickly declines if we fail to effectively communicate findings in meaningful ways to stakeholders. In my work presenting data to a wide variety of audiences from elected officials to university leaders, coworkers, academics, and partner organizations, I have put effort into being concise, accurate, and clear in my communications. I have been less intentional about designing the graphics and other elements of data presentation in ways that align with how our brains operate and how we best retain information. So I was very curious about what “Presenting data effectively: Communicating your findings for maximum impact,” by Stephanie Evergreen, has to offer to my work as a Cooperative Extension administrator, educator, program evaluator, and professor. I started my relationship with the book by briefly flipping through the pages. The contents include a liberal dosage of figures, inset boxes (which I later learned are “callout boxes”), easy-to-read text, and navigational icons. This format compelled me to jump right in as an eager learner. However, as a good academic, I wanted some proof that Dr. Evergreen knew what she was talking about. It didn’t take long to realize that her body of work is deeply based in communication science, piloted with experts and applied in the field. She leads Evergreen Data and Evaluation, LLC, a data presentation consulting firm and serves as a speaker, designer, and evaluator. Her Ph.D. in interdisciplinary evaluation focused on graphic design use in research reports, which spurred this book and her other work. She spent five years with the Evaluation Center at Western Michigan University and founded the American Evaluation Association Data Visualization and Reporting Topical Interest Group. She recently coedited two issues of New Directions for Evaluation on data visualization. The book covers four major topics to enhance data presentations: (1) compelling images, (2) intelligent colors, (3) intentional typefaces, and (4) appropriate arrangement of data elements. I found this information to be presented in a very accessible way. Evergreen provides figures showing weak and strong examples of the concepts she is presenting. She also delivers a narrative full of humor, personal stories, and convincing rationale for adopting new practices. For the most part, her suggestions for improving data visualization can be accomplished using standard Microsoft Office software and publicly available websites. However, I must admit that I’m not as much of a data nerd as she claims to be so I glazed over and started skimming text and figures when the details got deep. To retain the information for my experiential learning style, I will also need to join her for a hands-on workshop or follow the steps in the book while working at my computer. Page 1 of 4 http://mc.manuscriptcentral.com/ajeval American Journal of Evaluation 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 or Peer Rview I found the first chapter, “The justification for presenting data effectively,” to be the most useful. It provides a compelling case for why I should after all these years as a presenter and report writer, change what I do. Who can argue with enhancing data presentation to improve my professional and personal credibility and competence, save money, respect the time and intelligence of my learners/coworkers/ decision-makers, and make a difference in the world? She says, “The intention is to help you use the tools you already have to make more compelling data presentations so that you can better convince your stakeholders of the worth of your work, secure more attention and funding, and make the world a better place” (p. 16). The first chapter also provides an overview of research behind the practice changes that she is suggesting, drawn from the fields of vision science, cognitive psychology, communications, and graphic design. I appreciated the brief and direct applications of this body of science tied directly to data visualization practice. Evergreen specifically says, “Data presentation sits at the intersection of closely related fields like usability testing, user-interface design, graphic design, journalism, and document design” (p. 14). The core chapters on graphics, type, color, and arrangement provide depth on each topic, with multiple examples especially suited for academics and program evaluators. She provides the science and technical best practices on these topics but also shows the need to get at the emotion behind these concepts. Her goal is to tell an important and meaningful story and to be strategic through simplification and emphasis of elements. She ties this goal to the need for our audiences to have effective information uptake, gain early attention, and get that information into their long-term memory. Each chapter provides an overview of the learning objectives, the main content, key points to remember, activities to extend the learning, and multiple tools, tips, and references to gain more information. She basically tells us what we will learn, shares it, and then summarizes it. The main tips for each chapter and section are highlighted in callout boxes and also found in a useful summary appendix entitled, “Research and evaluation report layout checklist.” I kept a list of best practices she offers throughout the book and came up with twice as many as found in the checklist, so don’t limit yourself to just using the checklist. Some of my favorite gems not on the checklist include: (1) build the cost of stock photos for presentations into the project budget; (2) it takes only three seconds to capture attention so use images; (3) create a visual theme or brand throughout a presentation or report that you want to be remembered for; and (4) “Bar charts work best when comparing categories. Line charts are most appropriate for looking at change over time. Scatterplots most effectively represent correlations and linear relationships” (p. 50). Each chapter provides important detail on the topic presented. For example, the chapter on graphics covers how to create more effective posters and presentations. Evergreen’s research has discovered that imagery is the weakest part of evaluation and research reporting. Based on this research and her experience, she suggests creating a visual theme throughout the presentation that draws your readers’ attention to you and your information and then engages them throughout to increase information uptake. She advises using high quality graphics sized to match changes in meaning of the data. She also suggests principles for the proper placement of images with surrounding text, proper use of background images, and the use of icons for visual short cuts. Evergreen points out that images can enhance the Page 2 of 4 http://mc.manuscriptcentral.com/ajeval American Journal of Evaluation 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 or Peer Rview emotional aspect of the presentation. Finally, she advises that the repetition of effective images creates a brand for both the project and the evaluator or researcher. A companion student study website provides downloadable tools and color images from the book as well as a link to Evergreen’s blog that I found very helpful. I’d suggest that the title “Student study site” is a misnomer since the resources are not just for credit course students. The resources are simply an extension of the book that is important for all readers. Surprisingly, I sometimes found my academic experience thrown to the wind. I had to rely on my early years as a yearbook editor and college newspaper staff member to find my comfort zone. What do you mean I shouldn’t create a document title all in caps, or I shouldn’t

40.00, ISBN 9781452257365.

As program evaluators we aspire to conduct sterling evaluations. However, the value of these evaluations quickly declines if we fail to effectively communicate findings in meaningful ways to stakeholders. In my work presenting data to a wide variety of audiences from elected officials to university leaders, coworkers, academics, and partner organizations, I have put effort into being concise, accurate, and clear in my communications. I have been less intentional about designing the graphics and other elements of data presentation in ways that align with how our brains operate and how we best retain information. So I was very curious about what “Presenting data effectively: Communicating your findings for maximum impact,” by Stephanie Evergreen, has to offer to my work as a Cooperative Extension administrator, educator, program evaluator, and professor. Disciplines Educational Administration and Supervision | Educational Leadership | Instructional Media Design | Other Education Comments This manuscript is from the American Journal of Evaluation; December 35(2014):594-596. doi:10.1177/ 1098214014530800. Posted with permission. This book review is available at Iowa State University Digital Repository: http://lib.dr.iastate.edu/extension_research/15 or Peer Rview Presenting Data Effectively: Communicating Your Findings for Maximum Impact Journal: American Journal of Evaluation Manuscript ID: AJE-14-02-0002 Manuscript Type: Book Reviews http://mc.manuscriptcentral.com/ajeval American Journal of Evaluation This book review is from American Journal of Evaluation, 35(December 2014):594-596. doi: 10.1177/1098214014530800 Posted with permission. or Peer Rview Nancy Franz Presenting Data Effectively: Communicating Your Findings for Maximum Impact by Stephanie D. H. Evergreen Thousand Oaks, CA: Sage, 2014. 200 pp. Paperback

Presenting Data Effectively: Communicating Your Findings for Maximum Impact

40.00. ISBN: 9781452257365 Reviewed by: Nancy Franz, Iowa State University, Ames, IA, USA As program evaluators we aspire to conduct sterling evaluations. However, the value of these evaluations quickly declines if we fail to effectively communicate findings in meaningful ways to stakeholders. In my work presenting data to a wide variety of audiences from elected officials to university leaders, coworkers, academics, and partner organizations, I have put effort into being concise, accurate, and clear in my communications. I have been less intentional about designing the graphics and other elements of data presentation in ways that align with how our brains operate and how we best retain information. So I was very curious about what “Presenting data effectively: Communicating your findings for maximum impact,” by Stephanie Evergreen, has to offer to my work as a Cooperative Extension administrator, educator, program evaluator, and professor. I started my relationship with the book by briefly flipping through the pages. The contents include a liberal dosage of figures, inset boxes (which I later learned are “callout boxes”), easy-to-read text, and navigational icons. This format compelled me to jump right in as an eager learner. However, as a good academic, I wanted some proof that Dr. Evergreen knew what she was talking about. It didn’t take long to realize that her body of work is deeply based in communication science, piloted with experts and applied in the field. She leads Evergreen Data and Evaluation, LLC, a data presentation consulting firm and serves as a speaker, designer, and evaluator. Her Ph.D. in interdisciplinary evaluation focused on graphic design use in research reports, which spurred this book and her other work. She spent five years with the Evaluation Center at Western Michigan University and founded the American Evaluation Association Data Visualization and Reporting Topical Interest Group. She recently coedited two issues of New Directions for Evaluation on data visualization. The book covers four major topics to enhance data presentations: (1) compelling images, (2) intelligent colors, (3) intentional typefaces, and (4) appropriate arrangement of data elements. I found this information to be presented in a very accessible way. Evergreen provides figures showing weak and strong examples of the concepts she is presenting. She also delivers a narrative full of humor, personal stories, and convincing rationale for adopting new practices. For the most part, her suggestions for improving data visualization can be accomplished using standard Microsoft Office software and publicly available websites. However, I must admit that I’m not as much of a data nerd as she claims to be so I glazed over and started skimming text and figures when the details got deep. To retain the information for my experiential learning style, I will also need to join her for a hands-on workshop or follow the steps in the book while working at my computer. Page 1 of 4 http://mc.manuscriptcentral.com/ajeval American Journal of Evaluation 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 or Peer Rview I found the first chapter, “The justification for presenting data effectively,” to be the most useful. It provides a compelling case for why I should after all these years as a presenter and report writer, change what I do. Who can argue with enhancing data presentation to improve my professional and personal credibility and competence, save money, respect the time and intelligence of my learners/coworkers/ decision-makers, and make a difference in the world? She says, “The intention is to help you use the tools you already have to make more compelling data presentations so that you can better convince your stakeholders of the worth of your work, secure more attention and funding, and make the world a better place” (p. 16). The first chapter also provides an overview of research behind the practice changes that she is suggesting, drawn from the fields of vision science, cognitive psychology, communications, and graphic design. I appreciated the brief and direct applications of this body of science tied directly to data visualization practice. Evergreen specifically says, “Data presentation sits at the intersection of closely related fields like usability testing, user-interface design, graphic design, journalism, and document design” (p. 14). The core chapters on graphics, type, color, and arrangement provide depth on each topic, with multiple examples especially suited for academics and program evaluators. She provides the science and technical best practices on these topics but also shows the need to get at the emotion behind these concepts. Her goal is to tell an important and meaningful story and to be strategic through simplification and emphasis of elements. She ties this goal to the need for our audiences to have effective information uptake, gain early attention, and get that information into their long-term memory. Each chapter provides an overview of the learning objectives, the main content, key points to remember, activities to extend the learning, and multiple tools, tips, and references to gain more information. She basically tells us what we will learn, shares it, and then summarizes it. The main tips for each chapter and section are highlighted in callout boxes and also found in a useful summary appendix entitled, “Research and evaluation report layout checklist.” I kept a list of best practices she offers throughout the book and came up with twice as many as found in the checklist, so don’t limit yourself to just using the checklist. Some of my favorite gems not on the checklist include: (1) build the cost of stock photos for presentations into the project budget; (2) it takes only three seconds to capture attention so use images; (3) create a visual theme or brand throughout a presentation or report that you want to be remembered for; and (4) “Bar charts work best when comparing categories. Line charts are most appropriate for looking at change over time. Scatterplots most effectively represent correlations and linear relationships” (p. 50). Each chapter provides important detail on the topic presented. For example, the chapter on graphics covers how to create more effective posters and presentations. Evergreen’s research has discovered that imagery is the weakest part of evaluation and research reporting. Based on this research and her experience, she suggests creating a visual theme throughout the presentation that draws your readers’ attention to you and your information and then engages them throughout to increase information uptake. She advises using high quality graphics sized to match changes in meaning of the data. She also suggests principles for the proper placement of images with surrounding text, proper use of background images, and the use of icons for visual short cuts. Evergreen points out that images can enhance the Page 2 of 4 http://mc.manuscriptcentral.com/ajeval American Journal of Evaluation 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 or Peer Rview emotional aspect of the presentation. Finally, she advises that the repetition of effective images creates a brand for both the project and the evaluator or researcher. A companion student study website provides downloadable tools and color images from the book as well as a link to Evergreen’s blog that I found very helpful. I’d suggest that the title “Student study site” is a misnomer since the resources are not just for credit course students. The resources are simply an extension of the book that is important for all readers. Surprisingly, I sometimes found my academic experience thrown to the wind. I had to rely on my early years as a yearbook editor and college newspaper staff member to find my comfort zone. What do you mean I shouldn’t create a document title all in caps, or I shouldn’t

The nature of complex organizations: The case of Cooperative Extension

As program evaluators we aspire to conduct sterling evaluations. However, the value of these evaluations quickly declines if we fail to effectively communicate findings in meaningful ways to stakeholders. In my work presenting data to a wide variety of audiences from elected officials to university leaders, coworkers, academics, and partner organizations, I have put effort into being concise, accurate, and clear in my communications. I have been less intentional about designing the graphics and other elements of data presentation in ways that align with how our brains operate and how we best retain information. So I was very curious about what “Presenting data effectively: Communicating your findings for maximum impact,” by Stephanie Evergreen, has to offer to my work as a Cooperative Extension administrator, educator, program evaluator, and professor. Disciplines Educational Administration and Supervision | Educational Leadership | Instructional Media Design | Other Education Comments This manuscript is from the American Journal of Evaluation; December 35(2014):594-596. doi:10.1177/ 1098214014530800. Posted with permission. This book review is available at Iowa State University Digital Repository: http://lib.dr.iastate.edu/extension_research/15 or Peer Rview Presenting Data Effectively: Communicating Your Findings for Maximum Impact Journal: American Journal of Evaluation Manuscript ID: AJE-14-02-0002 Manuscript Type: Book Reviews http://mc.manuscriptcentral.com/ajeval American Journal of Evaluation This book review is from American Journal of Evaluation, 35(December 2014):594-596. doi: 10.1177/1098214014530800 Posted with permission. or Peer Rview Nancy Franz Presenting Data Effectively: Communicating Your Findings for Maximum Impact by Stephanie D. H. Evergreen Thousand Oaks, CA: Sage, 2014. 200 pp. Paperback