Diane Ebert-May

What We Say Is Not What We Do: Effective Evaluation of Faculty Professional Development Programs

Professional development (PD) workshops designed to help faculty move from teacher- to learner-centered science courses for undergraduates are typically evaluated with self-reported surveys that address facultys satisfaction with a workshop, what they learned, and what they applied in the classroom. Professional development outcomes are seldom evaluated through analysis of observed teaching practices. We analyzed videotapes of biology faculty teaching following PD to address three questions: (1) How learner centered was their teaching? (2) Did self-reported data about faculty teaching differ from the data from independent observers? (3) What variables predict teaching practices by faculty? Following PD, 89% of the respondents stated that they made changes in their courses that included active, learner-centered instruction. In contrast, observational data showed that participation in PD did not result in learner-centered teaching. The majority of faculty (75%) used lecture-based, teacher-centered pedagogy, showing a clear disconnect between facultys perceptions of their teaching and their actual practices.

Just the Facts? Introductory Undergraduate Biology Courses Focus on Low-Level Cognitive Skills

Introductory biology courses are widely criticized for overemphasizing details and rote memorization of facts. Data to support such claims, however, are surprisingly scarce. We sought to determine whether this claim was evidence-based. To do so we quantified the cognitive level of learning targeted by faculty in introductory-level biology courses. We used Blooms Taxonomy of Educational Objectives to assign cognitive learning levels to course goals as articulated on syllabi and individual items on high-stakes assessments (i.e., exams and quizzes). Our investigation revealed the following: 1) assessment items overwhelmingly targeted lower cognitive levels, 2) the cognitive level of articulated course goals was not predictive of the cognitive level of assessment items, and 3) there was no influence of course size or institution type on the cognitive levels of assessments. These results support the claim that introductory biology courses emphasize facts more than higher-order thinking.

The Other Half of the Story: Effect Size Analysis in Quantitative Research

Effect size measures are a key complement to statistical significance testing when reporting quantitative research findings. The authors provide a rationale for use of effect size and specific tools and guidelines for interpretation of results.

Learner-Centered Inquiry in Undergraduate Biology: Positive Relationships with Long-Term Student Achievement

We determined short- and long-term correlates of a revised introductory biology curriculum on understanding of biology as a process of inquiry and learning of content. In the original curriculum students completed two traditional lecture-based introductory courses. In the revised curriculum students completed two new learner-centered, inquiry-based courses. The new courses differed significantly from those of the original curriculum through emphases on critical thinking, collaborative work, and/or inquiry-based activities. Assessments were administered to compare student understanding of the process of biological science and content knowledge in the two curricula. More seniors who completed the revised curriculum had high-level profiles on the Views About Science Survey for Biology compared with seniors who completed the original curriculum. Also as seniors, students who completed the revised curriculum scored higher on the standardized Biology Field Test. Our results showed that an intense inquiry-based learner-centered learning experience early in the biology curriculum was associated with long-term improvements in learning. We propose that students learned to learn science in the new courses which, in turn, influenced their learning in subsequent courses. Studies that determine causal effects of learner-centered inquiry-based approaches, rather than correlative relationships, are needed to test our proposed explanation.

Functional mismatch in a bumble bee pollination mutualism under climate change.

Climate change decoupling mutualism Many coevolved species have precisely matched traits. For example, long-tongued bumblebees are well adapted for obtaining nectar from flowers with long petal tubes. Working at high altitude in Colorado, Miller-Struttmann et al. found that long-tongued bumblebees have decreased in number significantly over the past 40 years. Short-tongued species, which are able to feed on many types of flowers, are replacing them. This shift seems to be a direct result of warming summers reducing flower availability, making generalist bumblebees more successful than specialists and resulting in the disruption of long-held mutualisms. Science, this issue p. 1541 Warming summers shorten bees’ tongues and disrupt well-established mutualisms between bees and plants. Ecological partnerships, or mutualisms, are globally widespread, sustaining agriculture and biodiversity. Mutualisms evolve through the matching of functional traits between partners, such as tongue length of pollinators and flower tube depth of plants. Long-tongued pollinators specialize on flowers with deep corolla tubes, whereas shorter-tongued pollinators generalize across tube lengths. Losses of functional guilds because of shifts in global climate may disrupt mutualisms and threaten partner species. We found that in two alpine bumble bee species, decreases in tongue length have evolved over 40 years. Co-occurring flowers have not become shallower, nor are small-flowered plants more prolific. We argue that declining floral resources because of warmer summers have favored generalist foraging, leading to a mismatch between shorter-tongued bees and the longer-tubed plants they once pollinated.

Challenge faculty to transform STEM learning

Focus on core ideas, crosscutting concepts, and scientific practices Models for higher education in science, technology, engineering, and mathematics (STEM) are under pressure around the world. Although most STEM faculty and practicing scientists have learned successfully in a traditional format, they are the exception, not the norm, in their success. Education should support a diverse population of students in a world where using knowledge, not merely memorizing it, is becoming ever more important. In the United States, which by many measures is a world leader in higher education, the Presidents Council of Advisors on Science and Technology (PCAST) recommended sweeping changes to the first 2 years of college, which are critical for recruitment and retention of STEM students (1). Although reform efforts call for evidence-based pedagogical approaches, supportive learning environments, and changes to faculty teaching culture and reward systems, one important aspect needs more attention: changing expectations about what students should learn, particularly in college-level introductory STEM courses. This demands that faculty seriously discuss, within and across disciplines, how they approach their curricula.

Breaking the Cycle: Future Faculty Begin Teaching with Learner-Centered Strategies after Professional Development

The authors investigated the extent to which postdoctoral fellows believed in and implemented evidence-based pedagogies after completion of a 2-yr professional development program, FIRST IV. Postdocs reported greater use of learner-centered compared with teacher-centered instruction, and video ratings further documented learner-centered instruction.

Characterizing college science assessments: The three-dimensional learning assessment protocol

Many calls to improve science education in college and university settings have focused on improving instructor pedagogy. Meanwhile, science education at the K-12 level is undergoing significant changes as a result of the emphasis on scientific and engineering practices, crosscutting concepts, and disciplinary core ideas. This framework of “three-dimensional learning” is based on the literature about how people learn science and how we can help students put their knowledge to use. Recently, similar changes are underway in higher education by incorporating three-dimensional learning into college science courses. As these transformations move forward, it will become important to assess three-dimensional learning both to align assessments with the learning environment, and to assess the extent of the transformations. In this paper we introduce the Three-Dimensional Learning Assessment Protocol (3D-LAP), which is designed to characterize and support the development of assessment tasks in biology, chemistry, and physics that align with transformation efforts. We describe the development process used by our interdisciplinary team, discuss the validity and reliability of the protocol, and provide evidence that the protocol can distinguish between assessments that have the potential to elicit evidence of three-dimensional learning and those that do not.

Using Avida-ED for Teaching and Learning About Evolution in Undergraduate Introductory Biology Courses

Evolution is a complex subject that requires knowledge of basic biological concepts and the ability to connect them across multiple scales of time, space, and biological organization. Avida-ED is a digital evolution educational software environment designed for teaching and learning about evolution and the nature of science in undergraduate biology courses. This study describes our backward design approach to developing an instructional activity using Avida-ED for teaching and learning about evolution in a large-enrollment introductory biology course. Using multiple assessment instruments, we measured student knowledge and understanding of key principles of natural selection before and after instruction on evolution (including the Avida-ED activity). Assessment analysis revealed significant post-instruction learning gains, although certain evolutionary principles (most notably those including genetics concepts, such as the genetic origin of variation) remained particularly difficult for students, even after instruction. Students, however, demonstrated a good grasp of the genetic component of the evolutionary process in the context of a problem on Avida-ED. We propose that: (a) deep understanding of evolution requires complex systems thinking skills, such as connecting concepts across multiple levels of biological organization, and (b) well designed use of Avida-ED holds the potential to help learners build a meaningful and transferable understanding of the evolutionary process.

Teaching Assistant Professional Development in Biology: Designed for and Driven by Multidimensional Data

Graduate teaching assistants (TAs) may receive professional development (PD) to enhance their teaching of undergraduates. However, data about the effectiveness of these PD programs are almost entirely self-reported. Using an evaluation framework, we found that TA multidimensional data were more informative for evaluating the efficacy of TA PD.