Ross H. Nehm

Does Increasing Biology Teacher Knowledge of Evolution and the Nature of Science Lead to Greater Preference for the Teaching of Evolution in Schools

This study investigated whether or not an increase in secondary science teacher knowledge about evolution and the nature of science gained from completing a graduate-level evolution course was associated with greater preference for the teaching of evolution in schools. Forty-four precertified secondary biology teachers participated in a 14-week intervention designed to address documented misconceptions identified by a precourse instrument. The course produced statistically significant gains in teacher knowledge of evolution and the nature of science and a significant decrease in misconceptions about evolution and natural selection. Nevertheless, teachers’ postcourse preference positions remained unchanged; the majority of science teachers still preferred that antievolutionary ideas be taught in school.

A Cross‐Cultural Comparison of Korean and American Science Teachers’ Views of Evolution and the Nature of Science

Despite a few international comparisons of the evolutionary beliefs of the general public, comparatively less research has focused on science teachers. Cross‐cultural studies offer profitable opportunities for exploring the interactions among knowledge and belief variables in regard to evolution in different socio‐cultural contexts. We investigated the evolutionary worldviews of pre‐service science teachers from Asia (specifically South Korea), a region often excluded from international comparisons. We compared Korean and American science teachers’: (1) understandings of evolution and the nature of science, and (2) acceptance of evolution in order to elucidate how knowledge and belief relationships are manifested in different cultural contexts. We found that Korean science teachers exhibited ‘moderate’ evolutionary acceptance levels comparable to or lower than American science teacher samples. Gender was significantly related to Korean teachers’ evolution content knowledge and acceptance of evolution, with female Christian biology teachers displaying the lowest values on all measures. Korean science teachers’ understandings of nature of science were significantly related to their acceptance and understanding of evolution; this relationship appears to transcend cultural boundaries. Our new data on Korean teachers, combined with studies from more than 20 other nations, expose the global nature of science teacher ambivalence or antipathy toward evolutionary knowledge.

Evaluating Instrument Quality in Science Education: Rasch‐based analyses of a Nature of Science test

Given the central importance of the Nature of Science (NOS) and Scientific Inquiry (SI) in national and international science standards and science learning, empirical support for the theoretical delineation of these constructs is of considerable significance. Furthermore, tests of the effects of varying magnitudes of NOS knowledge on domain‐specific science understanding and belief require the application of instruments validated in accordance with AERA, APA, and NCME assessment standards. Our study explores three interrelated aspects of a recently developed NOS instrument: (1) validity and reliability; (2) instrument dimensionality; and (3) item scales, properties, and qualities within the context of Classical Test Theory and Item Response Theory (Rasch modeling). A construct analysis revealed that the instrument did not match published operationalizations of NOS concepts. Rasch analysis of the original instrument—as well as a reduced item set—indicated that a two‐dimensional Rasch model fit significantly better than a one‐dimensional model in both cases. Thus, our study revealed that NOS and SI are supported as two separate dimensions, corroborating theoretical distinctions in the literature. To identify items with unacceptable fit values, item quality analyses were used. A Wright Map revealed that few items sufficiently distinguished high performers in the sample and excessive numbers of items were present at the low end of the performance scale. Overall, our study outlines an approach for how Rasch modeling may be used to evaluate and improve Likert‐type instruments in science education.

Does the Segregation of Evolution in Biology Textbooks and Introductory Courses Reinforce Students’ Faulty Mental Models of Biology and Evolution?

The well-established finding that substantial confusion and misconceptions about evolution and natural selection persist after college instruction suggests that these courses neither foster accurate mental models of evolution’s mechanisms nor instill an appreciation of evolution’s centrality to an understanding of the living world. Our essay explores the roles that introductory biology courses and textbooks may play in reinforcing undergraduates’ pre-existing, faulty mental models of the place of evolution in the biological sciences. Our content analyses of the three best-selling introductory biology textbooks for majors revealed the conceptual segregation of evolutionary information. The vast majority of the evolutionary terms and concepts in each book were isolated in sections about evolution and diversity, while remarkably few were employed in other sections of the books. Standardizing the data by number of pages per unit did not alter this pattern. Students may fail to grasp that evolution is the unifying theme of biology because introductory courses and textbooks reinforce such isolation. Two goals are central to resolving this problem: the desegregation of evolution as separate “units” or chapters and the active integration of evolutionary concepts at all levels and across all domains of introductory biology.

What Do Experts and Novices “See” in Evolutionary Problems?

Considerable research has focused on differences in expert and novice problem representation and performance within physics, chemistry, and genetics. Here, we examine whether models of problem solving based on this work are useful within the domain of evolutionary biology. We utilized card sort tasks, interviews, and paper-and-pencil tests to: (1) delineate problem categorization rules, (2) quantify problem solving success, and (3) measure the relationships between the composition, structure, and coherence of problem solutions. We found that experts and novices perceived different item features to be of significance in card sort tasks, and that sensitivity to item surface features was adversely associated with problem solving success. As in other science domains, evolutionary problem representation and problem solving performance were tightly coupled. Explanatory coherence and the absence of cognitive biases were distinguishing features of evolutionary expertise. We discuss the implications of these findings for biology teaching and learning.

“Force-Talk” in Evolutionary Explanation: Metaphors and Misconceptions

The notion of “pressure” as an evolutionary “force” that “causes” evolution is a pervasive linguistic feature of biology textbooks, journal articles, and student explanatory discourse. We investigated the consequences of using a textbook and curriculum that incorporate so-called force-talk. We examined the frequency with which biology majors spontaneously used notions of evolutionary “pressures” in their explanations, students’ definitions and explanations of what they meant when they used pressures, and the structure of explanatory models that incorporated evolutionary pressures and forces. We found that 12–20 percent of undergraduates spontaneously used “pressures” and/or “forces” as explanatory factors but significantly more often in trait gain scenarios than in trait loss scenarios. The majority of explanations using “force-talk” were characterized by faulty evolutionary reasoning. We discuss the conceptual similarity between faulty notions of evolutionary pressures and linguists’ force-dynamic models of everyday reasoning and ultimately question the appropriateness of force-talk in evolution education.

Applying computerized-scoring models of written biological explanations across courses and colleges: prospects and limitations.

Our study explored the prospects and limitations of using machine-learning software to score introductory biology students’ written explanations of evolutionary change. We investigated three research questions: 1) Do scoring models built using student responses at one university function effectively at another university? 2) How many human-scored student responses are needed to build scoring models suitable for cross-institutional application? 3) What factors limit computer-scoring efficacy, and how can these factors be mitigated? To answer these questions, two biology experts scored a corpus of 2556 short-answer explanations (from biology majors and nonmajors) at two universities for the presence or absence of five key concepts of evolution. Human- and computer-generated scores were compared using kappa agreement statistics. We found that machine-learning software was capable in most cases of accurately evaluating the degree of scientific sophistication in undergraduate majors’ and nonmajors’ written explanations of evolutionary change. In cases in which the software did not perform at the benchmark of “near-perfect” agreement (kappa > 0.80), we located the causes of poor performance and identified a series of strategies for their mitigation. Machine-learning software holds promise as an assessment tool for use in undergraduate biology education, but like most assessment tools, it is also characterized by limitations.

Harnessing technology to improve formative assessment of student conceptions in STEM: Forging a national network

Concept inventories, consisting of multiple-choice questions designed around common student misconceptions, are designed to reveal student thinking. However, students often have complex, heterogeneous ideas about scientific concepts. Constructed-response assessments, in which students must create their own answer, may better reveal students’ thinking, but are time- and resource-intensive to evaluate. This report describes the initial meeting of a National Science Foundation–funded cross-institutional collaboration of interdisciplinary science, technology, engineering, and mathematics (STEM) education researchers interested in exploring the use of automated text analysis to evaluate constructed-response assessments. Participants at the meeting shared existing work on lexical analysis and concept inventories, participated in technology demonstrations and workshops, and discussed research goals. We are seeking interested collaborators to join our research community.

A Critical Analysis of Assessment Quality in Genomics and Bioinformatics Education Research

An analysis of assessment quality in genomics and bioinformatics education literature found that a minority (<10%) of studies provided any validity or reliability evidence. This is concerning as it is at odds with the principles of scientific education research and the educational assessment standards.

EvoGrader: an online formative assessment tool for automatically evaluating written evolutionary explanations

EvoGrader is a free, online, on-demand formative assessment service designed for use in undergraduate biology classrooms. EvoGrader’s web portal is powered by Amazon’s Elastic Cloud and run with LightSIDE Lab’s open-source machine-learning tools. The EvoGrader web portal allows biology instructors to upload a response file (.csv) containing unlimited numbers of evolutionary explanations written in response to 86 different ACORNS (Assessing COntextual Reasoning about Natural Selection) instrument items. The system automatically analyzes the responses and provides detailed information about the scientific and naive concepts contained within each student’s response, as well as overall student (and sample) reasoning model types. Graphs and visual models provided by EvoGrader summarize class-level responses; downloadable files of raw scores (in .csv format) are also provided for more detailed analyses. Although the computational machinery that EvoGrader employs is complex, using the system is easy. Users only need to know how to use spreadsheets to organize student responses, upload files to the web, and use a web browser. A series of experiments using new samples of 2,200 written evolutionary explanations demonstrate that EvoGrader scores are comparable to those of trained human raters, although EvoGrader scoring takes 99% less time and is free. EvoGrader will be of interest to biology instructors teaching large classes who seek to emphasize scientific practices such as generating scientific explanations, and to teach crosscutting ideas such as evolution and natural selection. The software architecture of EvoGrader is described as it may serve as a template for developing machine-learning portals for other core concepts within biology and across other disciplines.