From A Systematic Investigation of Faculty-Produced Think-Pair-Share Questions to Frameworks for Characterizing and Developing Fluency-Inspiring Activities
FFrom A Systematic Investigation of Faculty-Produced Think-Pair-Share Questions toFrameworks for Characterizing and Developing Fluency-Inspiring Activities
Rica Sirbaugh French ∗ Department of Physical Sciences; MiraCosta College; 1 Barnard Drive; Oceanside, CA 18 92056, USA andCenter for Astronomy Education, Department of Astronomy,Steward Observatory; University of Arizona; Tucson, AZ 85721, USA
Edward E. Prather † Center for Astronomy Education, Department of Astronomy,Steward Observatory; University of Arizona; Tucson, AZ 85721, USA (Dated: January 15, 2020)Our investigation of 353 faculty-produced multiple-choice Think-Pair-Share questions leads to keyinsights into faculty members’ ideas about the discipline representations and intellectual tasks thatcould engage learners on key topics in physics and astronomy. The results of this work illustratethat, for many topics, there is a lack of variety in the representations featured, intellectual tasksposed, and levels of complexity fostered by the questions faculty develop. These efforts motivatedand informed the development of two frameworks: (1) a curriculum characterization framework that allows us to systematically code active learning strategies in terms of the discipline represen-tations, intellectual tasks, and reasoning complexity that an activity offers the learner; and (2) a curriculum development framework that guides the development of activities deliberately focused onincreasing learners’ discipline fluency. We analyze the faculty-produced Think-Pair-Share questionswith our curriculum characterization framework, then apply our curriculum development frame-work to generate (1)
Fluency-Inspiring Questions , a more pedagogically powerful extension ofa well-established instructional strategy, and (2)
Student Representation Tasks , a brand newtype of instructional activity in astronomy that shifts the responsibility for generating appropriaterepresentations onto the learners. We explicitly unpack and provide examples of Fluency-InspiringQuestions and Student Representation Tasks, detailing their usage of
Pedagogical DisciplineRepresentations coupled with novel question and activity formats.
I. INTRODUCTION
In this article we describe how an investigation intofaculty-produced curricular materials provides unique in-sights into the choices instructors make when design-ing their own active learning strategies. The results ofthis investigation also expand the theoretical underpin-nings that inform our current curriculum developmentefforts. This investigation focuses on 353 multiple-choicequestions authored by faculty (during professional devel-opment workshops) to target students’ conceptual andreasoning difficulties associated with commonly taughttopics in introductory astronomy and physics. Our goalwas to characterize the information faculty choose to in-clude and emphasize in their questions, and better un-derstand how they structure intellectual tasks they be-lieve will help foster rigorous discourse among studentsduring Think-Pair-Share [1, 2] (or “Peer Instruction” asit is commonly referred to in the college-level introduc-tory physics community [3]). After an initial analysisof the questions, it was clear that we needed a rigorousmethodology that would allow us to meaningfully charac-terize the rich information contained within these ques-tions. Informing our investigation with the theory of so- ∗ [email protected]; https://tiny.cc/rfrenchmcc † [email protected] cial semiotics (the study of communication and meaning-making potentials via signs and symbols that are highlycontextualized within a community) has proven espe-cially valuable. We will describe the professional devel-opment experiences during which these faculty-producedquestions were created and provide insights into our find-ings on the discipline representations and cognitive tasksused in the questions.Additionally, we introduce two exciting new frame-works: a curriculum characterization framework anda curriculum development framework . Arising fromour systematic, socio-semiotic analysis of the faculty-produced Think-Pair-Share questions, our curriculumcharacterization framework allows consistent, objectivecharacterization of the information contained within anypiece of curriculum, set of instructional materials, activelearning strategy, etc. We outline its three-pronged de-sign and offer examples of its application to the faculty-produced multiple-choice questions. Informed by and ex-tending this work, our curriculum development frame-work guides the creation of instructional strategies thatemploy novel combinations of discipline representationsand intellectual tasks designed to help learners’ developtheir discipline fluency. We present Fluency-InspiringQuestions and
Student Representation Tasks astwo curricular objects generated from implementing ourcurriculum development framework. While Fluency-Inspiring Questions expand upon a well-established in-structional strategy, Student Representation Tasks are a r X i v : . [ phy s i c s . e d - ph ] J u l unlike anything previously developed for instruction inastronomy as they “flip the script,” requiring the learn-ers (rather than the instructors) to create the appropriaterepresentations.In the final sections of this paper we provide and ex-plicitly unpack example Fluency-Inspiring Questions andStudent Representation Tasks, highlighting how we as in-structors can shift our thinking towards designing morepedagogically powerful materials and learning experi-ences that can foster learners’ development of disciplinefluency. In the next section we offer some background onour prior curriculum development efforts and theoreticalperspectives. II. BACKGROUND AND THEORETICALUNDERPINNINGS
For the past two decades, the authors and their collab-orators at the Center for Astronomy Education (CAE)have conducted research on the development and effec-tiveness of active learning instructional strategies and as-sessment materials primarily for use in general educationintroductory astronomy courses (commonly referred toas “Astro 101” [4–6]). All of these efforts are informedby a multitude of different theoretical perspectives in-cluding, but not limited to, constructivism [7–10], con-ceptual change theory [11–13], cognitive load theory [14],ontological categories [15, 16], phenomenological primi-tives and knowledge in pieces [17, 18], activation of re-sources [19], facets of knowledge [20], and variation the-ory [21, 22]. These theoretical perspectives significantlyinfluence the design of all our instructional materials toensure they (1) are sensitive to the complexities of the ed-ucational contexts as well as students’ ideas, prior knowl-edge, and intellectual abilities, (2) offer a variety of rep-resentations and scenarios that provide developmentallyappropriate access to the topics, (3) foster critical andreflective thinking, (4) promote meaningful peer-to-peerdiscourse, and (5) engage students in a variety of cogni-tive tasks (e.g. draw, write, rank, sort, predict, calculate,etc.). Examples of materials informed by these theoreti-cal perspectives include Lecture-Tutorials [23–26], Rank-ing Tasks [27], Think-Pair-Share questions [2, 28], andconcept inventories [29–33]. See references [34–41] forresearch into the development and assessment of theseactive learning strategies and assessment materials.
A. Pedagogical Discipline Representations
Here, we outline a particular orientation of our workwith regard to curriculum development and the use ofrepresentations. The hierarchical and scaffolded natureof the learning sequences fostered by our instructionalstrategies requires that we break down complex astro-physical concepts into smaller more manageable chunksin terms of both content and cognitive load. This “chunk- ing” and sequencing allows novice learners to process andcoordinate the discipline information in ways that effec-tively facilitate the development of coherent explanatorymental models. As we endeavor to bring more advancedtopics and recent discoveries in astronomy and astro-physics into the classroom, we frequently find ourselvesneeding a new generation of representations, ones thatemphasize information in ways not typically employedin the discipline. These new representations depict styl-ized physical scenarios and highlight discipline relation-ships that, while invaluable pedagogically, have little tono value to experts and professionals working in thatfield. Generally speaking, the higher the pedagogicalvalue of a representation, the lower its value to disciplineexperts [42, 43]. For this reason, these new representa-tions are called
Pedagogical Discipline Representa-tions (PDRs) [24].Each Pedagogical Discipline Representation affordslearners access to discipline information in ways thatmost textbook and expert representations simply can-not. More precisely, PDRs are representations (waysof conveying discipline information) with specific, nar-rowly focused and well-understood disciplinary affor-dances (potentials for allowing access to pieces of disci-plinary information) that promote unpacking (disassem-bling a package of information and making the variouspieces and connections explicit) while enabling criticaland disciplinary discernment (coming to recognize andunderstand what to focus on and interpreting it or mak-ing meaning using the appropriate context) [42, 44–47,and references therein]. For a more complete discussionon Pedagogical Discipline Representations and their de-velopment with respect to our work in astronomy seeWallace et al. [24, 25] and Hatcher et al. [48]. Our workon the development and testing of PDRs has significantlyinfluenced the research described here to characterize therepresentations created by faculty members while devel-oping their instructional strategies.
B. Social Semiotics
The research and curriculum development describedherein is strongly centered on better understanding howdiscipline representations and intellectual tasks are con-nected to the learning of a discipline. Viewing our workthrough the lens of social semiotics has been particu-larly helpful. As previously stated, social semiotics isthe study of communication and meaning-making poten-tials via signs and symbols that are highly contextualizedwithin a community, culture, etc. [49, 50]. To facilitatemaking and conveying meaning, humans select and con-figure various modes of representation – channels for con-veying information – in complementary ways [51]. Thesemodes of representation (sometimes referred to as just“modes” or “representations” in this work) each have oneor more affordances – potentials to allow access to com-ponents of information via an individual’s perception ofand interaction with the representation itself and a spe-cific environment, context, etc. [52].Virtually every representation used in an instructionalenvironment is limited by its own set of disciplinary af-fordances and pedagogical values [42, 44–47]. Thus, el-evating learners’ knowledge of, and abilities in, the keyideas of a topic or discipline necessarily requires com-bining multiple representations coupled to multiple in-tellectual tasks in ways that facilitate unpacking anddiscerning [42]. Indeed, Fredlund et al. [46] and Lin-der [53] even suggest that the power and success of manyresearch-validated active learning materials and methodsmay lie within a theoretical framing in which the methoditself naturally facilitates the unpacking and disambigua-tion of the disciplinary affordances of a set of representa-tions. Thus, studying instructional strategies by recog-nizing which combinations of representations and intel-lectual tasks are employed can provide insight into thepedagogical beliefs of the authors – in our case, faculty.Next, we describe our efforts to characterize the infor-mation contained within the hundreds of multiple-choiceThink-Pair-Share questions produced by faculty partici-pating in professional development workshops.
III. INVESTIGATING FACULTY-PRODUCEDTHINK-PAIR-SHARE QUESTIONS
Our data is comprised of 353 multiple-choice Think-Pair-Share questions produced by faculty during profes-sional development workshops that included a session de-signed to help instructors better understand how to cre-ate effective multiple choice questions and employ bestpractices for implementing Think-Pair-Share [1, 2]. Notethat in these workshops, faculty were given experienceswith evaluating and designing multiple-choice questionsfeaturing an array of formats, levels of intellectual diffi-culty, and abilities to promote discourse amongst learn-ers.The majority of the questions (293) are from 41 CAETeaching Excellence Workshops [2, 54] held from 2005-2015, targeting current and prospective instructors ofcollege-level general education introductory astronomy,Earth, and space science, regardless of experience or ca-reer stage. These questions span the broad topical areasof the Earth-Sun-Moon system, Renaissance astronomy,solar system, light and atoms, stars, exoplanets and lifein the universe, and galaxies and cosmology. The remain-ing 60 questions are from four meetings of the Workshopfor New Faculty in Physics and Astronomy [55] held atthe American Center for Physics from 2015-2017. Theseprofessional development experiences support primarilyphysics and astronomy instructors who are in the first fewyears of their initial tenure-track appointments. Ques-tions from these workshops fall under the broad topi-cal areas of work and kinetic energy, inelastic collisions,rotational motion, heat and temperature, Gauss’s lawfor electric fields, Faraday’s and Lenz’s laws, simple har- monic motion, and the Bohr model of the atom.Through the careful investigation of these questions,we hoped to gain valuable insights into the choices fac-ulty make when developing instructional materials theybelieve will help students learn their discipline. Whataspects of a particular topic do faculty think are impor-tant? How do faculty choose to represent and emphasizecertain pieces of information? What kinds of cognitiveexercises do faculty think learners should experience? Dothe questions faculty develop address the intended learn-ing outcome(s)?From a cursory analysis of the data, we suspected therewas a lack of variety in the representations, intellectualtasks, and levels of difficulty. But without an explicitframework to help us rigorously document the informa-tion contained in the questions, we could not be certainof the actual distribution of these question features. Itwas clear that we needed a more objective, insightful, andsystematic way to characterize the abundant informationcontained within our data. Further, we believed the sys-tematic use of an objective coding framework informed bysocial semiotics would help us better understand whichrepresentations and intellectual tasks are over- or under-utilized for a particular topic, and could inform directionsfor future curriculum development. The work of Linder[53] and Airey and Linder [56] forms the basis for ourinitial efforts to develop a framework for systematicallycoding the modes of representation and intellectual tasksused, as well as the levels of discourse promoted in thefaculty-produced multiple-choice questions. In the nextsection, we discuss the development of our curriculumcharacterization framework . IV. CURRICULUM CHARACTERIZATIONFRAMEWORK
Our curriculum characterization framework is de-signed to meaningfully code the information containedin the multiple-choice questions using a three-prongedapproach grounded in the answers to the following: • What types and how many different ways of con-veying information are used? • What types of and how many different cognitiveexercises must the learner engage in? • How robust will the discourse be among learnersattempting to explain and defend the reasoning be-hind their answers?To address the first question, we developed a codingschema – heavily influenced by Linder [53] and Airey andLinder [56] – that allows us to classify how informationis represented. Identifying and systematically categoriz-ing the different ways that faculty choose to convey in-formation offers a key insight into understanding howfaculty perceive the various representations’ affordancesand intended ways of making meaning, which are corecomponents of socio-semiotic theory. To address the sec-ond question, we extended the socio-semiotic perspectiveon representations by creating a second coding schemafor cataloging the different types of intellectual tasks onemust engage in when working through an activity. Forthe third question, we refined an existing rubric (previ-ously created by members of CAE) known as the Ques-tion Complexity Rubric (QCR) [28]. This modified QCRis used to rank the complexity involved in unpacking,explaining, and justifying one’s reasoning.We arrived at final versions of the Question Complex-ity Rubric and coding schemata for the representationsand intellectual tasks via an iterative process. We assem-bled a small, but reasonably diverse set of the faculty-produced Think-Pair-Share questions (referred to as thecalibration questions) that exemplified a range of rep-resentations, tasks, and QCR codes. Using this set ofcalibration questions, we trained a team of collaboratorsin the application of the coding schemata and QuestionComplexity Rubric. Through an iterative process of cod-ing the calibration questions and reflecting on our results,we made important revisions that led to the final versionsof the coding schemata and QCR presented here. Sub-sequent efforts by the team to code additional questionsconfirms that our coding methodology leads to reliableand valid characterization of the data.The three-pronged approach above, the codingschemata for the modes of representation and intellectualtasks, and the modified Question Complexity Rubric to-gether comprise our curriculum characterization frame-work . We describe the coding schemata and QCR inmore detail in the following subsections.
A. Modes of representation
We define a mode of representation as a way of convey-ing information. The numbered modes of representationin Table I are general types of information delivery widelyused in teaching college-level science. Some modes havelettered subtypes that serve as common examples and areincluded to assist in characterizing the general mode ofrepresentation. While Think-Pair-Share questions typi-cally do not make use of modes 7–10 they are still in-cluded since our framework is easily generalized to othertypes of curricular materials and instructional methods.Each question is coded by identifying the variousmodes of representation used and listing their corre-sponding numbers from Table I (see Table III). The orderof the numbers is irrelevant but we often list them in theorder the representations are encountered when workingthrough the question.
B. Intellectual tasks
We define an intellectual task as a specific cognitiveexercise that one engages in to arrive at the answer to a
TABLE I. Modes of representation recognized in curricularmaterials. Modes of Representation1 . wordsa. writtenb. spoken a . pictures & diagramsa. photographsb. static imagesc. figuresd. sketches3 . graphs & charts4 . tables5 . mathematical formalism b . numbers c . animations d . simulations e . recordings of realitya. audiob. video10 . gestures a a. facial expressionsb. body movements a Real-time only; does not include recordings; see mode 9. b Equations and other mathematical expressions, e.g. λ max = 6000˚A. Does not include ranked answer choices. c Used anywhere except items explicitly covered by mode 5. d Moving pictures or diagrams with no user interaction. Mayinclude pause/resume/restart controls but disallows changingany variables or parameters. e Animated tool with user interaction mechanisms. User has theability to change and/or control one or more variables orparameters. question. Our list of task codes appears in Table II.Some intellectual tasks are ubiquitous for virtually allquestions, such as “recall” or “interpret.” Characteriz-ing a question using such ubiquitous tasks does not con-tribute meaningfully to our understanding of the essentialreasoning the question evokes, nor does it help distin-guish differences among questions. Such tasks, therefore,are not included in our work. Think-Pair-Share ques-tions do not make use of task 15 in Table II but, justas with modes 7–10 in Table I, it is included becauseour framework is easily generalized beyond the current
TABLE II. Intellectual tasks recognized in curricular materi-als. Intellectual Tasks1 . visualize 9 . rank2 . draw/sketch 10 . sort3 . model 11 . match4 . compare 12 . quantitative reasoning5 . identify 13 . calculate6 . predict 14 . apply/analyze7 . extrapolate 15 . write8 . count application.Unpacking the particulars of these intellectual taskshelps clarify how one categorizes the Think-Pair-Sharequestions in terms of the cognitive exercises they mightpromote. To visualize , one makes a mental image usinginformation in the provided representations, e.g. froma description, graph, table, etc. The task draw/sketch is the act of creating a pictorial or diagrammatic repre-sentation or simply adding detail to a pre-existing one.Here, model ing, means making a physical representationby gesturing and/or using props. To compare is to makeexplicit use of similarities and/or differences by consider-ing items, situations, etc. in relation to each other, e.g.which one is hottest. The task identify means one hasdistinguished or recognized a single item, case, etc. as fit-ting one or more characteristics, criteria, etc., e.g. whichof the labeled locations in the Hertzsprung-Russell dia-gram is a white dwarf. To predict means to forecast oneor more future events from the given information while extrapolate is to estimate a value outside a given rangeby assuming known a trend extends accordingly. Deter-mining how many items fit certain criteria is a count ingtask while rank means arranging items in a certain or-der based on criteria such as hottest to coldest, greatestto least, etc. Classifying or separating items into cate-gories or bins is a sort ing task. To match is to assignitems to corresponding other items. Quantitative rea-soning means doing numerical reasoning using analyticaland mathematical thinking (e.g. proportional reasoning)while calculate means one determines a precise numericalvalue. The apply/analyze task involves developing a lineof reasoning and drawing a conclusion or reaching a de-cision using discipline-specific relationships, rules, laws,etc. This task is used when other listed tasks (from Ta-ble II) cannot completely characterize all of the cognitiveexercises one could engage in to reason through the ques-tion and arrive at an answer . And finally, write meanscreating a coherent narrative and is typically promptedby something like “describe,” “explain,” etc.Most questions naturally require multiple intellectualtasks so it is necessary to differentiate between the main overarching task and any supplementary supporting tasksthat might be needed to arrive at an answer. The wordingof some questions automatically reveals the overarchingtask. For example, “How many of the following...” sig-nals count , while a question that asks for objects to bearranged in a particular order implies that rank is theoverarching task. Supporting tasks are those that, whilenot the main thrust of the question, are still likely to oc-cur when doing the reasoning necessary to arrive at ananswer. Tasks such as quantitative reasoning , compare ,and/or visualize , for instance, might necessarily precedethe overarching task rank and therefore must be includedas supporting tasks. Any task listed in Table II couldserve as either a supporting task or the overarching task.The list of supporting tasks for any given questionshould include all tasks that one could reasonably ex-pect a learner in the target population might engage in to arrive at an answer, whether they actually do so or not .For example, a question coded with the supporting tasks visualize , model , and sketch does not necessarily meanthat all three tasks are required or done by a learnerwhen trying to answer the question. Rather, where onelearner might find it sufficient to only visualize , a differ-ent learner may need to model and/or sketch instead of,or in addition to, visualize . Still, all three tasks must beincluded when coding that particular question.A question is coded by first identifying the overarchingintellectual task required to answer the question, identi-fying all reasonably possible supporting tasks, and thenlisting their corresponding numbers from Table II (seeTable III). We list the overarching task first with thesupporting tasks following in parentheses. While the or-der of the supporting tasks is not relevant, our lists oftencorrelate with the order of tasks that a learner mightengage in when formulating an answer to the question. C. Question Complexity Rubric (QCR)
Building upon the initial work of Cormier et al. [28],a question’s QCR code (Fig. 1) ranks the question’s de-gree of conceptual and cognitive complexity – a rankingwhich also characterizes the richness of the conversationintended to be evoked between learners attempting toexplain and defend the reasoning behind their answers.In this way, the QCR code represents the level of intel-lectual engagement required to convince someone else ofthe correct answer.Thus, to determine a question’s QCR code we con-sider what it takes to unpack and make explicit the af-fordances, pieces of knowledge, and reasoning necessaryto develop and articulate a coherent narrative that shouldconvince another learner of the correct answer. We then
FIG. 1. The Question Complexity Rubric (QCR) used tocode the level of cognitive complexity required to unpack anddefend the solution to a question, problem, etc. consider whether there are multiple concepts that mustbe integrated together while reasoning through the ques-tion. Each question is coded by assigning a number, 1through 4 (the QCR code), from Fig. 1 (see Table III).For a question coded as QCR = 1, the learner needs onlyto state a single fact or element of declarative knowl-edge, whereas a question coded as QCR = 4 requires thelearner to defend their answer using multiple pathwaysof sequential reasoning steps and integrate two or moreconcepts or topics.
V. DATA CHARACTERIZATION
Using our curriculum characterization framework , wesystematically coded the information contained in each ofthe 353 multiple-choice Think-Pair-Share questions writ-ten by faculty participating in professional developmentworkshops. While it is not within the scope of this paperto unpack the breadth and details of all of the data (aseparate paper for this is in preparation), we show twoquestions and offer a few key findings.Figures 2 and 3 each present a faculty-produced ques-tion from our data set. In both figures, the left sidepreserves the representations exactly as presented by thefaculty who created them while the right side shows a“transcript” of the question with italics indicating exacttext from the question and/or answer choices. Table IIIcontains our codes for these questions’ representations,intellectual tasks, and QCR rankings. We chose thesetwo questions because they highlight how multiple-choicequestions can effectively utilize multiple representationscoupled with several intellectual tasks to require com-plex multi-step reasoning – a combination that was notcommon in the questions in our data set.Having worked with thousands of physics and astron-omy faculty over the years in professional developmentsettings, we find that many believe it is extremely diffi-cult to use multiple-choice questions to engage learnersin higher-order thinking and reasoning (i.e. the upperlevels of Bloom’s taxonomy [57, 58]). Somewhat unsur-prisingly then, after applying our curriculum character-ization framework to all of the questions in the data,we find few with a QCR code of 4. Out of the entireset of 353 faculty-produced questions spanning 15 broadtopical areas, 1.7% are QCR = 1, 25.5% are QCR = 2,56.9% are QCR = 3, and only 15.9% are QCR = 4. Ped-agogically speaking, this result is problematic. Helpinglearners develop a more robust understanding of a topicinvolves scaffolding their learning, starting with novice-level situations and working up to problems that fea-ture a wide variety of representations coupled with com-plex reasoning tasks that promote expert-like thinking. Ifmultiple-choice questions are a primary source of engage-ment with, and assessment of, a particular topic, theremust be a diverse assortment of questions that span allQCR levels for that topic. Our data suggests that fac-ulty may be unlikely to produce a significant number of QCR = 4 questions spanning the vast array of topicsthey are likely to address over a term of physics or as-tronomy instruction. It is our experience that facultyoften utilize far too many low-level questions when incor-porating Think-Pair-Share into the classroom. Throughour efforts described here we are explicitly working to ex-pand our communities’ capacities to identify and developQCR = 4 questions and activities.We also found that there are some topics from thisdata whose questions are notably lacking in the diver-sity of representations and/or tasks. For example, the67 questions in the “stars” topical area show a con-siderable over-reliance on the Hertzsprung-Russell dia-gram. This diagram, and the words that frame thequestion and context of the problem, are frequently theonly representations used. It is also worth noting thatthe Hertzsprung-Russell diagram, while virtually indis-pensable, is arguably one of the most rationalized andinformation-rich representations in all of astronomy, soinstructors must be wary of trivializing its significantdepth [59–61]. Additionally, these same “stars” ques-tions show an overwhelming preference for the overar-ching intellectual task compare , with little variation insupplementary tasks. Similarly, most of the 75 questionson “galaxies and cosmology” use words as the lone repre-sentation and emphasize identification as the overarchingintellectual task, sometimes with no supporting tasks.The insights gained from developing and applying ourcurriculum characterization framework to these faculty-produced Think-Pair-Share questions drove us to (1) sys-tematically identify gaps in the diversity of modes andtasks, (2) generate questions to fill those gaps, (3) gener-ate more complex questions that combine multiple rep-resentations and tasks in ways not seen in the data, and(4) create more pedagogically interesting questions thatguide the unpacking of compound and/or complex ideasfor the learners rather than leave them to start from a“blank slate.” Our curriculum development framework ,described next, was informed by this work. VI. CURRICULUM DEVELOPMENTFRAMEWORK
We take “fluency” in a discipline idea to mean theability to easily unpack and transition through multiplemodes of representation (including generating them whennecessary), discern their disciplinary affordances, and en-gage in various cognitive exercises to develop, apply, andarticulate meaning in the appropriate disciplinary con- Representations commonly used to teach a particular concept(like those found in textbooks) frequently harbor key disciplinaryaspects that are not immediately discernible. These “rational-ized” representations contain information that has been compart-mentalized via extensive discussion and reconciliation by disci-pline experts, often over long periods of time [46].
FIG. 2. Example A from the faculty-produced Think-Pair-Share questions. The original is reproduced on the left with a“transcript” on the right. Italicized portions indicate text of question prompt and answer choices.TABLE III. Results of coding the questions in Figs. 2 and 3 using our curriculum characterization framework from § IV.ExampleQuestion Broad Topic Area Modes ofRepresentation a Intellectual Tasks:overarching (supporting) a QCRCodeA from Fig. 2 Earth-Sun-Moon system 2, 1, 6 b c
4B from Fig. 3 light and atoms 1, 2, 6, 3 d
11 (1, 3, 4, 12) e a Note that while we indicate only the numbers of the modes and tasks when coding our data, we include the corresponding names ofthe modes and tasks here in the table footnotes, to aid the reader. b pictures/diagrams, words, numbers c count (apply/analyze, predict, visualize, draw/sketch, model) d words, pictures/diagrams, numbers, graphs/charts e match (visualize, model, compare, quantitative reasoning) text(s). Thus, successful pedagogies must (1) acknowl-edge the disciplinary affordances of multiple complemen-tary representations, (2) integrate them with multipledifferent intellectual tasks, (3) organize the informationappropriately, and (4) offer plentiful opportunities forlearners to practice unpacking, discerning, making mean-ing, and articulating reasoning while fostering reflectionand self-assessment. When this host of processes becomesunproblematic and nearly second-nature or automatic,one is said to be fluent [42].Experts in a discipline routinely engage in “disciplinarydiscourse” [62], moving with practiced skill among a va-riety of representations – selecting, interpreting, explain-ing, reconciling, and generating them – within a contextthat is updated as discipline knowledge and understand-ing progresses. Facilitating the development of similar“representational competence” [63] in novice learners ischallenging, even more so since the discipline contentis often perceived as independent of the representationsused [64]. Novices lack the ability to recognize importantinformation and relationships in a variety of modes andfilter it through the appropriate context(s). That is, they cannot yet critically discern the disciplinary affordancesof multiple representations and coordinate them to makesense of disciplinary knowledge [65].Kohl and Finkelstein [64] argue that developing fluencyin disciplinary content cannot be separated from the rep-resentations used to teach that content. We concur withtheir finding that requiring students to engage in mul-timodal learning can improve learners’ performances.Airey and Linder [56] point out that learners who havenot been exposed to a wide variety of representationsfor a topic are unlikely to become fluent in that topic.Therefore, instructors who employ a limited set of repre-sentations cannot expect to move their students to disci-pline fluency since the learners’ opportunities to unpackand discern are restricted by the lack of variety. Addi-tionally, students tend to call upon the representation(s) In social semiotics, the term “multimodality” is often applied tothe use of multiple semiotic resources, e.g. modes of representa-tion, and their affordances working together to create an instanceof communication [49, 51, 66].
FIG. 3. Example B from the faculty-produced Think-Pair-Share questions. The original is reproduced on the left with a“transcript” on the right. Italicized portions indicate text of question prompt. most frequently used rather than the one(s) best suitedto the task at hand [47].Since different representations offer different disci-plinary affordances [46, 67], no single representation byitself is likely to capture all aspects of the physical situ-ation it models, regardless of rationalization. Multiplerepresentations, however, may work together to create a“collective disciplinary affordance” [53], providing a moreholistic model of the physical situation and more poten-tial points of access for the development of disciplinaryknowledge and fluency. Thus, it is pedagogically morepowerful to use multiple representations than to rely on asingle one to do the work of many [46]. This is a guidingprinciple behind the new kind of multiple-choice ques-tions we call
Fluency-Inspiring Questions (see § VII).However, just because a student answers one or morequestions correctly does not necessarily imply a deep dis-ciplinary understanding indicative of fluency. Learnersmust also be able to unpack those resources [46] and cometo notice, appreciate, and effectively coordinate the disci-plinary affordances of those modes lest they fall victim to discourse imitation – the ability to utilize representationsappropriately without having the associated discipline- specific understandings [42, 56]. This suggests the needfor learners to also be able to generate appropriate rep-resentations when needed, and not simply know how toutilize pre-existing ones. Such is the motivation behindthe novel activities we call
Student RepresentationTasks (see § VIII).Informed by these theoretical perspectives on disciplinefluency and our results from applying our curriculumcharacterization framework, we created our curriculumdevelopment framework . This framework is structured toaid the generation of learning opportunities that explic-itly promote fluency and requires developers to:(1) unpack a discipline topic in terms of the learningoutcomes for the intended audience, e.g. discernwhat is required in order for one to demonstratefluency;(2) examine the canonical discipline representationsused for the topic to determine whether they havethe appropriate affordances or whether they mis-lead or unintentionally create opaque learning en-vironments that result in learners developing in-complete, incorrect, and/or incoherent ideas andexplanations;(3) determine whether there are different modes ofrepresentation that could or should be used, andwhether the creation of new Pedagogical DisciplineRepresentations is necessary;(4) design complex scenarios including intellectualtasks that require the learner to use a combinationof essential features from various modes of repre-sentation in order to develop robust explanations;and finally,(5) structure the activity such that it challenges learn-ers at a high enough cognitive level to ultimatelyaid in developing and demonstrating fluency in thetopic, i.e. QCR = 4.We next unpack
Fluency-Inspiring Questions and
Student Representation Tasks , two innovative in-structional strategies generated from the application ofour curriculum development framework.
VII. FLUENCY-INSPIRING QUESTIONS
The multiple-choice questions we generate usingour curriculum development framework are
Fluency-Inspiring Questions (FIQs) in that they require thelearner to extract information from one representationand map it onto one or more others while engagingin multiple cognitive tasks. It is important to notethat Fluency-Inspiring Questions are for use in a post-instruction context which, in our case, means post-lectureand usually after implementing other collaborative activelearning strategies (such as Lecture-Tutorials). The fol-lowing examples illustrate how the structures and designsof FIQs purposefully facilitate discernment, unpacking,and mapping of information from one representation toanother, leading the learner to work through a series ofcomplementary tasks and complex discipline-specific sce-narios, resulting in a QCR code of 4. In particular, theseexamples employ unusual forms of fill-in-the-blank andmatching, two unexpectedly powerful question formatsthat can help foster robust intellectual engagement thatleads to more expert-like disciplinary discourse amonglearners.
A. Heat engine
When teaching thermodynamics, instructors have anexpectation that learners will be able to correctly con-nect the ideas of work and energy transfer to the pathsin a pV graph and to sequences of real, physical events,such as those depicted in commonly used piston dia-grams. Correctly connecting these ideas is a hallmark ofdemonstrating fluency with a heat engine process. Whatwe commonly find, though, are questions that deal with one part of the graphical path and a partial sequenceof the piston diagram process, or questions asking stu-dents to calculate the work for the displacement of thepiston under a partial set of conditions. These questionsstill address only pieces of the larger, interconnected setof concepts. If we are not careful, creative, and delib-erate in our question design, we may inadvertently in-spire only discourse imitation [42, 56]. What we reallyneed is a question that requires the learner to evaluateall aspects of a full thermodynamic cycle, connecting thetheoretical processes, physical manifestations, graphicalrepresentations, and mathematical consequences simul-taneously . The Fluency-Inspiring Question in Fig. 4 isdesigned to accomplish this. FIG. 4. A thermodynamics FIQ for a heat engine process.
Aside from the novel “matching list,” the representa-tions used are all fairly common, with similar versionsfound in most introductory physics and thermodynamicstextbooks and curricular materials. There are no Ped-agogical Discipline Representations in this example andnone are needed since the canonical representations af-ford access to the necessary pieces of disciplinary infor-mation. The use of a “matching list” with the overar-ching task count sets up a curiously powerful combina-tion of representations and tasks that orchestrates thelearner’s cognitive efforts by requiring one to map backand forth among a variety of pieces of discipline informa-tion while simultaneously calling upon multiple theoreti-cal and mathematical principles. This results in a uniquefluency-inspiring opportunity.Thus, we have explicitly addressed the five require-ments of our curriculum development framework ( § VI).And while the individual pieces of this question arewidely taught in introductory thermodynamics lessons,their combination in this format proves intellectuallychallenging – sometimes even for faculty working throughthe problem during professional development workshopson the design of Think-Pair-Share questions.0
B. The Bohr atom and EM radiation
Instruction on the electron transitions inside an atomtypically involves traditional energy level diagrams andassumes that students have (1) a functional understand-ing of the concepts of emission and absorption and(2) mastered the relationships between wavelength, fre-quency, and energy of a photon. Successful integration ofthese ideas indicates fluency with the interaction betweenphotons and atoms. The properties of light and (some-times) the processes of emission and absorption are oftenfirst dealt with in a more compartmentalized, piecewisefashion, and faculty often assume that students will au-tomatically transfer and coherently reason about theseideas when encountering representations of energy levelsin an atom. From our decades of classroom experienceteaching these topics, we find that this is very often notthe case. For example, for the cases of absorption andemission, students frequently confuse the direction of thearrow used to indicate the transition of the electron. Stu-dents also struggle to correctly connect the magnitude ofthe electron transition to the corresponding energy orwavelength of the associated photon, incorrectly reason-ing that length of the electron transition arrow is directlyproportional to the photon’s wavelength, or that transi-tions involving a greater number of energy levels alwaysresult in (or from) a photon of greater energy.The Fluency-Inspiring Question in Fig. 5 uses a fill-in-the-blank format to guide the students’ thought pro-cesses. In this question we see aspects of traditionalrepresentations that have been altered to serve as Ped-agogical Discipline Representations. For example, theBohr atom PDR in Fig. 5 combines information aboutenergy levels and all possible bound – “arrowed” – elec-tron transitions with the productive orbital features ofthe Bohr atom in a somewhat unconventional way thataffords learners access to the phenomena of emission andabsorption and the corresponding movement of electronsbetween energy levels. Similarly, “wiggly” arrows aresometimes shown when textbooks introduce photons andelectromagnetic waves. But combining multiple wiggly
FIG. 5. FIQ integrating electromagnetic radiation with theBohr model of the atom. arrows of varying wavelength, frequency, and amplitudeinto a single diagram used to represent different physi-cal parameters of photons in this way is uncommon intraditional astronomy and physics instruction. Yet it isinvaluable when the differentiation of these photon prop-erties is a goal of instruction on the topic. Thus, thesewiggly arrows become a PDR. And while the frequenciesin the table are unrealistic for an actual atom, they fa-cilitate paying attention to the relationships among thefrequencies that result from the physical situation. Thishelps novice learners to focus on the appropriate rela-tionships rather than determining precise mathematicalvalues. Thus, this frequency table is also a PDR.The format of this Fluency-Inspiring Question requiressimultaneously mapping back and forth between the ar-rows on the Bohr atom, the properties of the wiggly pho-tons, ideas about emission versus absorption, and photonfrequencies and energies, thus elevating this question toa QCR = 4.
C. Detecting exoplanets via gravitationalmicrolensing
The Fluency-Inspiring Question in Fig. 6 uses a single-outcome matching format to focus a student’s thinking.This question relies strongly on the information providedin the question stem, which clearly defines the single out-come and requires learners to both evaluate the graphsand diagrams, and find a matching set. The features andpatterns of the graphs and diagrams, along with the com-binations represented in the answer choices, are selectedto emulate a wide range of possible outcomes and encap-sulate the most common incorrect reasoning displayed bystudents during classroom testing on the topic [24]. Asan example, students often believe that the left or rightposition of a planet relative to the companion star will bepreserved in the left or right location (respectively) of thebrightness peak caused by the planet on the brightnessvs. time graph. These students do not always realizethat, depending upon the system’s direction of travel, a
FIG. 6. FIQ on detecting exoplanets via gravitational mi-crolensing.
VIII. STUDENT REPRESENTATION TASKS
Our curriculum development framework helped us tocreate Fluency-Inspiring Questions, a more pedagogicallypowerful extension of a well-established active learningstrategy. But perhaps a more noteworthy curricular ob-ject generated from the application of our curriculumdevelopment framework is a brand new type of activelearning strategy, one not previously seen in astronomyinstruction:
Student Representation Tasks (SRTs).These fluency-inspiring active learning strategies are es-pecially innovative in that they “flip the script,” makingthe learners responsible for creating the representations.This shifting of cognitive load, from interpreting to cre-ating representations, is a dramatic pedagogical designchoice and was informed by the theoretical perspective ofsocial semiotics. This unique shift in how representationsare used to motivate fluency warrants specific examplesthat unpack how SRTs focus the learners’ cognitive ef-forts.In general, Student Representation Tasks are designedto intellectually engage learners by requiring them toevaluate and make connections between complex astro-physical relationships and reflect upon those ideas as theyproduce their own discipline representations to depict aspecific physical scenario. The student-generated disci-pline representations commonly take the form of a dia-gram or sketch of the physical situation accompanied bylabels, arrows, data tables, graphs, etc., that characterizethe relevant astrophysics. Tytler et al. [68] suggest thatdrawing both promotes and influences reasoning in sev- eral ways: (1) reasoning occurs via the act of drawing,(2) the drawing itself facilitates further reasoning, and(3) the resulting drawing stands as a representation ofreasoning. They also suggest that three conditions mustbe met in order for drawing to facilitate and support rea-soning and scaffold learning [68]. First, the activity mustbe carefully structured such that the act of creating therepresentation(s) is viewed by the learners as a reason-ing process, allowing them to interpret and explain oneor more phenomena. The activity’s structure, then, re-quires a sufficiently clear focus and carefully coordinatedmodes and associated affordances such that learners cansuccessfully critique their own work while allowing forsome diversity in the resulting drawings. Second, thestudents must already have the necessary background inboth content knowledge and exposure to the relevant con-ventional representations. And finally, instructors mustprovide meaningful real-time and ongoing guidance, feed-back, and scaffolding support.From the perspectives of social semiotics, instructionaldesign, and student engagement, the creation of StudentRepresentation Tasks necessitates considering some ofthe requirements of our curriculum development frame-work ( § VI) slightly differently. The first requirementguides development in essentially the same way sinceSRTs are still focused on one or more particular disci-pline topics and associated learning outcomes. SRT de-velopment is somewhat different, however, with regard tohow framework requirements 2–5 play out. We still ex-amine the canonical discipline representations, considertheir pedagogical values, and carefully consider how othermodes of representations and intellectual tasks mightbetter align with the discipline topic’s learning outcomes.However, in deciding on the modes of representation andcreating the corresponding physical situation that willmeaningfully connect them, we must consider that for anSRT, ultimately it is the students who will create the rep-resentations . The activity design must then orchestratestudents working in groups to engage in the discoursethat (hopefully) leads to the thoughtful creation of ap-propriate discipline representations. In this way, StudentRepresentation Tasks are quite different from most activelearning activities in astronomy as they explicitly shift therole of generating representations onto the students – ashift that fosters unique opportunities from both a ped-agogical design and a student learning standpoint.Student Representation Tasks are post-instruction ac-tivities (in our case this means post-lecture and af-ter implementing other active learning strategies suchas Lecture-Tutorials, Ranking Tasks, and Think-Pair-Share) designed to be completed in small collaborativegroups of two or three students. To begin, studentsare given a problem statement that provides essentialdetails about a highly contextualized physical scenarioalong with key information regarding the desired repre-sentations. The peer-to-peer discussion fostered by anSRT brings about a negotiation of choices the studentsmust make in order to create the representations. This2disciplinary discourse is the vehicle that furthers the de-velopment of fluency with the topic.Next, we highlight two Student Representation Tasksto provide insight into how our curriculum developmentframework leads to particular instructional design choicesand pedagogical outcomes. The first example, on thetopic of Doppler shift, shows how an SRT’s instructionsand context can motivate the creation of several differ-ent representations (all on the same topic) to result inthe construction of a more complex, final representationthat confronts known student conceptual and reasoningdifficulties. In the second example, we provide studentswith an opportunity to think about one of the most in-triguing ideas in science, lookback time , and the notionthat telescopes serve as time machines, allowing us toobserve events that occurred in the past. Additionally,this activity gives students the chance to connect look-back time with their knowledge about stellar properties,stellar evolution, and a canonical discipline representa-tion that is used extensively in nearly every astronomycourse: the Hertzsprung-Russell diagram.
A. Doppler shift
Students’ prior knowledge about, and experienceswith, Doppler shift can make it surprisingly challengingfor them to comprehend that the phenomenon is due onlyto the relative motion between the source and observer.Even after instruction on the topic, this struggle persists.Our Doppler shift Student Representation Task (Fig. 7)targets some of the key misconceptions and reasoning dif-ficulties encountered when helping students become flu-ent with the concept.The sequence of prompts in the Doppler shift SRT asksstudents to first create sketches of dark line absorptionspectra, then make a data table, and finally sketch energyoutput vs. wavelength graphs. Students are given infor-mation that is physically relevant and information thattargets key na¨ıve ideas and reasoning difficulties, such asrelating Doppler shift to a star’s color or distance. It isimportant to note that neither the students’ na¨ıve ideasnor the usual course treatment of the topic (at this pointin a typical term’s sequence) are connected to photomet-ric or cosmological redshifts, the Hubble-Lemaˆıtre Law,or interstellar reddening.While the first representation provides sufficient chal-lenge to those students with only a rudimentary under-standing, it is in doing the later parts of this activity thatstudents begin to engage with their peers at deeper levels.Note that in the second part of the activity we provideinformation that is explicitly chosen to confront the twomost common difficulties students have when reasoningabout physical situations involving the Doppler shift.Our decades of classroom experience teaching Dopplershift shows us that many students invoke an incorrect re-lationship between an object’s color and the color labelused in science to identify the change in wavelength due
FIG. 7. SRT on Doppler shift. to the relative motion between the source and observer.These learners incorrectly reason, for example, that a starthat is actually red must be moving away from the ob-server, or that a star whose light is redshifted must actu-ally be red in its true color. Additionally, some studentsincorrectly associate an object’s distance with the direc-tional sense of the Doppler shift and with the color of thewavelength change, reasoning that a star moving towardthe observer must be closer than a star that is movingaway, or that the shift to a shorter, “bluer” wavelengthimplies that the object must be closer than one whoselight is shifted to longer, “redder” wavelengths. Theinformation provided in the activity regarding a star’scolor, distance, and position relative to the moving ob-server are deliberately varied in order to provide studentgroups with sufficient contrasting cases that address thedifferent Doppler-specific reasoning difficulties. Althoughnot explicitly called for, we find that a majority of stu-dent groups choose to make drawings of the physical sce-nario described in the second part – a sign that they findthe representation valuable in, and possibly necessary to,facilitating their unpacking and discernment of this phe-nomenon.In the third part of the Doppler shift SRT, studentsmust estimate a numerical wavelength for one of theshifted absorption lines associated with each of the threestars while considering the other information about color,distance, and relative position (direction of motion). We3find this to be a highly discriminating task, and one thatstudents rarely, if ever, encounter in traditional instruc-tion or active learning activities on this topic (includingthose we have previously developed).The final part of this Student Representation Task re-quires students to create two energy output vs. wave-length graphs for one of the three stars. This is an im-portant intellectual engagement opportunity for studentswho incorrectly reason that only the absorption line atthe end of the spectrum corresponding to the color ofthe light associated with the shift (redshift or blueshift)will be shifted from its “rest” wavelength. Students arealso challenged to correctly draw what is essentially ablackbody curve such that the peak is correct for theactual color of the star (Wien’s law) and the three dips(absorption features) are repositioned relatively correctlyin accordance with the sense of Doppler shift occurring(“red” or “blue”). It is this part of the SRT that elevatesthe activity to a QCR = 4.
B. Lookback times and stellar properties
The information provided in the Lookback Times andStellar Properties Student Representation Task (Fig. 8)includes the evolutionary state of a star, its spectral type,its main sequence lifetime, and distances from the star tothree observers in the universe. The provided informa-tion establishes a particular set of outcomes for differentlocations in the universe. We first ask students to drawtwo circles centered on the star, one with a radius of 3million light-years and another of 13 million light-years –
FIG. 8. SRT on lookback times and stellar properties. distances that correspond to the light travel time to thebeginning of the star’s main sequence phase and to thebeginning of its red giant phase. Observer distances arechosen such that one observer sees the star as a (highmass) main sequence star, another sees it as a red giant,and another sees nothing. This combination of parame-ters creates a set of potential circumstances that allowsus to differentiate between students who have a robustmodel of lookback time and those who believe that thedistance between the star and an observer relates to thetime since the beginning of the star’s life rather thanthe time back from the star’s current state of existence.By making students reason simultaneously about look-back time, stellar evolution, and stellar properties on aHertzsprung-Russell (H-R) diagram, this SRT providesstudents the opportunity to engage at the QCR = 4 level.Instructors are quickly able to discern the reasoningpathways students use by seeing how they depict on theirH-R diagrams the different evolutionary states each ob-server will see for Star X. That is, a student who places adot (or large blue circle) to indicate a main sequence starthat is both luminous and hot for the observer at locationA, and a dot (or large red circle) to indicate a star thatis both luminous and cool for the observer at locationC, is simultaneously indicating the correct evolutionarystates while demonstrating an incorrect understanding ofhow lookback time changes how (and when) the observerperceives an object or event.
IX. CONCLUSIONS
We highlight the theoretical perspectives that informour past curriculum development work and the develop-ment of our curriculum characterization framework usedto identify the variety of modes of representation and in-tellectual tasks used in, as well as the levels of disciplinarydiscourse required to explain the reasoning behind one’sanswers to, an active learning activity. We briefly dis-cuss the application of our curriculum characterizationframework to systematically code 353 faculty-producedmultiple-choice Think-Pair-Share questions and the in-sights this work provides into the decisions faculty makewhen given the opportunity to design curriculum in aprofessional development setting. This investigation re-vealed that, for some astronomy and physics topics, thereappears to be an over-reliance on a small assortment ofdiscipline representations and intellectual tasks, and apredisposition towards relatively low levels of complex-ity. We acknowledge that our results may be incompleteas the professional development workshops’ settings hadseveral limitations that may have restricted faculty mem-bers’ abilities to generate higher level questions. Addi-tionally, the faculty who produced these questions are notrepresentative of all faculty, as many were in the earlystages of their careers, some with little to no teachingexperience.What we learned from our investigation into these4Think-Pair-Share questions informed the creation of asecond framework – our curriculum development frame-work – that is useful for generating active learning strate-gies that move students towards discipline fluency by cre-ating rich opportunities for students to practice discern-ment, unpacking, reflection, and metacognition.We used our curriculum development framework todesign
Fluency-Inspiring Questions , which help stu-dents make robust connections amongst a complex setof complementary discipline representations, cognitivetasks, and discipline-specific ideas. We also provide in-sight into another new type of active learning activitygenerated using this framework –
Student Represen-tation Tasks . SRTs shift the responsibility of creatingdiscipline representations onto the shoulders of the learn-ers and are pedagogically very powerful, both in terms ofthe richness of the student learning experience fosteredand the discriminatory abilities these tasks offer the in-structor with regard to revealing what has or has notbeen learned through prior instruction.Developing and using our two frameworks also informshow we, as instructors and researchers, think about ourdisciplines, and provides a pathway for exploring morepedagogically interesting and powerful opportunities to help learners develop their discipline fluency. Our it-erative and continuously evolving process of research-informed curriculum development – supported by numer-ous theoretical perspectives – leads to innovations in thedevelopment and assessment of active learning strategiesand provides new insights into the teaching and learn-ing of physics and astronomy. It is exciting to note thatsharing preliminary versions of our frameworks with theDBER community has (1) led to collaborations withfaculty who were inspired to create their own FIQs anddevelop new curricular materials on topics as yet unad-dressed with active learning strategies, and (2) opened updialogues with faculty outside of physics and astronomyabout the extension and application of our frameworksto their disciplines. ACKNOWLEDGMENTS
Many thanks go to Nate Goss for his substantial inputinto numerous sessions iterating on and refining the cod-ing schemata, and to Gina Brissenden for her assistancecompiling the questions from the various workshops. Weare also grateful to three referees for their thoughtfulcomments that vastly improved this manuscript. [1] G. Brissenden and E. Prather, Think-Pair-Share: A revised “how-to” guide, website (n.d.), https://astronomy101.jpl.nasa.gov/download/workshopfiles/Think-Pair-ShareHow-ToGuide.pdf .[2] E. E. Prather and G. Brissenden, Development and ap-plication of a situated apprenticeship approach to pro-fessional development of astronomy instructors, Astron.Educ. Rev. , 1 (2009).[3] E. Mazur, Peer Instruction: A User’s Manual (PearsonPrentice Hall, Upper Saddle River, NJ, 1997).[4] R. S. French, Everyone’s universe: Teaching astronomyin community colleges, in
Astronomy Education, Volume1: Evidence-based instruction for introductory courses ,2514-3433, Vol. 1, edited by C. Impey and S. Buxner(IOP Publishing Ltd., Bristol, UK, 2019) Chap. 11.[5] A. L. Rudolph, E. E. Prather, G. Brissenden, D. Con-siglio, and V. Gonzaga, A national study assessing theteaching and learning of introductory astronomy Part II:The connection between student demographics and learn-ing, Astron. Educ. Rev. (2010).[6] E. E. Prather, A. L. Rudolph, and G. Brissenden, Teach-ing and learning astronomy in the 21st century, Phys.Today , 41 (2009).[7] E. von Glasersfeld, Radical Constructivism: A Way ofKnowing and Learning , edited by P. Ernest, Studies inMathematics Education, Vol. 6 (Falmer Press, London,UK, 1995).[8] L. S. Vygotsky,
Mind in Society: The Developmentof Higher Psychological Processes , edited by M. Cole, Discipline Based Education Research
V. John-Steiner, S. Scribner, and E. Souberman (Har-vard University Press, Cambridge, MA, 1978).[9] J. Piaget, Part I Cognitive development in children: Pi-aget: Development and learning, J. Res. Sci. Teach. ,176 (1964).[10] J. S. Bruner, The Process of Education (Harvard Univer-sity Press, Cambridge, MA, 1960).[11] R. Duit and D. F. Treagust, Conceptual change: A pow-erful framework for improving science teaching and learn-ing, Int. J. Sci. Educ. , 671 (2003).[12] P. H. Scott, H. M. Asoko, and R. H. Driver, Teaching forconceptual change: A review of strategies, in Researchin Physics Learning: Theoretical Issues and EmpiricalStudies, Proceedings of an International Workshop , 131,edited by R. Duit, F. Goldberg, and H. Niederer (IPN,Kiel, Germany (1991), 1992) p. 310.[13] G. J. Posner, K. A. Strike, P. W. Hewson, and W. A.Gertzog, Accommodation of a scientific conception: To-ward a theory of conceptual change, Sci. Educ. , 22(1982).[14] P. Chandler and J. Sweller, Cognitive load theory and theformat of instruction, Cognition and Instr. , 293 (1991).[15] M. T. H. Chi, Three types of conceptual change: Beliefrevision, mental model transformation, and categoricalshift, in Handbook of Research on Conceptual Change ,edited by S. Vosniadou (Lawrence Erlbaum Associates,Inc., Hillsdale, NJ, 2008) Chap. 3, p. 61.[16] M. T. H. Chi and R. D. Roscoe, The processes and chal-lenges of conceptual change, in
Reconsidering Concep-tual Change: Issues in Theory and Practice , edited byM. Lim´on and L. Mason (Springer, Dordrecht, Nether-lands, 2002) Chap. 1. [17] E. E. Prather, T. F. Slater, and E. G. Offerdahl, Hintsof a fundamental misconception in cosmology, Astron.Educ. Rev. , 28 (2003).[18] A. A. diSessa, Knowledge in pieces, in Constructivismin the Computer Age , edited by G. Forman and P. Pufall(Lawrence Erlbaum Associates, Inc., Hillsdale, NJ, 1988)Chap. 3, p. 49.[19] D. Hammer, A. Elby, R. E. Scherr, and E. F. Redish,Resources, framing, and transfer, in
Transfer of Learn-ing from a Modern Multidisciplinary Perspective , Cur-rent Perspectives on Cognition, Learning and Instruc-tion, edited by J. P. Mestre (Information Age Publishing,Greenwich, CT, 2006) Chap. 9.[20] J. Minstrell, Facets of students’ knowledge and relevantinstruction, in
Research in Physics Learning: TheoreticalIssues and Empirical Studies, Proceedings of an Interna-tional Workshop , 131, edited by R. Duit, F. Goldberg,and H. Niederer (IPN, Kiel, Germany (1991), 1992) p.110.[21] M. L. Lo,
Variation Theory and the Improvement ofTeaching and Learning , G¨oteborg Studies in Educa-tional Sciences No. 323 (G¨oteborg : Acta UniversitatisGothoburgensis, G¨oteborg, Sweden, 2012) http://hdl.handle.net/2077/29645 .[22] F. Marton and M. F. Pang, On some necessary conditionsof learning, J. Learning Sci. , 193 (2006).[23] C. S. Wallace and E. E. Prather, Lecture-Tutorials in in-troductory astronomy, in Astronomy Education, Volume1: Evidence-based instruction for introductory courses ,2514-3433, Vol. 1, edited by C. Impey and S. Buxner(IOP Publishing Ltd., Bristol, UK, 2019) Chap. 3.[24] C. S. Wallace, T. G. Chambers, E. E. Prather, andG. Brissenden, Using graphical and pictorial represen-tations to teach introductory astronomy students aboutthe detection of extrasolar planets via gravitational mi-crolensing, Am. J. Phys. , 335 (2016).[25] C. S. Wallace, E. E. Prather, S. D. Hornstein, J. O.Burns, W. M. Schlingman, and T. G. Chambers, A newLecture-Tutorial for teaching about molecular excitationsand synchrotron radiation, Phys. Teach. , 40 (2016).[26] E. E. Prather, T. F. Slater, J. P. Adams, J. M. Bailey,L. V. Jones, and J. A. Dostal, Research on a Lecture-Tutorial approach to teaching introductory astronomy fornonscience majors, Astron. Educ. Rev. , 122 (2005).[27] D. W. Hudgins, E. E. Prather, D. J. Grayson, and D. P.Smits, Effectiveness of collaborative Ranking Tasks onstudent understanding of key astronomy concepts, As-tron. Educ. Rev. , 1 (2007).[28] S. Cormier, E. Prather, and G. Brissenden, An online na-tional archive of multiple-choice question for Astro 101and the development of the Question Complexity Rubric,in Earth and Space Science: Making Connections in Ed-ucation and Public Outreach , San Francisco, CA, Astro-nomical Society of the Pacific Conference Series, Vol. 443,edited by J. B. Jensen, J. G. Manning, and M. G. Gibbs(ASP, 2011) p. 439.[29] K. E. Williamson, S. Willoughby, and E. E. Prather, De-velopment of the Newtonian Gravity Concept Inventory,Astron. Educ. Rev. (2013).[30] J. M. Bailey, B. Johnson, E. E. Prather, and T. F. Slater,Development and validation of the Star Properties Con-cept Inventory, Int. J. Sci. Educ. , 2257 (2012).[31] C. S. Wallace, E. E. Prather, and D. K. Duncan, A studyof general education astronomy students’ understandings of cosmology. Part I. Development and validation of fourconceptual cosmology surveys, Astron. Educ. Rev. (2011).[32] C. S. Wallace, E. E. Prather, and D. K. Duncan, A studyof general education astronomy students’ understandingsof cosmology. Part II. Evaluating four conceptual cosmol-ogy surveys: A classical test theory approach, Astron.Educ. Rev. (2011).[33] E. M. Bardar, E. E. Prather, K. Brecher, and T. F.Slater, Development and validation of the Light andSpectroscopy Concept Inventory, Astron. Educ. Rev. ,103 (2007).[34] C. S. Wallace, T. G. Chambers, and E. E. Prather, Itemresponse theory evaluation of the Light and SpectroscopyConcept Inventory national data set, Phys. Rev. ST –Phys. Educ. Res. (2018).[35] J. W. Eckenrode, E. E. Prather, and C. S. Wallace, Cor-relations between students’ written responses to Lecture-Tutorial questions and their understandings of key astro-physics concepts, J. Coll. Sci. Teach. , 86 (2015).[36] W. M. Schlingman, E. E. Prather, C. S. Wallace, A. L.Rudolph, and G. Brissenden, A classical test theory anal-ysis of the Light and Spectroscopy Concept Inventorynational study data set, Astron. Educ. Rev. (2012).[37] C. S. Wallace, E. E. Prather, and D. K. Duncan, A studyof general education astronomy students’ understandingsof cosmology. Part III. Evaluating four conceptual cos-mology surveys: An item response theory approach, As-tron. Educ. Rev. (2012).[38] C. S. Wallace, E. E. Prather, and D. K. Duncan, A studyof general education astronomy students’ understandingsof cosmology. Part IV. Common difficulties students ex-perience with cosmology, Astron. Educ. Rev. (2012).[39] C. S. Wallace, E. E. Prather, and D. K. Duncan, A studyof general education astronomy students’ understandingsof cosmology. Part V. The effects of a new suite of cos-mology Lecture-Tutorials on students’ conceptual knowl-edge, Int. J. Sci. Educ. , 1297 (2012).[40] M. C. LoPresto, Comparing a lecture with a tutorial inintroductory astronomy, Phys. Educ. , 196 (2010).[41] E. E. Prather, A. L. Rudolph, G. Brissenden, and W. M.Schlingman, A national study assessing the teaching andlearning of introductory astronomy. Part I. The effect ofinteractive instruction, Am. J. Phys. , 320 (2009).[42] J. Airey and C. Linder, Social semiotics in universityphysics education, in Multiple Representations in PhysicsEducation , Models and Modeling in Science Education,Vol. 10, edited by D. F. Treagust, R. Duit, and H. E. Fis-cher (Springer, Cham, Switzerland, 2017) Chap. 5, p. 95.[43] J. Airey, Social semiotics in higher education: Ex-amples from teaching and learning in undergraduatephysics, in
SACF Singapore-Sweden Excellence Semi-nars (Swedish Foundation for International Coopera-tion in Research in Higher Education (STINT), 2015)p. 103, http://uu.diva-portal.org/smash/get/diva2:867422/FULLTEXT01.pdf .[44] U. Eriksson, Disciplinary discernment: Reading the skyin astronomy education, Phys. Rev. ST - Phys. Educ.Res. (2019).[45] J. Airey, U. Eriksson, T. Fredlund, and C. Linder, Onthe disciplinary affordances of semiotic resources, Pre-sented at the First Conference of the International As-sociation for Cognitive Semiotics (Lund, Sweden, 2014)p. 54, http://urn.kb.se/resolve?urn=urn:nbn:se:uu: diva-233144 .[46] T. Fredlund, C. Linder, J. Airey, and A. Linder, Unpack-ing physics representations: Towards an appreciation ofdisciplinary affordance, Phys. Rev. ST - Phys. Educ. Res. (2014).[47] T. Fredlund, J. Airey, and C. Linder, Exploring therole of physics representations: an illustrative examplefrom students sharing knowledge about refraction, Eur.J. Phys. , 657 (2012).[48] C. Hatcher, C. Wallace, E. Prather, J. Kamenetzky,and T. Chambers, New interferometry Lecture-Tutorialfacilitates high learning gain in Astro 101 (AAPT,2018) https://aapt.org/Conferences/sm2018/upload/SM18Program_Final_A.pdf .[49] MODE, Glossary of multimodal terms, website (2012), https://multimodalityglossary.wordpress.com , Re-trieved 25 April 2019.[50] T. van Leeuwen, Introducing Social Semiotics (Rout-ledge, London, UK, 2005).[51] C. Jewitt, ed.,
The Routledge Handbook of MultimodalAnalysis , 2nd ed. (Routledge, London, UK, 2009).[52] J. J. Gibson, The theory of affordances, in
The Ecolog-ical Approach to Visual Perception (Houghton Mifflin,Boston, MA, 1979) Chap. 8.[53] C. Linder, Disciplinary discourse, representation, and ap-presentation in the teaching and learning of science, Eur.J. Sci. Math. Educ. , 43 (2013).[54] https://astronomy101.jpl.nasa.gov/workshops .[55] .[56] J. Airey and C. Linder, A disciplinary discourse perspec-tive on university science learning: Achieving fluency ina critical constellation of modes, J. Res. Sci. Teach. ,27 (2009).[57] L. W. Anderson, D. R. Krathwohl, and B. S. Bloom, A Taxonomy for Learning, Teaching, and Assessing: ARevision of Bloom’s Taxonomy of Educational Objectives (Longman, New York, NY, 2001).[58] M. D. Engelhart, E. J. Furst, W. H. Hill, and D. R.Krathwohl,
Taxonomy of Educational Objectives: TheClassification of Educational Goals , edited by B. S.Bloom, Cognitive Domain, Vol. 1 (Longmans, London,UK, 1956).[59] J. Airey and U. Eriksson, Unpacking the Hertzsprung- Russell diagram: A social semiotic analysis of the disci-plinary and pedagogical affordances of a central resourcein astronomy, Designs for Learning , 99 (2019).[60] U. Eriksson, M. Rosberg, and A. Redfors, Disciplinarydiscernment from Hertzsprung-Russell diagrams, in Syn-opsis Book of the Proceedings of the 12th Nordic ResearchSymposium on Science Education , edited by A. P´alsd´ottir(Trondheim, Norway, 2017) p. 59.[61] E. Brogt,
Pedagogical and curricular thinking of profes-sional astronomers teaching the Hertzsprung-Russell di-agram in introductory astronomy courses for non-sciencemajors , Ph.D. dissertation, University of Arizona (2009).[62] J. Airey, The disciplinary literacy discussion matrix: Aheuristic tool for initiating collaboration in higher edu-cation, Across the Disciplines (2011).[63] A. Linder, J. Airey, N. Mayaba, and P. Webb, Fosteringdisciplinary literacy? South African physics lecturers’educational responses to their students’ lack of represen-tational competence, African J. Res. Math. Sci. Tech.Educ. (2014).[64] P. B. Kohl and N. Finkelstein, Understanding and pro-moting effective use of representations in physics learn-ing, in Multiple Representations in Physics Education ,Models and Modeling in Science Education, Vol. 10,edited by D. F. Treagust, R. Duit, and H. E. Fischer(Springer, Cham, Switzerland, 2017) Chap. 11, p. 231.[65] U. Eriksson, C. Linder, J. Airey, and A. Redfors, Intro-ducing the anatomy of disciplinary discernment: an ex-ample from astronomy, Eur. J. Sci. Math. Educ. , 167(2014).[66] G. Kress and T. van Leeuwen, Multimodal Discourse:The Modes and Media of Contemporary Communication ,Hodder Arnold Publication (Bloomsbury Academic, Lon-don, UK, 2001).[67] L. C. McDermott, A view from physics, in
Toward A Sci-entific Practice of Science Education , edited by M. Gard-ner, J. G. Greeno, F. Reif, A. H. Schoenfeld, A. diSessa,and E. Stage (Routledge, New York, NY, 1990) Chap. 1,p. 3.[68] R. Tytler, V. Prain, G. Aranda, J. Ferguson, andR. Gorur, Drawing to reason and learn in science, J. Res.Sci. Teach.