All aboard! Challenges and successes in professional development for physics lab TAs
AAll aboard! Challenges and successes in professional development for physics lab TAs
Danny Doucette, Russell Clark, and Chandralekha Singh
Department of Physics and Astronomy, University of Pittsburgh, Pittsburgh, PA, USA 15260
At large research universities in the USA, introductory physics labs are often run by graduate student teachingassistants (TAs). Thus, efforts to reform introductory labs should address the need for effective and relevant TAprofessional development. We developed and implemented a research-based professional development programthat focuses on preparing TAs to effectively support inquiry-based learning in the lab. We identify positiveeffects by examining three possible ways in which the professional development might have impacted TAsand their work. First, we examine lab TAs’ written reflections to understand the effect of the program onTAs’ ways of thinking about student learning. Second, we observe and categorize TA-student interactions inthe lab in order to investigate whether TA behaviors are changing after the professional development. Third,we examine students’ attitudes toward experimental science and present one example case in which students’attitudes improve for those TAs who ‘buy in’ to the professional development. Our results suggest lab TAprofessional development may have a tangible positive impact on TA performance and student learning. a r X i v : . [ phy s i c s . e d - ph ] N ov . INTRODUCTION AND FRAMEWORK At large research universities where introductory physicslabs are often taught by graduate student teaching assistants(TAs), these TAs may not receive adequate professional de-velopment for their work [1–10]. We are seeking to under-stand and develop a model for how TAs can most effectivelyhelp support student learning in physics labs, and what typeof professional development can most effectively help TAs fa-cilitate lab activities so that all students can learn. Our goal isto establish a lab TA professional development program thatteaches TAs how students learn in an experimental physicssetting, empowers lab TAs to improve their instructional prac-tice, and ultimately produces long-overdue learning and atti-tudinal outcomes [11–16] for all students in an inclusive andequitable learning environment.Past work has identified several important elements of ef-fective TA professional development [17], including adaptinggood ideas to one’s local context [18], establishing a purpose-ful community of practice [19], and focusing on the develop-ment of the TA’s beliefs and identity as an educator [20, 21].Another key issue is ‘buy-in’: TAs who do not believe in thevalue of the learning activity will tend to implement it withlow fidelity [22], which generally negatively impacts studentlearning [23]. Achieving buy-in is a complex effort that de-pends on the context of the professional development and avariety of social cues that, when effective, work together tohelp TAs come to value the planned learning activities [24].We adopt the cognitive apprenticeship model [25] as a the-oretical framework for understanding both the evolution ofTA learning in our professional development program andthe nature of student learning in the labs. In this view, werecognize that learning requires three stages: modeling, scaf-folding and coaching, and weaning. First, TAs in our train-ing program need to witness explicit modeling of the de-sired outcomes. Second, learning requires careful scaffold-ing that supports evidence-based active engagement, and soTAs should be coached and provided guidance and support inlearning how to provide this type of assistance. And third, thisscaffolding and support should be gradually removed, givingTAs opportunities to practice independently. Thus, we un-derstand TA professional development to be effective if ourTAs learn about and employ effective strategies for support-ing student learning, and if our students demonstrate elevatededucational outcomes as a result of their TA’s support.
II. TA PROFESSIONAL DEVELOPMENT
At our large research university, approximately 33 intro-ductory physics lab sessions are run every year. In each‘cookbook’-style lab, up to 24 students work at lab benches,typically in pairs. Labs meet for 3 hours, once per week, forone semester. Although assignments vary, typically 5 to 12TAs are assigned to teach the introductory labs in any givensemester. For many, it is their first time teaching a lab course. Some professional development is already provided forgraduate student TAs in the physics department at our uni-versity. However, all of this professional development as-sumes that the TAs will be small-class lecturers or recitationleaders, rather than lab instructors. This professional devel-opment includes a day-long workshop that focuses on TAs’formal responsibilities as employees, a 3-hour workshop de-signed to give them strategies to effectively lead recitations,and a one-credit course that teaches about effective pedagogyand affords practice with recitation-style work.Thus, while graduate student TAs receive a variety of in-structional professional development at our university, therewas nothing designed specifically for lab TAs. This was aconcern because some of the skills, attitudes, and approachesthat are required for TAs to be effective in lab settings arenot the same as those needed in recitations. Additionally, af-ter their first semester in graduate school, most TAs are un-likely to receive any formal professional development. Torectify these deficiencies, we designed a new professional de-velopment program for lab TAs that started in the fall 2018semester and was replicated in the spring 2019 and sum-mer 2019 semesters. These sessions involved students using‘cookbook’-style labs that are, in general, not very effectivein promoting student learning [26]. Nonetheless, while wetransition to an inquiry-based curriculum, we wanted to in-vestigate the impact of professional development on lab ef-fectiveness given this constrained setting.In our program [27], lab TAs meet weekly on Friday after-noons, including the Friday before the first week of classes,to prepare for the forthcoming week’s lab, and to learn andpractice lab-relevant pedagogy. The meetings wrap up mid-semester in order to reinforce the idea that the professionaldevelopment program is a scaffold from which the TAs canbe weaned, in line with our cognitive apprenticeship model,and that we expect TAs to continue developing as educatorsbeyond our professional development program. The programwas developed via extensive discussions and iterations be-tween the three authors, all experienced physics educators.We identified early on that motivation would be key tomaking this professional development program effective.Following the interest framework of Hidi and Renninger [28],we developed learning activities for TAs that would triggersituational interest in their work as TAs, reinforce that inter-est through meaningful social reflections, and consequentlyestablish sustained individual interest in becoming effectiveinstructors in their labs and beyond.Situational interest reinforcement happens at the start ofour weekly meetings, when all the TAs share a student inter-action from the past week they found surprising, concerning,or encouraging. These reflections provoke cross-discussionin which TAs celebrate their progress as educators, reaffirmshared commitments to helping students learn meaningfully,or brainstorm approaches to uncommon problems. These dis-cussions are moderated by the training leader, using standardmethods for establishing and maintaining positive interactiv-ity [29]. At all times, we focus on giving TAs opportunitieso speak out in order to develop their confidence as educators.Most of the lab TA meeting is dedicated to one or two rel-evant learning activities, which are designed to promote TAs’individual interest as educators. The learning activities areintended to help TAs directly improve their skills at workingwith students, better understand the nature of student learn-ing [30], and develop both proficiency with the apparatus andincreased levels of motivation to support students.One learning activity we employed is role-playing student-TA interactions around key points in the lab. For this, thetrainer sets up ‘sabotaged’ experiments in which the appa-ratus is miscalibrated, the analysis is incorrectly done, or asimilar common issue. The scaffolding in this experiment al-lows TAs to practice interactions that support evidence-basedactive learning. After role-playing through an interaction, theTAs and trainer debrief and move to another experiment. Twoother approaches we used occasionally are demonstrations,which serve as models for TAs to replicate, and conductingcarefully-moderated whole-group discussions. Some samplelearning activities are listed in Table I.
TABLE I. Sample of learning activities in the lab TA professionaldevelopment program.Activity Type WeeksIcebreaker demonstration firstReflections group discussion all’Sabotaged’ experiments role-playing mostEquitable learning environments group discussion thirdTeaching about the nature of science group discussion fifth
III. METHODOLOGY AND RESULTS
We assess the impact of our lab TA professional develop-ment program by analyzing three different examples of waysin which the training program might have impacted TAs orstudents. For clarity, the methods and results of each of thesethree examples are presented together. We begin by askingwhether the program changes how TAs think about their workin supporting student learning. Next, we investigate the be-havior of TAs as they work in their lab sections to under-stand if the professional development has changed the natureof their interactions with students. Last, we ask whether theprogram has a ‘second-order’ effect by improving students’learning outcomes.
A. TAs’ attitudes toward student learning.
To assessthe direct effect of the professional development program, labTAs were asked to write short responses to variations on thequestion, “How will you help students have effective learningexperiences?” at the start and end of the program. In total, 13responses were collected at the start, and 11 at the end, of thefall and spring programs.The responses show a marked transformation in how theTAs viewed their role in helping students to learn. At the beginning of the professional development program, most ofthe TAs described their role in terms of a traditional transmis-sion model of education [31], as these representative excerptsillustrate:
I will explain to them which equipment corresponds towhich concept. Then they can build connections of thephysics concepts to the lab.[I will] add in some physical explanation into thedemo at the start of lab... a feeling on physics willbuild up subconsciously after they leave the lab.
The use of the verb ‘explain’ is abundant in these earlyresponses, as the TAs view themselves as either telling stu-dents about physics concepts or clarifying the procedure forthe lab-work. Other responses emphasize the importance ofgood lecture structure, clear expectations for lab report for-matting, and creating a “relaxing” environment for the stu-dents. Given that most of the TAs have experienced a tradi-tional, transmission-based style of education – and are contin-uing to experience that model as graduate students – it is notsurprising that they rely on the transmission model of learn-ing to frame their work. Likewise, it makes sense that the TAsview their role as helping to make the lab easier for students,as that is likely how they viewed their own TAs during theirrecent undergraduate experiences.By the end of the the professional development program,however, the emphasis shifts and most of the TAs’ responsescelebrate students figuring things out on their own and withtheir lab partners, as seen in these typical quotations:
I would encourage a student... to collaborate withpeers, ask themselves more rigorous questions, etc.I asked her to think of the problem in a physical sense,instead of plugging in given equations. She actuallycame up with the right answer... by herself.
These final reflections indicate that after the professionaldevelopment the TAs have generally discarded the transmis-sion model in favor of a student-centered view of learning,which was one of our goals. ‘Explain’ is no longer used,and the responses tend to center the student’s experience in-stead of the TA’s work. Other responses emphasize the impor-tance of encouraging positive collaboration between studentsand explain techniques the TAs use for supporting studentmeaning-making without directly supplying information.It is not possible to conclusively determine what caused theshift in TAs’ views about learning. Was it the professionaldevelopment program? Their experiences as a TA in the lab?Something else? However, by comparing experienced TAswho have taught the lab before (in semesters before the pro-fessional development program) with newer TAs who havenever taught labs before, it seems likely that the common fac-tor – the lab TA professional development – played at leastsome role in their movement from a transmission to a con-structivist view of learning. . TA-Student Interactions.
Since one goal of the lab TAprofessional development is to get TAs to help students frameand answer their own questions, rather than just offering ad-vice and explanations, we hypothesized that there would bea change in the nature of student-TA interactions after intro-ducing the program. To measure the extent of these changes,we used the Real-time Instructor Observing Tool (RIOT) [32]to categorize these interactions for the same group of TAs.The RIOT allows an observer to continuously categorizethe types of activities undertaken by an instructor, in this casethe lab TA. When the instructor switches from one type ofactivity to another, the observer records the nature of the newactivity along with a timestamp. Other than infrequent occa-sions when the TA would be briefly checking personal notesor be out of the room, the seven categories shown in Fig. 1capture the full breadth of TA activities during the lab ses-sions that we observed. The data presented in Fig. 1 depictsinteractions for 13 TAs over 99 hours of instruction. We choseto observe during weeks 3 and 6 of our 13-week semester be-cause the lab interactions should have reached a ‘steady state’and because the lab-work for those weeks was typical, not ex-cessively intricate, and didn’t require that the lights be turnedoff. We also observed during week 11 for the TAs that re-ceived the professional development to explore if the effectof the training diminished after the meetings wrapped up inweek 6. Complete definitions and examples of the seven typesof interactions we observed are available in Ref. [32].We adopted slightly different labels for some categories forclarity, but use the same definitions and meanings for theseinteraction types. For example, we use ‘Actively Listening’(to question) to emphasize that the TA is engaging with non-verbal listening cues, unlike in the case of ‘Watching’. Like-wise, ‘Explaining Content’ and ‘Clarifying Instructions’ areforms of "talking at students" [32]. We exclude several inter-
FIG. 1. RIOT data from a semester before the introduction of alab TA professional development program compared with data fromthe fall 2018 semester, when the program was implemented. Thevertical axis shows the fraction of time spent in each of 7 differenttypes of interaction, averaged over the 6 and 7 TAs observed duringthat semester. action types that are not relevant to our labs and the relativelyrare cases when the TA is not interacting with students.Two specific examples of these categories will be relevantto our analysis. ‘Open Dialogue’ is an interaction in whicha student is contributing more than half the words (or ideas)to a conversation and is actively developing an understandingof physics or lab ideas, while the TA plays a supporting roleby asking prompting questions or helping students to frametheir ideas, as opposed to ‘Closed Dialogue’ which is TA-dominated. ‘Actively Listening’ means that the TA is neara group of students and showing an active interest in theirdiscussion through non-verbal cues such as establishing eyecontact, body positioning, or gestures, but isn’t participatingin the conversation as a contributor.Two differences stand out in Fig. 1. First, the amount oftime that TAs devote to ‘Open Dialogue’ is larger for the TAswho have completed the professional development program,and it seems to increase while the TAs are engaged in theprogram. This behavior aligns with our goals and with the re-sponses we discussed in section IIIA: the TAs are more likelyto let students lead their conversations, prompting rather thantelling, and helping the students to formulate and answer theirown questions.Second, the TAs who have taken the professional devel-opment program devote approximately twice as much time to‘Active Listening’ as those who haven’t taken the program. Inpractice, this manifests as the TAs being more willing to en-gage with students and offer non-verbal support and attentionas the students complete their lab-work. In our observations,this increased level of ‘Active Listening’ appears to be alignedwith TAs being increasingly comfortable interacting with stu-dents when they can position themselves as guides rather thanfeeling compelled to adopt the role of an all-knowing expert.In other words, the TAs were comfortable simply being thereand talking with the students, and didn’t feel the need to adopta ‘hide or provide’ behavior, in which they only approachedstudent groups if they felt they had some information to share.Teaching is certainly a complex endeavor, and no one in-teraction type is necessarily better than any other in all cir-cumstances. Good educators typically use a combination ofmany types of interaction [33]. Overall, though, we find someevidence that the balance of interaction types after the pro-fessional development program is better-aligned to the goalof supporting student-led active learning for TAs who havetaken the program.
C. Student Outcomes.
While both TA attitudes and inter-actions show improvements, the fundamental goal in our newTA professional development program is to improve studentlearning in the labs. Thus, in order to ascertain the impactof our lab TA program on student learning, we need to com-pare students whose TAs did deliver the learning support wedesigned in our professional develop program with those stu-dents whose TAs did not deliver this support. Here we discussit in relation to the lab TA professional development modulerelated to the nature of science (NoS).The nature of science is a set of beliefs about the episte-ology of science, i.e.: principles that we might identify asfundamental characteristics of Western scientific work. Theseinclude such ideas as ‘theories require evidence’, ‘sciencemakes predictions’, and ‘scientists seek to avoid bias’ [34].We observed that for the fall 2018 implementation, our pro-fessional development module for the nature of science hadan abnormally low level of engagement from our TAs. Dur-ing the program, most of the TAs merely went through themotions, and during lab observations later in the semesterwe saw no evidence that they used the proposed strategiesin their labs. However, two TAs clearly and unambiguouslybucked the trend: they engaged in a lengthy discussion aboutthe value of explicit instruction on the nature of science thatwent beyond the training session, and we observed them us-ing our strategies in their lab sessions on multiple occasions.Thus, by comparing the 74 students who received this NoStreatment from their TA to the 207 who did not, we estimatethe impact of having a TA engaged with NoS instruction.As a dependent variable, we rely on a categorization ofE-CLASS items [35] that identified a cluster of 6 items asrelevant to NoS in our context [36]. E-CLASS scores indicatethe degree to which a student agrees with expert-like attitudestoward 30 items related to experimental science. As shown inFig. 2, when we consider the ‘Other TAs’ that did not engagewith the value of teaching NoS, the average change in scorefor the 6 NoS-related items is more negative than for eachof the 24 other E-CLASS items, indicating that our studentstypically regress more on the nature of science items than onthe other items in E-CLASS. And while the decrease in oursetting is somewhat more negative than in the national samplefrom [35], a similar pattern appears, suggesting that these 6NoS-related items are particularly vulnerable.For the TAs who have adopted NoS instruction, we observea decrease on the 24 non-NoS items that is comparable to the
FIG. 2. Change in E-CLASS item scores between pre- and post-instruction assessments, averaged over the 24 and 6 E-CLASS itemsidentified as either not, or belonging to, a cluster related to the natureof science. TAs who were active in discussing NoS during the pro-gram and who explicitly talked about it in their labs are compared tothose who did not. National comparison sample from [35]. decrease seen by the other TAs, but a sizable increase on the 6 NoS-related items. Using a mixed-effect ANCOVA modelcontrolling for overall pre-test E-CLASS score, and with sat-isfactory normality and homogeneity of variance, we find astatistically significant difference between our two TA groupsand the two item categories ( F (1 , . , p < . ).We note that the lab-work in this study employed a‘cookbook’-style approach that will be replaced with anevidence-based active learning approach [26] starting in fall2019. We expect that overall E-CLASS score improvementswill require both that TAs are effectively trained (and adoptthe strategies learned in this professional development) andthat inquiry-based learning activities are in place.We can draw two conclusions from this result. First, stu-dents attitudes about experimental physics, as measured by atleast some of the E-CLASS items, can be influenced by theinteractions and learning that are offered by graduate studentTAs in the introductory lab. Second, TA buy-in for particularinstructional strategies is essential in order for this to happen. IV. DISCUSSION AND IMPLICATIONS
At three levels of analysis, we find evidence that an ef-fective lab TA professional development program has the po-tential to positively impact the work undertaken by TAs andthe learning of their students. After the professional devel-opment program, lab TAs demonstrated a shift in how theyviewed their role as instructors and how they thought aboutthe nature of student learning. It appears that the lab TAswho completed the program were also more likely to ‘walkthe walk,’ interacting with students in ways that are commen-surate with what we sought to teach them about supportingactive engagement learning. And when lab TAs took up thenew approach to teaching a topic, such as explicitly engagingin discussions about the nature of science, student attitudescorresponding to that topic became more expert-like.The nature of the work done by lab TAs, the activities in-volved in our program, and the improvements noted aboveact together to shine light on the importance of specializedprofessional development for TAs who are working in labsettings. While there is overlap with how we might trainTAs to support evidence-based active engagement learning,the need to develop mastery at troubleshooting apparatus andto address issues such as the nature of science means that labTAs should be receiving dedicated professional developmentin order to be effective in their work.As seen in the case of our nature of science module, someimprovement is still needed for our professional developmentprogram. TA ‘buy-in’ remains a vital issue, especially for theNoS module, and as we go forward we will continue workingto improve the social and structural factors that can drive TAbuy-in [24]. However, even with an imperfect professionaldevelopment program and ‘cookbook’-style labs, and throughthe inevitable noise of implementation, these results suggeststhat lab TA professional development has the potential to havea positive impact on TA performance and effectiveness.
1] J. A. Gilreath and T. F. Slater, Training graduate teaching as-sistants to be better undergraduate physics educators, PhysicsEducation , 200 (1994).[2] C. Singh, Categorization of problems to assess and improveproficiency as teachers and learners, American Journal ofPhysics , 73 (2009).[3] S.-Y. Lin, C. Henderson, W. Mamudi, C. Singh, andE. Yerushalmi, Teaching assistants’ beliefs regarding examplesolutions in introductory physics, Phys. Rev. ST Phys. Educ.Res. , 010120 (2013).[4] M. Good, E. Marshman, E. Yerushalmi, and C. Singh, Physicsteaching assistants’ views of different types of introductoryproblems: Challenge of perceiving the instructional benefits ofcontext-rich and multiple-choice problems, Phys. Rev. Phys.Educ. Res. , 020120 (2018).[5] E. Marshman, R. Sayer, C. Henderson, and C. Singh, Con-trasting grading approaches in introductory physics and quan-tum mechanics: The case of graduate teaching assistants, Phys.Rev. Phys. Educ. Res. , 010120 (2017).[6] A. Maries and C. Singh, Exploring one aspect of pedagogicalcontent knowledge of teaching assistants using the Test of Un-derstanding Graphs in Kinematics, Phys. Rev. ST Phys. Educ.Res. , 020120 (2013).[7] A. Maries and C. Singh, Teaching assistants’ performance atidentifying common introductory student difficulties in me-chanics revealed by the Force Concept Inventory, Phys. Rev.Phys. Educ. Res. , 010131 (2016).[8] N. I. Karim, A. Maries, and C. Singh, Exploring one aspect ofpedagogical content knowledge of teaching assistants using theConceptual Survey of Electricity and Magnetism, Phys. Rev.Phys. Educ. Res. , 010117 (2018).[9] M. Wilcox, C. C. Kasprzyk, and J. Chini, Observing teach-ing assistant differences in tutorials and inquiry-based labs, in Physics Education Research Conference 2015 (College Park,MD, 2015) pp. 371–374.[10] G. DeBeck, S. Settelmeyer, S. Li, and D. Demaree, TA beliefsin a SCALE-UP style classroom, AIP Conference Proceedings , 121 (2010).[11] C. Wieman and N. Holmes, Measuring the impact of an in-structional laboratory on the learning of introductory physics,American Journal of Physics , 972 (2015).[12] B. R. Wilcox and H. J. Lewandowski, A summary of research-based assessment of students’ beliefs about the nature of exper-imental physics, American Journal of Physics , 212 (2018).[13] B. M. Zwickl, T. Hirokawa, N. Finkelstein, and H. J.Lewandowski, Epistemology and expectations survey aboutexperimental physics: Development and initial results, Phys.Rev. ST Phys. Educ. Res. , 010120 (2014).[14] D. Hu and B. M. Zwickl, Examining students’ views aboutvalidity of experiments: From introductory to Ph.D. students,Phys. Rev. Phys. Educ. Res. , 010121 (2018).[15] K. N. Quinn, K. L. McGill, M. M. Kelley, E. M. Smith, andN. Holmes, Who does what now? How physics lab instruc-tion impacts student behaviors, in Physics Education ResearchConference 2018 (Washington, DC, 2018).[16] K. Koenig, K. E. Wood, L. J. Bortner, and L. Bao, Modifyingtraditional labs to target scientific reasoning, Journal of CollegeScience Teaching , 28 (2019).[17] C. Sandifer and E. Brewe, eds., Recruiting and Educating Fu- ture Physics Teachers: Case Studies and Effective Practices (Physics Teacher Education Coalition, 2015).[18] E. L. Jossem, Resource letter EPGA-1: The education ofphysics graduate assistants, American Journal of Physics ,502 (2000).[19] N. G. Holmes, M. S. Martinuk, J. Ives, and M. Warren, Teach-ing assistant professional development by and for TAs, ThePhysics Teacher , 218 (2013).[20] R. M. Goertzen, R. E. Scherr, and A. Elby, Tutorial teachingassistants in the classroom: Similar teaching behaviors are sup-ported by varied beliefs about teaching and learning, Phys. Rev.ST Phys. Educ. Res. , 010105 (2010).[21] A. L. Gretton, T. Bridges, and J. M. Fraser, Transformingphysics educator identities: TAs help TAs become teachingprofessionals, American Journal of Physics , 381 (2017).[22] M. Wilcox, Y. Yang, and J. J. Chini, Quicker method for as-sessing influences on teaching assistant buy-in and practicesin reformed courses, Phys. Rev. Phys. Educ. Res. , 020123(2016).[23] K. M. Koenig, R. J. Endorf, and G. A. Braun, Effectiveness ofdifferent tutorial recitation teaching methods and its implica-tions for TA training, Phys. Rev. ST Phys. Educ. Res. , 010104(2007).[24] R. M. Goertzen, R. E. Scherr, and A. Elby, Accounting for tu-torial teaching assistants’ buy-in to reform instruction, Phys.Rev. ST Phys. Educ. Res. , 020109 (2009).[25] A. Collins, J. S. Brown, and A. Holum, Cognitive appren-ticeship: Making thinking visible, American Educator , 6(1991).[26] D. R. Sokoloff, P. W. Laws, and R. K. Thornton, RealTimePhysics: Active learning labs transforming the introductorylaboratory, Eur. J. Phys. , S83 (2007).[27] D. Doucette, Lab TA TRAINING (2019).[28] S. Hidi and K. A. Renninger, The four-phase model of interestdevelopment, Educational psychologist , 111 (2006).[29] E. G. Cohen and R. A. Lotan, Designing Groupwork: Strate-gies for the Heterogeneous Classroom , 3rd ed. (Teachers Col-lege Press, 2014).[30] J. Piaget,
The Origins of Intelligence in Children (InternationalUniversities Press, Inc., New York, 1952).[31] J. Dewey,