Factors Contributing to Attitudinal Gains in Introductory Astronomy Courses
Adam S. Trotter, Daniel E. Reichart, Aaron P. LaCluyzé, Rachel Freed
RRobotic Telescopes, Student Research and Education (RTSRE) ProceedingsConference Proceedings, San Diego, California, USA, Jul 23-27, 2018Editor, E., Editor E., Eds. Vol. 1, No. 1, (2018)ISSN XXXX-XXXX (online) / doi : doidoidoidoi / CC BY-NC-ND licensePeer Reviewed Article. rtsre.org/ojs
Factors Contributing to Attitudinal Gains inIntroductory Astronomy Courses
Adam S. Trotter , Daniel E. Reichart *, Aaron P. LaCluyz ´e & Rachel Freed Abstract
Most students do not enroll in introductory astronomy as part of their major; for many, it is thelast science course they will ever take. Thus, it has great potential to shape students’ attitudestoward STEM fields for the rest of their life. We therefore argue that it is less important,when assessing the effectiveness of introductory astronomy courses, to explore traditionalcurricular learning gains than to explore the effects that various course components have onthis attitude. We describe the results of our analysis of end-of-semester surveys returnedby a total of 749 students in 2014-2015, at 10 institutions that employed at least part of theintroductory astronomy lecture and lab curriculum we first implemented at the University ofNorth Carolina at Chapel Hill in 2009. Surveys were designed to measure each student’sattitude, and to probe the correlation of attitude with their utilization of, and satisfaction with,various course components, along with other measures of their academic background andtheir self-assessed performance in the course. We find that students’ attitudes are significantlypositively correlated with the grade they expect to receive, and with their rating of the course’soverall effectiveness. To a lesser degree, we find that students’ attitudes are positivelycorrelated with their mathematical background, with whether they intend to major or pursuea career in STEM, and with their rating of the effectiveness of the instructor. We find thatstudents’ attitudes are negatively correlated with the amount of work they perceived the courseto involve, and, surprisingly, with the size and reputation of their home institution. We also findthat, for the subsets of students who were exposed to them, students’ attitudes are positivelycorrelated with their perception of the helpfulness of the lecture component of the course, andof telescope-based labs that utilized UNC-CH’s Skynet Robotic Telescope Network. Department of Physics & Astronomy, University of North Carolina at Chapel Hill Department of Physics, Central Michigan University Sonoma State University * Corresponding author : [email protected]
Received ?; revised ?; accepted ?
Background
Increasing interest and participation in STEM fieldshas been a major goal at the national level for many years, as the United States struggles to keep up glob-ally with scientific and engineering pursuits (AAAS1990; NRC 2007) while simultaneously decliningin global rankings of science education (Kastberget al., 2016; Provasnik et al., 2016). Meanwhile,little progress has been made on the front of in- a r X i v : . [ phy s i c s . e d - ph ] O c t actors Contributing to Attitudinal Gains in Introductory Astronomy Courses — 2/15 creasing the number and quality of highly trainedscientists and engineers in this country or of pro-ducing scientifically literate citizens (Alper, 2016).Concurrently, as a potential solution to theseissues, there have been dozens of attempts over thepast two and a half decades to provide telescopeaccess for education (Gomez and Fitzgerald, 2017),often under the presumption made by project per-sonnel that, if the telescope is available and acces-sible, educators and students will inevitably use itfor learning (Slater et al., 2014). In contrast, manyof the programs developed over the past 25 yearshave not succeeded in their goals, with several evenfailing to launch after publication of their intendedexistence.Astronomy is often referred to as the “GatewayScience” (NRC 2010), with an estimated 240,000students taking introductory astronomy or “IntroAstro” in the US, according to a 2012 survey by theAmerican Institute of Physics (Mulvey and Nichol-son, 2014). It is often noted that Intro Astro is thelast science class many students will ever take andis thus poised in an important position to promotescientific understanding and literacy for citizens asthey leave the academic world and enter the work-force.Skynet (Reichart et al., 2005; Martin et al., 2018)has in large part solved the decades-old struggle toprovide telescope learning experiences for students,particularly at large enrollment scales. Since itsinception in 2004, Skynet has grown to one of thelargest robotic telescope networks in the world, withnearly 30 optical telescopes ranging in size from14 to 40 inches in diameter, a 20-meter radio tele-scope, and with several more telescopes soon to beadded. These telescopes are all controlled through aweb-based portal used by professional astronomersand students alike. Approximately 50,000 students,from middle school through to senior undergradu-ate, have used Skynet to date.A few researchers have pointed out that thevalue of remote telescope use in settings with largeenrollments is unclear due to the current lack ofrisk-benefit analysis in the literature (e.g., Slater,2018). While much of the focus on astro101 hasbeen on learning gains (e.g., Prather et al., 2009; Schlingman et al., 2012; Williamson et al., 2016),much less attention has been paid to attitudes to-wards science, and astronomy in particular, in IntroAstro. This is due in part to a lack, until recently,of reliable and validated attitude assessment toolsfor astronomy (Bartlett et al., 2018), but also to thedifficulties of curriculum design connecting expen-sive telescope resources to large enrollments (Slater,2007).In this paper, we explore the effects on students’attitudes towards astronomy (Zeilik et al., 1999),based on responses to end-of-semester surveys of749 Intro Astro students at 10 institutions between2014 and 2015. These students undertook, in wholeor in part, an introductory astronomy lecture andlab curriculum first implemented at University ofNorth Carolina at Chapel Hill (UNC-CH). This isthe first known exploration of students’ attitudescombining robotic telescopes and large enrollmentIntro Astro courses. Project Intro Astro
In 2009, we introduced a new introductory astron-omy lecture and lab curriculum at UNC-CH. Atmost universities, introductory astronomy is taughtas a two-semester sequence, but at UNC-CH it hadalways been taught in a single semester, which forthe students was akin to drinking from a fire hose. In2009, we split the old course into two new courses:
ASTR 101: The Solar System
Celestial motions of Earth, the sun, the moon, andthe planets; the nature of light; ground and space-based telescopes; comparative planetology; Earthand the moon; terrestrial and gas planets and theirmoons; dwarf planets, asteroids, and comets; plan-etary system formation; extrasolar planets; thesearch for extraterrestrial intelligence (SETI).
ASTR 102: Stars, Galaxies, and Cosmology
The sun; stellar observables; star birth, evolution,and death; novae and supernovae; white dwarfs,neutron stars, and black holes; Einstein’s theory ofrelativity; the Milky Way galaxy; normal galaxies,active galaxies, and quasars; dark matter and dark actors Contributing to Attitudinal Gains in Introductory Astronomy Courses — 3/15 energy; cosmology; the early universe.
This created time to explore the material more thor-oughly and more enjoyably, to introduce new mate-rial (e.g., a week of relativity in ASTR 102), and tointroduce in-class demonstrations. Altogether, wedeveloped over 50 in-class demonstrations, whichwe found to be particularly effective at convey-ing otherwise difficult concepts and at generatingdiscussion, even in the largest classes. We havenow taught these courses successfully to as fewas approximately 10 students and to as many asapproximately 400 students, where so far successhas been measured by end-of-course evaluationsthat are among the highest in our department, aswell as by rapidly growing introductory astronomyenrollment.The centerpiece of our new introductory astron-omy curriculum has been the modernization of ourintroductory astronomy laboratory course, ASTR101L. For decades, ASTR 101L made use of the the-ater of the Morehead Planetarium and Science Cen-ter on the UNC-CH campus, for five day labs andsmall telescopes on our campus observing decksfor five night labs. However, both sets of labs wereproblematic. Measurements within the planetariumchamber suffered from often greater than 100% er-ror depending on where you sat. The visual observ-ing labs suffered from Chapel Hill’s weather, brightskies, proximity to athletic field lights ruining darkadaptation, inability to see the north star, which isnecessary to properly align the telescopes, outdatedand difficult to use telescopes, and a weak set ofbackup labs. Finally, neither set of labs stronglyreinforced the lecture curriculum. Feedback fromthese labs was generally negative.We developed a series of eight new labs, twoof which are two-week labs, and six of which uti-lize UNC-CH’s Skynet Robotic Telescope Network.After an introductory lab in which students learnhow to use Skynet, the labs strongly reinforce boththe new ASTR 101/102 lecture curriculum and oneanother. Among other things, students use Skynetto collect their own data to distinguish between geo-centric and heliocentric models using the phase andangular size of Venus, to measure the mass of a Jovian planet using the orbit of one of its moonsand Kepler’s third law, to measure the distance toan asteroid using parallax measured simultaneouslyby Skynet telescopes in different hemispheres, andto measure the distance to a globular cluster usingan RR Lyrae star as a standard candle. More is donewith archival data that takes longer than a semesterto collect (e.g., Cepheid stars, Type Ia supernovae,etc.)In addition to the lecture, demo and lab curric-ula, we developed a set of multiple-choice home-work problems and detailed solutions for both Astro101 and Astro 102 within the WebAssign frame-work. Also, in an effort to explore the effective-ness of “flipping the classroom”, we developed aset of in-class polling questions, and an interactivee-polling tool that allows the instructor to displayand analyze numerical responses in real-time. Wealso provided all students free online access viaYouTube to a complete archive of videotaped Astro101 and, soon, Astro 102 lectures compiled fromprevious semesters.After implementing this curriculum at UNC-CHin 2009, lab enrollments increased over 150%, allintroductory astronomy enrollments increased over100% – now one in four UNC-CH students takeat least one of our courses – and astronomy-trackmajors and minors increased ≈ ≈ ≈
20 per year). Encouraged by this initial success,we soon began partnering with other regional in-stitutions to help them adopt and adapt those partsof the lecture course, in-class exercises and demos,homework, and labs that were compatible with theirbroader curricula and educational philosophies. Asof today, 14 institutions have adopted our curricu-lum in whole or in part, with a handful more sched-uled to join in the coming year. In this report, weanalyze student survey responses collected from10 schools, ranging from 2-year community col-leges to Research I universities, over 4 semesters in2014-2015.While we provided instructors at these partnerinstitutions access to our full sets of homework, lab,e-book, e-polling, video, and other curriculum re-sources, they were free to accept, reject or adapt anyelement to best suit their institutional needs and ed- actors Contributing to Attitudinal Gains in Introductory Astronomy Courses — 4/15 ucational goals. Table 1 summarizes the institutionsthat employed our curriculum in whole or in part,and whose students responded to the end-of-coursesurvey, during the period of 2014-2015. Table 2describes in greater detail the components of ourcurriculum that each instructor chose to implementin their section.
Survey Structure, and Definitionof Dependent and IndependentVariables
Near the end of each semester, students in participat-ing sections were provided with a link to a Qualtricssurvey about their experience in Introductory As-tronomy. For the four semesters analyzed in thisreport, we received an initial total of 827 completedsurveys. After eliminating incomplete or obviouslyfraudulent instances, we arrived at a final dataset of749 responses.The survey consists of 43 multiple-choice andshort-answer questions, some of which consist ofmultiple parts. The questions include basic de-mographic information and assessments of a stu-dent’s background and preparation for the course,but are primarily geared towards determining a stu-dent’s opinion of the course and their attitude to-wards specific course components and towards as-tronomy and science in general. Some questionsask students to rank their opinion of a course com-ponent, or their level of agreement with a state-ment, on a four- or five-step scale (quantitativequestions). A number of these quantitative surveyquestions consist of multiple sub-questions. A fewquestions are in yes/no format, or otherwise estab-lish whether or not a student engaged with partic-ular components of the course (binary questions).The full text of the survey can be downloaded at:https://tinyurl.com/introastroreportIn order to facilitate analysis, responses to allquestions were reassigned to a uniform numericalscale ranging from -1 to +1. For binary questions,this is as simple as assigning a “Yes” answer thevalue +1, and a “No” answer the value -1. For quan-titative questions, this required both renormalizingthe numerical range of the responses, and, in some cases, flipping the sign of the response to correct forwhether the question had a “positive” or “negative”attitudinal orientation.The responses to some multi-part quantitativequestions were averaged (after numerical range nor-malization and attitudinal orientation correction) toproduce a single numerical index for that question.An illustrative example is the astronomy/science“Attitude Index”, which serves as the single depen-dent variable in the analysis that follows. This Atti-tude Index is computed from the respondents’ an-swers to 33 questions that were designed to probetheir attitudes towards Astronomy and science ingeneral, after having taken Introductory Astronomyat their institution. Each question is in the formof a statement; students were instructed to indicatetheir level of agreement with each statement, from 1(strongly disagree) to 3 (neither agree nor disagree)to 5 (strongly agree). By design, some statementswere positively oriented (e.g., “I like astronomy”,“Scientific concepts are easy to understand”, “Scien-tific skills will make me more employable”), whilesome were negatively oriented (e.g., “Astronomyis irrelevant to my life”, “I felt insecure when Ihad to do astronomy homework”, “I find it difficultto understand scientific concepts”). Each responsewas converted to a numerical scale ranging from-1 (negative attitude) to +1 (positive attitude), tak-ing into account the orientation of each question,and the results were averaged over the 33 ques-tions, producing a single Attitude Index for eachstudent respondent. While the perceived orientationof certain of these statements may be qualitative,with different students seeing the same statementas either positive or negative, the majority are un-ambiguous. The orientations we assigned to theAttitude Index questions are presented in Table 3.In the analysis that follows, we explore the sta-tistical dependence of Attitude Index (dependentvariable) on a variety of other survey responses/indices(independent variables), using simultaneous multi-ple linear regression. After initially performinglinear regression with 16 independent variables,we iteratively removed those independent variablesthat were uncorrelated with Attitude Index at the p > .
05 level, refitting at each iteration. The re- actors Contributing to Attitudinal Gains in Introductory Astronomy Courses — 5/15Table 1.
Summary of institutional participation, by institution. Institution types: 1 = 2-year communitycollege; 2 = 4-year college or university; 3 = Research I university
Institution Type Semesters Sections Instructors Responses
Ashland Community& Technical College (ACTC) 2 2 3 1 12Francis Marion University (FMU) 2 1 1 1 6Fayetteville State University (FSU) 2 3 6 1 34Glenville State College (GSC) 2 1 1 1 5High Point University (HPU) 2 2 2 1 15North Carolina Agricultural & TechnicalState University (NCAT) 2 3 3 2 29North Carolina State University (NCSU) 3 2 2 1 6University of North Carolinaat Chapel Hill (UNC-CH) 3 4 11 2 427University of Virginia (UVa) 3 2 2 1 28Wake TechnicalCommunity College (WTCC) 1 4 10 5 187
Total=10 NA 4 41 16 749 sults are summarized in Table 4. We found the fol-lowing variables to exhibit significant correlation (in decreasing order of correlation coefficient): • Course Attitude Index (Q48 in original sur-vey; see Appendix): measures a student’s at-titude to the course as a whole, based on anaverage of responses to 10 statements, scaledto -1 = strongly disagree to +1 = stronglyagree.
Positively correlated. • Grade Index (Q17): what grade students ex-pected to receive in the course at the timethey took the survey. -1 = F, 0 = C, +1 = A.
Positively correlated. • Career Index (Q12): measures the degree towhich a student’s academic and career pathis oriented towards STEM in general, andastronomy & physics in particular. -1 = plan-ning a career in a non-STEM field; 0 = plan-ning a career in a STEM field; 1 = Planninga career in a STEM field, and majoring orminoring in astronomy or physics.
Positivelycorrelated. • Instructor Index (Q64): measures a student’sattitude towards the primary course instruc-tor, based on an average of responses to 11statements (Q64), scaled to -1 = strongly dis- agree to +1 = strongly agree.
Positively cor-related. • Math Index (Q8): measures a student’s aca-demic mathematics training background. Rangesfrom -1 = some algebra to +1 = beyond cal-culus.
Positively correlated. • Institution Index (see Table 1): measure ofthe type of institution the course was offeredat: -1 = 2yr college, 0 = 4yr college, +1 =research I university.
Negatively correlated. • Work Index (Q23): based on the response tothe statement “I worked harder than I thoughtI would in order to meet the instructor’s stan-dards or expectations.” -1 = strongly disagreeto +1 = strongly agree.
Negatively corre-lated.
The following independent variables were foundto exhibit no significant correlation with AttitudeIndex at the p < .
05 level: • Skynet Index (see Table 2): -1 = studentwas offered no Skynet-based labs; +1 = stu-dent was offered Skynet-based labs. • Lab Index (see Table 2): -1 = no lab com-ponent to course at all; +1 = some lab com-ponent to course. actors Contributing to Attitudinal Gains in Introductory Astronomy Courses — 6/15 • Online Index (see Table 2): -1 = traditionallecture course; +1 = online course. • Engagement Index (Q35): measures a stu-dent’s level of engagement with the course,based on an average of their responses to 6questions about how often they employed var-ious study habits (doing readings, completingassignments, engaging in classroom discus-sion, etc.). • Hours Index (Q15): the number of hoursthe student spent per week on course-relatedwork. Ranges from -1 = fewer than 3, to 0 =7-9 hours, to +1 = 12 or more hours. • Credits Index (Q13): how many credit hoursthe student was enrolled in while taking theintro astro course. Ranges from -1 = 6 orfewer credit hours to 0 = 7-9 credit hours to+1 = 19 or more credit hours. • Year Index (Q19): the academic year of thestudent. Ranges from -1 = first year to +1 =5th+ year. • Attendance Index (Q22): based on the ques-tion “It is possible to do well in this coursewithout attending class regularly”, ranges from-1 = strongly disagree to +1 = strongly agree. • UNC HW Index (see Table 2): measureof how many UNC-provided homework setswere assigned in the student’s section. Rangesfrom -1 = none to +1 = all of the 9 availablesets.
Baseline Model
As described above, we found that 7 of our indepen-dent variables were significantly correlated with theAttitude Index at the p < .
05 level; the results aresummarized in Table 4.We consider each of these variables in turn, indescending order of correlation coefficient:1.
Course Attitude Index:
It is not surprisingthat the Astronomy/Science Attitude Index’sstrongest and most significant correlation isthat with the student’s attitude towards andopinion about the course overall. The ques-tions that comprise the Course Attitude in-dex (Q48 in survey) focus on whether a stu- dent feels that the course and the work in-volved were effective in helping them learn,whether sufficient feedback was provided ona student’s progress, and whether the studentfound the course inspiring and challenging.As with all of these correlations, we mustspeculate on causal relationships with cau-tion. Does a positive experience in the coursecreate a positive attitude towards science, orare students who were predisposed to viewscience favorably more likely to appreciate acourse in introductory astronomy in the firstplace? It’s not possible to disentangle thesetwo with this analysis, but we can at leastinfer that the most impactful strategy for aninstitution to take, if its goal is to increasepositive attitudes towards science in general,is to foster positive attitudes towards the stu-dent experience of an introductory course it-self – its goals, pacing, feedback, and level ofintellectual challenge.2.
Grade Index:
It is also not surprising that astudent’s attitude towards science in general,after taking an introductory science course,would be correlated with the grade that theyexpect to receive. As with the previous in-dex, it is not possible to say whether this isjust correlation or causation. But by account-ing for these strongly correlated Grade andCourse Attitude Indices in the simultaneousmultiple linear regression analysis, we can atleast begin to unmask some of the subtler cor-relations that follow. We chose to explore theself-reported expected grade both because itis much easier, logistically and ethically, thanattempting to assign actual grades to ostensi-bly anonymous surveys, and because, when itcomes to attitudes, a student’s self-perceivedgrade at the time of the survey is more likelyto matter than what they actually end up get-ting.3.
Career Index:
After Course Attitude Index,the STEM Career Index is the most signifi-cantly correlated independent variable. Again,as with the previous variables, it is not possi-ble to say whether students who had already actors Contributing to Attitudinal Gains in Introductory Astronomy Courses — 7/15 decided to pursue STEM careers are predis-posed to have more positive attitudes towardsscience, or whether positive attitudes engen-dered by the course prompted some studentsto consider STEM careers for the first time.This is a case where giving the survey bothat the beginning and at the end of the coursewould be very helpful in interpreting the re-sults. It is worth noting that 370 out of 749,or nearly 50% of the total respondents in-dicated that they did not intend to pursueSTEM-related careers. As a group, thesenon-STEM students receive a less positiveimpact on science attitude than do their gen-eral STEM-major peers, who in turn are im-pacted less than those who specifically plancareers in astronomy or physics.4.
Instructor Index:
While it makes sense thatstudents who view their instructor positivelymight emerge from the course with a morepositive attitude towards science, it is inter-esting that the correlation, while positive, isboth relatively low and marginally significant.Also, as discussed in the following section,when we look only at the subsets of studentswho attended a lecture course, or who wereexposed to Skynet during the course, and in-clude their ratings of these components’ help-fulness as independent variables in the re-gression analysis, the correlation of AttitudeIndex with Instructor Index disappears. Themessage seems to be that instructor qualityhelps to shape attitudes towards science, butnot nearly as much as the perceived qualityof the course curriculum and experience as awhole5.
Math Index:
This is another correlation thatis unsurprisingly positive but surprisingly weak.Having a more extensive mathematical course-work background corresponds to more posi-tive science attitudes at the end of the course,but not by much. This would suggest that ourIntro Astro curriculum (which requires onlybasic algebra) is relatively equally accessibleand impactful to every student, regardless ofmathematical background. 6.
Institution Index:
There is a weak but sta-tistically significant negative correlation ofAttitude Index with the type of institutionthe course is offered at, with 2-year commu-nity colleges doing better, in general, than4-year colleges, and both doing better thanResearch I universities. This trend may re-flect a dependence on class size and instructoravailability, with the smaller, more personalenvironments typical of community collegeclassrooms serving better to instill positive at-titudes towards science than large auditorium-style lecture formats typical of major univer-sities.7.
Work Index:
Students who perceive the courseto require more work than average emergewith more negative attitudes towards science.This would suggest that one straightforwardway to boost science attitudes would be to re-duce the workload in introductory sciencecourses. It is important to keep in mind,however, that the very significantly positivelycorrelated Course Attitude Index is partiallya measure of how intellectually challengingand instructive the course is perceived to be.Make the course too easy, and you risk nega-tively impacting attitude towards the course,and so towards science in general.
Baseline Model + Helpfulness ofCourse Components
Students were asked to rate the Helpfulness Indexof various components of their introductory astron-omy courses, if present, which we scale from -1 =not helpful to +1 = extremely helpful (Q52). Thestudents were given the option to indicate that anycomponent was not applicable to their experience.The course components included: • Attending class lectures • Watching videos of lectures • Supplementary notes (e.g., e-book onWebAssign) • Homeworks • In-class exercises/polling (e.g., clickers, pollingcards, e-polling) actors Contributing to Attitudinal Gains in Introductory Astronomy Courses — 8/15 • Textbook • Out-of-class exercises • Office hours • Online discussion forum (e.g., Sakai or Black-board) • Skynet-based telescope labs • Other telescope labs (not part of UNC’s cur-riculum, but employed at some participatinginstitutions) • Non-telescope labs (e.g., The Earth and theSeasons, or Hubble’s Law)We found that the mere presence of any of theseindividual course components (included in the lin-ear regression analysis as binary independent vari-ables where -1 = not used in course and +1 = used incourse) did not significantly impact the students’ at-titudes about science. However, we did find that, fortwo components – attending in-class lectures, andSkynet-based labs – how helpful the students foundthese course components to be did matter. For theother course components, neither the existence ofthe component nor how helpful the students foundit to be impacted the Attitude Index.For each component, we analyzed the subsetof non-N/A respondents and performed multiplelinear regression on Attitude Index vs the sameset of independent variables described earlier, butincluding the Helpfulness Index of that component,again iteratively eliminating independent variablesfor which the correlation significance was low. TheHelpfulness Indices range from -1 = not helpfulto my learning to +1 = extremely helpful to mylearning.Out of the 749 total student responses to thesurvey, 712 students attended in-classroom lectures.The distribution of Lecture Helpfulness Index rat-ings in this subsample is plotted in Figure 1,Theresults of multiple linear regression on this subsetare presented in Table 5. We find a weak but statis-tically significant positive correlation between theLecture Helpfulness Index and the Attitude Index:the more helpful a student finds attending class tobe, the more positive their attitude towards scienceat the end of the course. However, note that theInstructor Attitude Index, which exhibited weak but significant positive correlation in the earlier base-line analysis (Table 4), is no longer significantlycorrelated in this subset ( p = . Figure 1.
Distribution of our sample of 712students who rated the helpfulness of in-classlectures. Those who found the lectures helpful leftthe course with more positive attitudes aboutastronomy and STEM fields in general. Other thanour Skynet-based labs, no other course componenthad a similar effect. The Helpfulness Indices rangefrom -1 = not helpful to my learning to +1 =extremely helpful to my learning.Out of the 749 total student responses to thesurvey, 508 students participated in at least oneSkynet-based lab during the semester. The distri-bution of Skynet Helpfulness Index ratings in thissubsample is plotted in Figure 2, and the resultsof multiple linear regression on this subset are pre-sented in Table 6. As with Lecture Helpfulness, wefind a weak but statistically significant correlationbetween Science Attitude Index and Skynet Help-fulness Index for the subset of students who wereexposed to Skynet-based labs. Those students whofound the labs helpful, left the course with a morepositive attitude towards science overall. Note that actors Contributing to Attitudinal Gains in Introductory Astronomy Courses — 9/15 we performed this same analysis for the helpfulnessof other lab components, including indoor labs andnon-Skynet-based telescope labs, and found no sig-nificant impact. Adding at least one Skynet-basedlab (and working to present it and integrate it in away that is perceived as helpful to students’ under-standing of the course material) appears to be oneway to significantly boost attitudes towards sciencefor Intro Astro students.
Figure 2.
Distribution of our sample of 508students who rated the helpfulness of Skynet-basedtelescope labs. Those who rated these labs ashelpful left the course with more positive attitudesabout astronomy and STEM fields in general.Other than in-class lectures, no other coursecomponent had a similar effect. The HelpfulnessIndices range from -1 = not helpful to my learningto +1 = extremely helpful to my learning.
Conclusion
The majority of students do not enroll in Introduc-tory Astronomy as part of their major; for many, itis the last science course they will ever take, andhas the potential to shape their attitudes towardsSTEM fields for the rest of their life. It is less im-portant, therefore, when assessing the effectivenessof Intro Astro courses to explore traditional curric-ular learning gains, than it is to explore the effectsthat various course components have on this atti-tude. We first arrived at a baseline model (Table 5)describing the correlation, for the entire sample, ofAttitude Index with a variety of independent vari-ables describing students’ attitudes, backgrounds, and plans. We then analyzed, one at a time, subsetsof the sample that reported engaging with variouscourse components, and included as a new inde-pendent variable their rating of each component’shelpfulness.We found that the only course components whosehelpfulness indices exhibit correlation with overallastronomy and STEM attitudes were in-class lec-tures and Skynet-based labs. While considerableeffort has been expended to add new components tothe Intro Astro curriculum, from in-class e-pollingsystems and questions, to providing videotapes oflectures to all students, to writing supplementary e-book materials, we cannot say at this time that theyhave had any effect one war or the other on attitudes.That is not to say that they have no effects at all –they very well may be found to improve traditionallearning outcomes, for instance. But the results ofthis analysis suggest that an instructor’s best bet forboosting attitudes with our Intro Astro curriculumis to concentrate on improving the quality of theirlectures and of the Skynet-based telescope labs thatthey offer.
Acknowledgments
We gratefully acknowledge the support of the Na-tional Science Foundation, through the followingprograms and awards: ESP 0943305, MRI-R2 0959447,AAG 1009052, 1211782, and 1517030, ISE 1223235,hbcu-up 1238809, TUES 1245383, and STEM+C1640131. We are also appreciative to have beensupported by the Mt. Cuba Astronomical Founda-tion, the Robert Martin Ayers Sciences Fund, andthe North Carolina Space Grant Consortium. Theauthors wish to thank Collin Wallace and MichaelFitzgerald for their very helpful and thoughtful com-ments on this work.
References
Alper, J., editor (2016).
Developing a NationalSTEM Workforce Strategy: A Workshop Sum-mary , Washington, DC. The National AcademiesPress.American Association for the Advancement of Sci- actors Contributing to Attitudinal Gains in Introductory Astronomy Courses — 10/15 ence (AAAS) (1990).
Project 2061: Sciencefor all Americans , New York. Oxford UniversityPress.Bartlett, S., Fitzgerald, M. T., McKinnon, D. H.,Danaia, L., and Lazendic-Galloway, J. (2018).Astronomy and Science Student Attitudes(ASSA): A short review and validation of a newinstrument.
Journal of Astronomy & Earth Sci-ences Education , 5(1):1–22.Gomez, E. L. and Fitzgerald, M. T. (2017). Robotictelescopes in education.
Astronomical Review ,13(1):28–68.Kastberg, D., Chan, J. Y., and Murray, G. (2016).
Performance of U.S. 15-year-old students in sci-ence, reading, and mathematics literacy in aninternational context: First look at PISA 2015 .National Center for Education Statistics, U.S. De-partment of Education, Washington, DC.Martin, J. R., Reichart, D. E., Dutton, D. A.,Maples, M. P., Berger, T. A., Ghigo, F. D., Hais-lip, J. B., Shaban, O. H., Trotter, A. S., Barnes,L. M., Paggen, M. L., Gao, R. L., Salemi, C. P.,Langston, G. I., Bussa, S., Duncan, J. A., White,S., Heatherly, S. A., Karlik, J. B., Johnson,E. M., Reichart, J. E., Foster, A. C., Koupri-anov, V. V., Mazlin, S., and Harvey, J. (2018,in press). Skynet Algorithm for Single-Dish Ra-dio Mapping I: Contaminant-Cleaning, Mapping,and Photometering Small-Scale Structures.
ApJS .Mulvey, P. and Nicholson, S. (2014).
Astron-omy Enrollments and Degrees: Results from the2012 Survey of Astronomy Enrollments and De-grees . American Institute of Physics StatisticalResearch Center, College Park, MD.National Research Council (NRC) (2007).
Tak-ing Science to School: Learning and TeachingScience in Grades K–8 , Washington, DC. TheNational Academies Press.National Research Council (NRC) (2010).
NewWorlds, New Horizons in Astronomy and As-trophysics , Washington, DC. The NationalAcademies Press. Prather, E. E., Rudolph, A. L., Brissenden, G., andSchlingman, W. M. (2009). A national studyassessing the teaching and learning of introduc-tory astronomy. Part I. The effect of interactiveinstruction.
American Journal of Physics , 77:320–330.Provasnik, S., Malley, L., Stephens, M., Landeros,K., Perkins, R., and Tang, J. H. (2016).
High-lights from TIMSS and TIMSS Advanced 2015:Mathematics and Science Achievement of U.S.Students in Grades 4 and 8 and in AdvancedCourses at the End of High School in an Inter-national Context . National Center for EducationStatistics, U.S. Department of Education, Wash-ington, DC.Reichart, D., Nysewander, M., Moran, J., Bartelme,J., Bayliss, M., Foster, A., Clemens, J. C., Price,P., Evans, C., Salmonson, J., Trammell, S., Car-ney, B., Keohane, J., and Gotwals, R. (2005).PROMPT: Panchromatic Robotic Optical Mon-itoring and Polarimetry Telescopes.
Nuovo Ci-mento C Geophysics Space Physics C , 28:767.Schlingman, W. M., Prather, E. E., Wallace, C. S.,Rudolph, A. L., and Brissenden, G. (2012). AClassical Test Theory Analysis of the Light andSpectroscopy Concept Inventory National StudyData Set.
Astronomy Education Review , 11(1).Slater, T. F. (2007). Using remote-control tele-scopes in ASTR 101. In
Cosmos in the Class-room: Symposium on Teaching Introductory As-tronomy , San Francisco, CA. Astronomical Soci-ety of the Pacific.Slater, T. F. (2018, accepted). To telescope ornot to telescope? In Fitzgerald, M., James,C. R., Buxner, S., and White, S., editors,
RoboticTelescopes, Student Research and Education(RTSRE) Proceedings .Slater, T. F., Burrows, A. C., French, D. A., Sanchez,R. A., and Tatge, C. B. (2014). A proposed as-tronomy learning progression for remote tele-scope observation.
Journal of College Teaching& Learning , 11:197. actors Contributing to Attitudinal Gains in Introductory Astronomy Courses — 11/15
Williamson, K., Prather, E. E., and Willoughby, S.(2016). Applicability of the newtonian gravityconcept inventory to introductory college physicsclasses.
American Journal of Physics , 84(6):458–466.Zeilik, M., Schau, C., and Mattern, N. (1999).Conceptual astronomy. II. Replicating concep-tual gains, probing attitude changes across threesemesters.
American Journal of Physics , 67:923–927. actors Contributing to Attitudinal Gains in Introductory Astronomy Courses — 12/15Table 2.
More detailed breakdown of student survey responses by section, including number of UNC labsand homeworks each instructor utilized, whether they used any of the Skynet-based telescope labs,whether the section was online, and whether any there was any lab component to the course at all.
Semest. Institut. Instructor actors Contributing to Attitudinal Gains in Introductory Astronomy Courses — 13/15Table 3.
The questions that were used to compute the astronomy and science Attitude Index dependentvariable. Student responses to each statement were scaled from -1 = strongly disagree to +1 = stronglyagree. The sign of responses was flipped for those statements with a science-negative orientation, and thenall were averaged to arrive at the Attitude Index.
Attitude Index Question Orientation
Astronomy is a subject learned quickly by most people. +I have trouble understanding astronomy because of how I think. -Astronomy concepts are easy to understand. +Astronomy is irrelevant to my life. -I was under stress during astronomy class. -I understand how to apply analytical reasoning to astronomy. +Learning astronomy requires a great deal of discipline. -I have no idea of what’s going on in astronomy. -I like astronomy. +What I learned in astronomy will not be useful in my career. -Most people have to learn a new way of thinking to do astronomy. -Astronomy is highly technical. -I felt insecure when I had to do astronomy homework. -I find it difficult to understand astronomy concepts. -I enjoyed taking this astronomy course. +I made a lot of errors applying concepts in astronomy. -Astronomy involves memorizing a massive collection of facts. -Astronomy is a complicated subject. -I can learn astronomy. +Astronomy is worthless. -I am scared of astronomy. -Science is a part of everyday life. +Scientific concepts are easy to understand. +Science is not useful to the typical professional. -The thought of taking a science course scares me. -I like science. +I find it difficult to understand scientific concepts. -I can learn science. +Scientific skills will make me more employable. +Science is a complicated subject. -I use science in my everyday life. +Scientific thinking is not applicable to my life outside my job. -Science should be a required part of my professional training. + actors Contributing to Attitudinal Gains in Introductory Astronomy Courses — 14/15Table 4.
Multiple linear regression correlation coefficients for the entire survey data set ( N = p > . Variable Coefficient p -value Course Attitude Index 0.25 2.5E-26Grade Index 0.16 1.6E-12Career Index 0.11 2.0E-21Instructor Attitude Index 0.073 5.4E-03Math Index 0.053 1.5E-04Institution Index -0.048 6.1E-06Work Index -0.10 7.8E-10
Table 5.
Results of multiple linear regression on Attitude Index vs. significant variables, includingLecture Helpfulness Index, for the subset of N =
712 students who attended in-class lectures.
Top : Fitwith Instructor Attitude Index (which is not correlated with Attitude Index at the p < .
05 level);
Bottom :Fit with Instructor Attitude Index excluded.
Variable Coefficient p-value
Course Attitude Index 0.24 1.3E-21Grade Index 0.14 4.2E-10STEM Index 0.11 1.6E-18Lecture Helpfulness Index 0.071 1.7E-04Math Index 0.063 1.0E-05
Instructor Attitude Index 0.036 2.0E-01
Institution Index -0.048 6.6E-06Work Index -0.11 1.0E-09
Variable Coefficient p-value
Course Attitude Index 0.26 2.2E-28Grade Index 0.14 2.5E-10STEM Index 0.11 2.6E-18Lecture Helpfulness Index 0.077 2.1E-05Math Index 0.064 7.7E-06Institution Index -0.046 1.2E-05Work Index -0.11 1.0E-09 actors Contributing to Attitudinal Gains in Introductory Astronomy Courses — 15/15Table 6.
Results of multiple linear regression on Attitude Index vs. significant variables, including SkynetHelpfulness Index, for the subset of N =
508 students who participated in Skynet-based telescope labs.
Top : Fit with Instructor Attitude Index (which is not correlated with Attitude Index at the p < .
05 level);
Bottom : Fit with Instructor Attitude Index excluded.
Variable Coefficient p-value
Grade Index 0.21 4.1E-13Course Attitude Index 0.20 5.8E-11Career Index 0.11 1.1E-13
Instructor Attitude Index 0.057 6.5E-02
Skynet Helpfulness Index 0.044 1.5E-02Math Index 0.036 5.1E-02Institution Index -0.050 1.2E-04Work Index -0.095 1.2E-05