FF1000Research
Software Carpentry: Lessons Learned
Greg Wilson Mozilla Foundation / [email protected]
Over the last 15 years, Software Carpentry has evolved from a week-long trainingcourse at the US national laboratories into a worldwide volunteer effort to raisestandards in scientific computing. This article explains what we have learned alongthe way the challenges we now face, and our plans for the future.
Page 1 of 10 a r X i v : . [ c s . G L ] J a n Introduction
In January 2012, John Cook posted this to his widely-read blog [ ] :In a review of linear programming solvers from1987 to 2002, Bob Bixby says that solvers bene-fited as much from algorithm improvements asfrom Moore’s law: “Three orders of magnitudein machine speed and three orders of magni-tude in algorithmic speed add up to six ordersof magnitude in solving power. A model thatmight have taken a year to solve 10 years agocan now solve in less than 30 seconds.”A million-fold speedup is impressive, but hardware andalgorithms are only two sides of the iron triangle of pro-gramming. The third is programming itself, and whileimprovements to languages, tools, and practices haveundoubtedly made software developers more productivesince 1987, the speedup is percentages rather than or-ders of magnitude. Setting aside the minority who dohigh-performance computing (HPC), the time it takes the“desktop majority” of scientists to produce a new com-putational result is increasingly dominated by how longit takes to write, test, debug, install, and maintain soft-ware.The problem is, most scientists are never taught how todo this. While their undergraduate programs may in-clude a generic introduction to programming or a statis-tics or numerical methods course (in which they’re oftenexpected to pick up programming on their own), they arealmost never told that version control exists, and rarelyif ever shown how to design a maintainable program ina systematic way, or how to turn the last twenty com-mands they typed into a re-usable script. As a result, theyroutinely spend hours doing things that could be done inminutes, or don’t do things at all because they don’t knowwhere to start [
2, 3 ] .This is where Software Carpentry comes in. We ran 91workshops for over 4300 scientists in 2013. In them,more than 100 volunteer instructors helped attendeeslearn about program design, task automation, versioncontrol, testing, and other unglamorous but time-testedskills [ ] . Two independent assessments in 2012 showedthat attendees are actually learning and applying at leastsome of what we taught; quoting [ ] :The program increases participants’ computa-tional understanding, as measured by morethan a two-fold (130%) improvement in testscores after the workshop. The program alsoenhances their habits and routines, and leadsthem to adopt tools and techniques that areconsidered standard practice in the softwareindustry. As a result, participants express ex-tremely high levels of satisfaction with their in-volvement in Software Carpentry (85% learnedwhat they hoped to learn; 95% would recom-mend the workshop to others). Despite these generally positive results, many researchersstill find it hard to apply what we teach to their ownwork, and several of our experiments—most notably ourattempts to teach online—have been failures. From Red to Green
Some historical context will help explain where and whywe have succeeded and failed.
Version 1: Red Light
In 1995-96, the author organized a series of articles in
IEEE Computational Science & Engineering titled, “WhatShould Computer Scientists Teach to Physical Scientistsand Engineers?” [ ] . The articles grew out of the frus-tration he had working with scientists who wanted torun before they could walk—i.e., to parallelize complexprograms that weren’t broken down into self-containedfunctions, that didn’t have any automated tests, and thatweren’t under version control [ ] .In response, John Reynders (then director of the Ad-vanced Computing Laboratory at Los Alamos NationalLaboratory) invited the author and Brent Gorda (now atIntel) to teach a week-long course on these topics to LANLstaff. The course ran for the first time in July 1998, andwas repeated nine times over the next four years. It even-tually wound down as the principals moved on to otherprojects, but two valuable lessons were learned:1. Intensive week-long courses are easy to schedule(particularly if instructors are travelling) but by thelast two days, attendees’ brains are full and learningdrops off significantly.2. Textbook software engineering is not the right thingto teach most scientists. In particular, careful docu-mentation of requirements and lots of up-front de-sign aren’t appropriate for people who (almost bydefinition) don’t yet know what they’re trying to do.Agile development methods, which rose to promi-nence during this period, are a less bad fit to re-searchers’ needs, but even they are not well suitedto the “solo grad student” model of working so com-mon in science. Versions 2 and 3: Another Red Light
The Software Carpentry course materials were updatedand released in 2004-05 under a Creative Commons li-cense thanks to support from the Python Software Foun-dation [ ] . They were used twice in a conventional term-long graduate course at the University of Toronto aimedat a mix of students from Computer Science and the phys-ical and life sciences.The materials attracted 1000-2000 unique visitors amonth, with occasional spikes correlated to courses andmentions in other sites. But while grad students (and theoccasional faculty member) found the course at Torontouseful, it never found an institutional home. Most Com-puter Science faculty believe this basic material is too Page 2 of 10 easy to deserve a graduate credit (even though a signifi-cant minority of their students, particularly those comingfrom non-CS backgrounds, have no more experience ofpractical software development than the average physi-cist). However, other departments believe that courseslike this ought to be offered by Computer Science, in thesame way that Mathematics and Statistics departmentsroutinely offer service courses. In the absence of an insti-tutional mechanism to offer credit courses at some inter-departmental level, this course, like many other interdis-ciplinary courses, fell between two stools.
It Works Too Well to be Interesting
We have also found that what we teach sim-ply isn’t interesting to most computer scientists.They are interested in doing research to ad-vance our understanding of the science of com-puting; things like command-line history, tabcompletion, and “select * from table” have beenaround too long, and work too well, to be pub-lishable any longer. As long as universities re-ward research first, and supply teaching last, itis simply not in most computer scientists ownbest interests to offer this kind of course.Secondly, despite repeated invitations, other people didnot contribute updates or new material beyond an occa-sional bug report. Piecemeal improvement may be nor-mal in open source development, but Wikipedia aside, itis still rare in other fields. In particular, people often useone another’s slide decks as starting points for their owncourses, but rarely offer their changes back to the origi-nal author in order to improve it. This is partly becauseeducators’ preferred file formats (Word, PowerPoint, andPDF) can’t be handled gracefully by existing version con-trol systems, but more importantly, there simply isn’t a“culture of contribution” in education for projects likeSoftware Carpentry to build on.The most important lesson learned in this period was thatwhile many faculty in science, engineering, and medicineagree that their students should learn more about com-puting, they won’t agree on what to take out of the cur-rent curriculum to make room for it. A typical under-graduate science degree has roughly 1800 hours of classand laboratory time; anyone who wants to add more pro-gramming, statistics, writing, or anything else must eitherlengthen the program (which is financially and institu-tionally infeasible) or take something out. However, ev-erything in the program is there because it has a passion-ate defender who thinks it’s vitally important, and who islikely senior to those faculty advocating the change.
It Adds Up
Saying, “We’ll just add a little computing to ev-ery other course,” is a cheat: five minutes perhour equals four entire courses in a four-yearprogram, which is unlikely to ever be imple-mented. Pushing computing down to the highschool level is also a non-starter, since that cur-riculum is also full. The sweet spot for this kind of training is therefore thefirst two or three years of graduate school. At that point,students have time (at least, more time than they’ll haveonce they’re faculty) and real problems of their own thatthey want to solve.
Version 4: Orange Light
The author rebooted Software Carpentry in May 2010with support from Indiana University, Michigan StateUniversity, Microsoft, MITACS, Queen Mary University ofLondon, Scimatic, SciNet, SHARCNet, and the UK MetOffice. More than 120 short video lessons were recordedduring the subsequent 12 months, and six more week-long classes were run for the backers. We also offered anonline class three times (a MOOC avant la lettre ).This was our most successful version to date, in part be-cause the scientific landscape itself had changed. Openaccess publishing, crowd sourcing, and dozens of otherinnovations had convinced scientists that knowing how toprogram was now as important to doing science as know-ing how to do statistics. Despite this, though, most stillregarded it as a tax they had to pay in order to get theirscience done. Those of us who teach programming mayfind it interesting in its own right, but as one course par-ticipant said, “If I wanted to be a programmer instead ofa chemist, I would have chosen computer science as mymajor instead of chemistry.”Despite this round’s overall success, there were severaldisappointments:1. Once again, we discovered that five eight-hour daysare more wearying than enlightening.2. And once again, only a handful of other people con-tributed material, not least because creating videosis significantly more challenging than creating slides.Editing or modifying them is harder still: while atypo in a slide can be fixed by opening PowerPoint,making the change, saving, and re-exporting thePDF, inserting new slides into a video and updatingthe soundtrack seems to take at least half an hourregardless of how small the change is.3. Most importantly, the MOOC format didn’t work:only 5-10% of those who started with us finished,and the majority were people who already knewmost of the material. Both figures are in line withcompletion rates and learner demographics for otherMOOCs [ ] , but are no less disappointing because ofthat.The biggest take-away from this round was the needcome up with a scalable, sustainable model. One instruc-tor simply can’t reach enough people, and cobbling to-gether funding from half a dozen different sources everytwelve to eighteen months is a high-risk approach. Version 5: Green Light
Software Carpentry restarted once again in January 2012with a new grant from the Sloan Foundation, and back-ing from the Mozilla Foundation. This time, the model
Page 3 of 10 ( ( " ( ( " ( ( " ( ( " ( ( " ( ( " ( ( " ( ( " ( ( " ( ( " ( ( " ( ( " ( ( " ( ( " ( ( " Figure 1. Cumulative Number of Workshops -‐ -‐ -‐ -‐ -‐ -‐ -‐ -‐ -‐ -‐ -‐ -‐ -‐ -‐ -‐ -‐ -‐ -‐ -‐ -‐ -‐ -‐ -‐ -‐ -‐ -‐ Figure 2. Cumulative Enrolment was two-day intensive workshops like those pioneered byThe Hacker Within, a grassroots group of grad studentshelping grad students at the University of Wisconsin –Madison.Shortening the workshops made it possible for more peo-ple to attend, and increased the proportion of materialthey retained. It also forced us to think much harderabout what skills scientists really needed. Out wentobject-oriented programming, XML, Make, GUI construc-tion, design patterns, and software development lifecy-cles. Instead, we focused on a handful of tools (discussedin the next section) that let us introduce higher-level con-cepts without learners really noticing.Reaching more people also allowed us to recruit more in-structors from workshop participants, which was essen-tial for scaling. Switching to a “host site covers costs”model was equally important: we still need funding forthe coordinator positions (the author and two part-timeadministrative assistants at Mozilla, and part of one staffmember’s time at the Software Sustainability Institute inthe UK), but our other costs now take care of themselves.Our two-day workshops have been an unqualified suc-cess. Both the number of workshops, and the number ofpeople attending, have grown steadily:More importantly, feedback from participants is stronglypositive. While there are continuing problems with soft-ware setup and the speed of instruction (discussed be-low), 80-90% of attendees typically report that they wereglad they attended and would recommend the workshopsto colleagues.
What We Do
So what does a typical workshop look like? • Day 1 a.m. : The Unix shell. We only show par-ticipants a dozen basic commands; the real aim isto introduce them to the idea of combining single-purpose tools (via pipes and filters) to achieve de-sired effects, and to getting the computer to repeatthings (via command completion, history, and loops)so that people don’t have to. • Day 1 p.m. : Programming in Python (or sometimesR). The real goal is to show them when, why, andhow to grow programs step-by-step as a set of com-prehensible, reusable, and testable functions. • Day 2 a.m. : Version control. We begin by emphasiz-ing how it’s a better way to back up files than creat-ing directories with names like “final”, “really_final”,“really_final_revised”, and so on, then show themthat it’s also a better way to collaborate than FTPor Dropbox. • Day 2 p.m. : Using databases and SQL. The realgoal is to show them what structured data actuallyis—in particular, why atomic values and keys areimportant—so that they will understand why it’s im-portant to store information this way.As the comments on the bullets above suggest, our realaim isn’t to teach Python, Git, or any other specific tool:it’s to teach computational competence . We can’t do thisin the abstract: people won’t show up for a hand-wavingtalk, and even if they do, they won’t understand. If weshow them how to solve a specific problem with a specifictool, though, we can then lead into a larger discussion ofhow scientists ought to develop, use, and curate software.We also try to show people how the pieces fit together:how to write a Python script that fits into a Unix pipeline,how to automate unit tests, etc. Doing this gives us achance to reinforce ideas, and also increases the odds ofthem being able to apply what they’ve learned once theworkshop is over.Of course, there are a lot of local variations around thetemplate outlined above. Some instructors still use thecommand-line Python interpreter, but a growing numberhave adopted the IPython Notebook, which has proven tobe an excellent teaching and learning environment.We have also now run several workshops using R insteadof Python, and expect this number to grow. While somepeople feel that using R instead of Python is like usingfeet and pounds instead of the metric system, it is the lin-gua franca of statistical computing, particularly in the lifesciences. A handful of workshops also cover tools suchas LaTeX, or domain-specific topics such as audio file pro-cessing. We hope to do more of the latter going forwardnow that we have enough instructors to specialize.We aim for no more than 40 people per room at a work-shop, so that every learner can receive personal attentionwhen needed. Where possible, we now run two or morerooms side by side, and use a pre-assessment question-naire as a sorting hat to stream learners by prior expe-rience, which simplifies teaching and improves their ex-perience. We do not to shuffle people from one room to
Page 4 of 10 another between the first and second day: with the bestinter-instructor coordination in the world, it still results induplication, missed topics, and jokes that make no sense.Our workshops were initially free, but we now often havea small registration fee (typically $20–40), primarily be-cause it reduces the no-show rate from a third to roughly5%. When we do this, we must be very careful not totrip over institutional rules about commercial use of theirspace: some universities will charge us hundreds or thou-sands of dollars per day for using their classrooms if anymoney changes hands at any point. We have also exper-imented with refundable deposits, but the administrativeoverheads were unsustainable.
Commercial Offerings
Our material is all covered by the CreativeCommons – Attribution license, so anyone whowants to use it for corporate training can do sowithout explicit permission from us. We encour-age this: it would be great if graduate studentscould help pay their bills by sharing what theyknow, in the way that many programmers earnpart or all of their living from working on opensource software.What does require permission is use of our nameand logo, both of which are trademarked. We’rehappy to give that permission if we’ve certi-fied the instructor and have a chance to double-check the content, but we do want a chance tocheck: we have had instances of people call-ing something “Software Carpentry” when ithad nothing to do with what we usually teach.We’ve worked hard to create material that ac-tually helps scientists, and to build some namerecognition around it, and we’d like to makesure our name continues to mean something.As well as instructors, we rely local helpers to wan-der the room and answer questions during practicals.These helpers may be participants in previous workshopswho are interested in becoming instructors, grad studentswho’ve picked up some or all of this on their own, ormembers of the local open source community; where pos-sible, we aim to have at least one helper for every eightlearners.We find workshops go a lot better if people come ingroups (e.g., 4–5 people from one lab) or have otherpre-existing ties (e.g., the same disciplinary background).They are less inhibited about asking questions, and cansupport each other (morally and technically) when thetime comes to put what they’ve learned into practiceafter the workshop is over. Group signups also yieldmuch higher turnout from groups that are otherwise of-ten under-represented, such as women and minority stu-dents, since they know in advance that they will be in asupportive environment.
Small Things Add Up
As in chess, success in teaching often comes from the ac-cumulation of seemingly small advantages. Here are afew of the less significant things we do that we believehave contributed to our success.
Live Coding
We use live coding rather than slides: it’s more convinc-ing, it enables instructors to be more responsive to “whatif?” questions, and it facilitates lateral knowledge transfer(i.e., people learn more than we realized we were teach-ing them by watching us work). This does put more ofa burden on instructors than a pre-packaged slide deck,but most find it more fun.
Open Everything
Our grant proposals, mailing lists, feedback from work-shops, and everything else that isn’t personally sensitiveis out in the open. While we can’t prove it, we believethat the fact that people can see us actively succeeding,failing, and learning buys us some credibility and respect.
Open Lessons
This is an important special case of the previous point.Anyone who wants to use our lessons can take what wehave, make changes, and offer those back by sending us apull request on GitHub. As mentioned earlier, this work-flow is still foreign to most educators, but it is allowing usto scale and adapt more quickly and more cheaply thanthe centralized approaches being taken by many high-profile online education ventures.
Use What We Teach
We also make a point of eating our own cooking, e.g.,we use GitHub for our web site and to plan workshops.Again, this buys us credibility, and gives instructors achance to do some hands-on practice with the thingsthey’re going to teach. The (considerable) downside isthat it can be quite difficult for newcomers to contributematerial; we are therefore working to streamline thatprocess.
Meet the Learners on Their Own Ground
Learners tell us that it’s important to them to leave theworkshop with their own working environment set up.We therefore continue to teach on all three major plat-forms (Linux, Mac OS X, and Windows), even thoughit would be simpler to require learners to use just one.We have experimented with virtual machines on learners’computers to reduce installation problems, but those in-troduce problems of their own: older or smaller machinessimply aren’t fast enough. We have also tried using VMsin the cloud, but this makes us dependent on university-quality WiFi. . .
Page 5 of 10
Collaborative Note-Taking
We often use Etherpad for collaborative note-taking andto share snippets of code and small data files with learn-ers. (If nothing else, it saves us from having to ask stu-dents to copy long URLs from the presenter’s screen totheir computers.) It is almost always mentioned posi-tively in post-workshop feedback, and several workshopparticipants have started using it in their own teaching.We are still trying to come up with an equally good way toshare larger files dynamically as lessons progress. Versioncontrol does not work, both because our learners are newto it (and therefore likely to make mistakes that affectclassmates) and because classroom WiFi frequently can’thandle a flurry of multi-megabyte downloads.
Sticky Notes and Minute Cards
Giving each learner two sticky notes of different col-ors allows instructors to do quick true / false questions asthey’re teaching. It also allows real-time feedback duringhands-on work: learners can put a green sticky on theirlaptop when they have something done, or a red stickywhen they need help. We also use them as minute cards:before each break, learners take a minute to write onething they’ve learned on the green sticky, and one thingthey found confusing (or too fast or too slow) on the redsticky. It only takes a couple of minutes to collate these,and allows instructors to adjust to learners’ interests andspeed. Pair Programming
Pairing is a good practice in real life, and an even betterway to teach: partners can not only help each other outduring the practical, but clarify each other’s misconcep-tions when the solution is presented, and discuss com-mon research interests during breaks. To facilitate it, westrongly prefer flat seating to banked (theater-style) seat-ing; this also makes it easier for helpers to reach learnerswho need assistance.
Keep Experimenting
We are constantly trying out new ideas (though not al-ways on purpose). Among our current experiments are:
Partner and Adapt
We have built a very fruitful partner-ship with the Software Sustainability Institute (SSI),who now manage our activities in the UK, and areadapting our general approach to meet particular lo-cal needs.
A Driver’s License for HPC
As another example of thiscollaboration, we are developing a “driver’s license”for researchers who wish to use the DiRAC HPC fa-cility. During several rounds of beta testing, we haverefined an hour-long exam to assess people’s profi-ciency with the Unix shell, testing, Makefiles, andother skills. This exam was deployed in the fall of2013, and we hope to be able to report on it by mid-2014.
New Channels
On June 24-25, 2013, we ran our firstworkshop for women in science, engineering, andmedicine. This event attracted 120 learners, 9 in-structors, a dozen helpers, and direct sponsorshipfrom several companies, universities, and non-profitorganizations. Our second such workshop will runin March 2014, and we are exploring ways to reachother groups that are underrepresented in comput-ing.
Smuggling It Into the Curriculum
Many of our instruc-tors also teach regular university courses, and sev-eral of them are now using part or all of our materialas the first few lectures in them. We strongly encour-age this, and would welcome a chance to work withanyone who wishes to explore this themselves.
Instructor Training
To help people teach, we now run an online trainingcourse for would-be instructors. It takes 2–4 hours / weekof their time for 12–14 weeks (depending on schedulinginterruptions), and introduces them to the basics of edu-cational psychology, instructional design, and how thesethings apply to teaching programming. It’s necessarilyvery shallow, but most participants report that they findthe material interesting as well as useful.Why do people volunteer as instructors? To make the world a better place.
The two things weneed to get through the next hundred years are morescience and more courage; by helping scientists domore in less time, we are helping with the former.
To make their own lives better.
Our instructors are of-ten asked by their colleagues to help with comput-ing problems. The more those colleagues know, themore interesting those requests are.
To build a reputation.
Showing up to run a workshop isa great way for people to introduce themselves tocolleagues, and to make contact with potential col-laborators. This is probably the most important rea-son from Software Carpentry’s point of view, sinceit’s what makes our model sustainable.
To practice teaching.
This is also important to peoplecontemplating academic careers.
To help diversify the pipeline.
Computing is 12-15% fe-male, and that figure has been dropping since the1980s. While figures on female participation in com-putational science are hard to come by, a simplehead count shows the same gender skew. Someof our instructors are involved in part because theywant to help break that cycle by participating in ac-tivities like our workshop for women in science andengineering in Boston in June 2013.
To learn new things, or learn old things in more detail.
Working alongside an instructor with more experi-ence is a great way to learn more about the tools, aswell as about teaching.
Page 6 of 10
It’s fun.
Our instructors get to work with smart peoplewho actually want to be in the room, and don’t haveto mark anything afterward. It’s a refreshing changefrom teaching undergraduate calculus. . .
TODO
We’ve learned a lot, and we’re doing a much better jobof reaching and teaching people than we did eighteenmonths ago, but there are still many things we need toimprove.
Too Slow and
Too Fast
The biggest challenge we face is the diversity of our learn-ers’ backgrounds and skill levels. No matter what weteach, and how fast or how slow we go, 20% or moreof the room will be lost, and there’s a good chance that adifferent 20% will be bored.The obvious solution is to split people by level, butif we ask them how much they know about particu-lar things, they regularly under- or over-estimate theirknowledge. We have therefore developed a short pre-assessment questionnaire (listed in the appendix) thatasks them whether they could accomplish specific tasks.While far from perfect, it seems to work well enough forour purposes.
Finances
Our second-biggest problem is financial sustainability.The “host site covers costs” model allows us to offer moreworkshops, but does not cover the 2 full-time equivalentcoordinating positions at the center of it all. We do askhost sites to donate toward these costs, but are still look-ing for a long-term solution.
Long-Term Assessment
Third, while we believe we’re helping scientists, we havenot yet done the long-term follow-up needed to provethis. This is partly because of a lack of resources, butit is also a genuinely hard problem: no one knows how tomeasure the productivity of programmers, or the produc-tivity of scientists, and putting the two together doesn’tmake the unknowns cancel out.What we’ve done so far is collect verbal feedback at theend of every workshop (mostly by asking attendees whatwent well and what didn’t) and administer surveys im-mediately before and afterwards. Neither has been donesystematically, though, which limits the insight we can ac-tually glean. We are taking steps to address that, but thelarger question of what impact we’re having on scientists’productivity still needs to be addressed.
Meeting Our Own Standards
One of the reasons we need to do long-termfollow-up is to find out for our own benefitwhether we’re teaching the right things theright way. As just one example, some of us be-lieve that Subversion is significantly easier for novices to understand than Git because thereare fewer places data can reside and fewer stepsin its normal workflow. Others believe just asstrongly that there is no difference, or that Gitis actually easier to learn. While learnabilityisn’t the only concern—the large social networkcentered around GitHub is a factor as well—wewould obviously be able to make better deci-sions if we had more quantitative data to basethem on. “Is It Supposed to Hurt This Much?”
Fourth, getting software installed is often harder than us-ing it. This is a hard enough problem for experiencedusers, but almost by definition our audience is inexperi-enced , and our learners don’t (yet) know about systempaths, environment variables, the half-dozen places con-figuration files can lurk on a modern system, and so on.Combine that with two version of Mac OS X, three ofWindows, and two oddball Linux installations, and it’s al-most inevitable that every time we introduce a new tool,it won’t work as expected (or at all) for at least one per-son in the room. Detailed documentation has not proveneffective: some learners won’t read it (despite repeatedprompting), and no matter how detailed it is, it will beincomprehensible to some, and lacking for others.
Edit This
And while it may seem like a trivial thing, edit-ing text is always harder than we expect. Wedon’t want to encourage people to use naiveeditors like Notepad, and the two most popu-lar legacy editors on Unix (Vi and Emacs) areboth usability nightmares. We now recommenda collection of open and almost-open GUI edi-tors, but it remains a stumbling block.
Teaching on the Web
Challenge
What vs. How
Sixth on our list is the tension between teaching the“what” and the “how” of programming. When we teach ascripting language like Python, we have to spend time upfront on syntax, which leaves us only limited time for thedevelopment practices that we really want to focus on,but which are hard to grasp in the abstract. By compar-ison, version control and databases are straightforward:what you see is what you do is what you get.We also don’t as good a job as we would like teachingtesting. The mechanics of unit testing with an xUnit-styleframework are straightforward, and it’s easy to come up
Page 7 of 10 with representative test cases for things like reformattingdata files, but what should we tell scientists about test-ing the numerical parts of their applications? Once we’vecovered floating-point roundoff and the need to use “al-most equal” instead of “exactly equal”, our learners quitereasonably ask, “What should I use as a tolerance for mycomputation?” for which nobody has a good answer.
Standardization vs. Customization
What we actually teach varies more widely than the con-tent of most university courses with prescribed curricula.We think this is a strength—one of the reasons we recruitinstructors from among scientists is so that they can cus-tomize content and delivery for local needs—but we needto be more systematic about varying on purpose ratherthan by accident.
Watching vs. Doing
Finally, we try to make our teaching as interactive as pos-sible, but we still don’t give learners hands-on exercisesas frequently as we should. We also don’t give them asdiverse a range of exercises as we should, and those thatwe do give are often at the wrong level. This is partly dueto a lack of time, but disorganization is also a factor.There is also a constant tension between having studentsdo realistic exercises drawn from actual scientific work-flows, and giving them tasks that are small and decou-pled, so that failures are less likely and don’t have knock-on effects when they occur. This is exacerbated by thediversity of learners in the typical workshop, though wehope that will diminish as we organize and recruit alongdisciplinary lines instead of geographically.
Better Teaching Practices
Computing education researchers have learned a lot inthe past two decades about why people find it hard tolearn how to program, and how to teach them more effec-tively [
10, 11, 12, 13, 14 ] . We do our best to cover theseideas in our instructor training program, but are less goodabout actually applying them in our workshops. Conclusions
To paraphrase William Gibson, the future is alreadyhere—it’s just that the skills needed to implement it aren’tevenly distributed. A small number of scientists can eas-ily build an application that scours the web for recently-published data, launch a cloud computing node to com-pare it to home-grown data sets, and push the result toa GitHub account; others are still struggling to free theirdata from Excel and figure out which of the nine backupversions of their paper is the one they sent for publica-tion.The fact is, it’s hard for scientists to do the cool thingstheir colleagues are excited about without basic comput-ing skills, and impossible for them to know what othernew things are possible. Our ambition is to change that:not just to make scientists more productive today, but to allow them to be part of the changes that are transform-ing science in front of our eyes. If you would like to help,we’d like to hear from you.
Competing Interests
The author is an employee of the Mozilla Foundation.Over the years, Software Carpentry has received supportfrom: • The Sloan Foundation • Microsoft • NumFOCUS • Continuum Analytics • Enthought • The Python Software Foundation • Indiana University • Michigan State University • MITACS • The Mozilla Foundation • Queen Mary University London • Scimatic Inc. • SciNET • SHARCNET • The UK Met Office • The MathWorks • Los Alamos National Laboratory • Lawrence Berkeley National Laboratory
Grant Information
Software Carpentry is currently supported by a grantfrom the Sloan Foundation.
Acknowledgements
The author wishes to thank Brent Gorda, who helped cre-ate Software Carpentry sixteen years ago; the hundredsof people who have helped organize and teach workshopsover the years; and the thousands of people who havetaken a few days to learn how to get more science donein less time, with less pain. Particular thanks go to the fol-lowing for their comments, corrections, and inspiration: • Azalee Bostroem (Space Telescope Science Institute) • Chris Cannam (Queen Mary, University of London) • Stephen Crouch (Software Sustainability Institute) • Matt Davis (Datapad, Inc.) • Luis Figueira (King’s College London)
Page 8 of 10 • Richard “Tommy” Guy (Microsoft) • Edmund Hart (University of British Columbia) • Neil Chue Hong (Software Sustainability Institute) • Katy Huff (University of Wisconsin) • Michael Jackson (Edinburgh Parallel ComputingCentre) • W. Trevor King (Drexel University) • Justin Kitzes (University of California, Berkeley) • Stephen McGough (University of Newcastle) • Lex Nederbragt (University of Oslo) • Tracy Teal (Michigan State University) • Ben Waugh (University College London) • Lynne J. Williams (Rotman Research Institute) • Ethan White (Utah State University)
References [ ] John D. Cook. Moore’s Law Squared, 2012. Viewed July2013. [ ] Jo Erskine Hannay, Hans Petter Langtangen, CarolynMacLeod, Dietmar Pfahl, Janice Singer, and Greg Wilson.How do scientists develop and use scientific software? In
Second International Workshop on Software Engineering forComputational Science and Engineering (SECSE09) , 2009. [ ] Prakash Prabhu, Thomas B. Jablin, Arun Raman, YunZhang, Jialu Huang, Hanjun Kim, Nick P. Johnson, FengLiu, Soumyadeep Ghosh, Stephen Beard, Taewook Oh,Matthew Zoufaly, David Walker, and David I. August. Asurvey of the practice of computational science. In
Pro-ceedings of the 24th ACM / IEEE Conference on High Per-formance Computing, Networking, Storage and Analysis ,2011. [ ] Greg Wilson, D. A. Aruliah, C. Titus Brown, Neil P. ChueHong, Matt Davis, Richard T. Guy, Steven H.D. Haddock,Kathryn D. Huff, Ian M. Mitchell, Mark D. Plumbley, BenWaugh, Ethan P. White, and Paul Wilson. Best practicesfor scientific computing.
PLoS Biology , 12(1):e1001745,January 2014. [ ] Jorge Aranda. Software Carpentry Assessment Report,2012. [ ] Gregory V. Wilson. What Should Computer ScientistsTeach to Physical Scientists and Engineers?
IEEE Compu-tational Science and Engineering , Summer and Fall 1996. [ ] Greg Wilson. Where’s the Real Bottleneck in ScientificComputing?
American Scientist , January-February 2006. [ ] Greg Wilson. Software Carpentry: Getting Scientists toWrite Better Code by Making Them More Productive.
Computing in Science & Engineering , November-December2006. [ ] Katy Jordan. MOOC completion rates: The data, 2013.Viewed July 2013. [ ] Mark Guzdial. Why is it so hard to learn to program? InAndy Oram and Greg Wilson, editors,
Making Software:What Really Works, and Why We Believe It , pages 111–124.O’Reilly Media, 2010. [ ] Mark Guzdial. Exploring hypotheses about media com-putation. In
Proc. Ninth Annual International ACM Con-ference on International Computing Education Research ,ICER’13, pages 19–26. ACM, 2013. [ ] Orit Hazzan, Tami Lapidot, and Noa Ragonis.
Guide toTeaching Computer Science: An Activity-Based Approach .Springer, 2011. [ ] Leo Porter, Mark Guzdial, Charlie McDowell, and Beth Si-mon. Success in introductory programming: What works?
Communications of the ACM , 56(8), 2013. [ ] Juha Sorva.
Visual Program Simulation in IntroductoryProgramming Education . PhD thesis, Aalto University,2012.
A Pre-Assessment Questionnaire • What is your career stage? – Undergraduate – Graduate – Post-doc – Faculty – Industry – Support Staff – Other: • What is your discipline? – Space sciences – Physics – Chemistry – Earth sciences (geology, oceanography, meteo-rology) – Life science (ecology, zoology, botany) – Life science (biology, genetics) – Brain and neurosciences – Medicine – Engineering (civil, mechanical, chemical) – Computer science and electrical engineering – Economics – Humanities and social sciences – Tech support, lab tech, or support programmer – Administration – Other: • In three sentences or less, please describe your cur-rent field of work or your research question. • What OS will you use on the laptop you bring to theworkshop? – Linux – Apple OS X – Windows – I do not know what operating system I use. • With which programming languages, if any, couldyou write a program from scratch which importssome data and calculates mean and standard devi-ation of that data?