Supporting Real Demands in Software Engineering with a Four Steps Project-Based Learning Approach
Leonardo Humberto Silva, Renata Xavier Castro, Marice Costa Guimaraes
SSupporting Real Demands in Software Engineeringwith a Four Steps Project-Based Learning Approach st Leonardo Humberto Silva
Information Systems (IFNMG)Federal Institute Northern Minas Gerais
Salinas, [email protected] nd Renata Xavier Castro
Student Monitoring (IFNMG)Federal Institute Northern Minas Gerais
Salinas, [email protected] rd Marice Costa Guimaraes
Budget and Finance (IFNMG)Federal Institute Northern Minas Gerais
Salinas, [email protected]
Abstract —Project-based learning (PBL) is a student-centeredand learn-by-doing approach that organizes learning aroundprojects. While entrepreneurship and PBL in SE education arethrilling research topics, there seems to be very little workfocusing on the pros and cons of involving external stakeholdersto support real demands in software engineering education.Working on real projects also supports students to acquireleadership skills, such as communication, project management,and teamwork. This paper describes a case study integratingstudents from different Software Engineering programs and in-volving external stakeholders, underpinned by PBL concepts. Wepresent how this study was designed and implemented in a largeinstitution, in four steps, summarized as follows: (I) requirementsgathering and design; (II) information system development andimplementation; (III) integration tests and deployment process;(IV) support and maintenance activities.The study had the participation of 59 students from aprofessional technical course in step one, working in teams,and 10 undergraduate students from a Bachelor program inInformation Systems in the following steps, working in pairs.Overall, the feedback from stakeholders and students exceededexpectations, although it increased the workload of teachers. Wewere able to distill a new set of lessons learned, and we expectthat at least some of them will be useful for anyone implementinga similar course. As a consequence of this study, we plan toinstitutionally formalize the PBL course improvement processby defining specific outcomes and measurements.
Index Terms —Project-Based Learning, PBL, Software Engi-neering Education, External Stakeholders
I. I
NTRODUCTION
Software Engineering (SE) deals with the application of sys-tematic, disciplined, and quantifiable approaches to develop,operate, maintain, and evolve software [1], [2]. Currently, SEsubjects are present in several undergraduate courses, althoughthere is no consensus on what methods should be used to teachthem [3], [4]. Balancing theory and practice is a recurringchallenge in Software Engineering Education. A key objectivein a SE program is to provide students with the necessary toolsto begin professional engineering practice [5]. Additionally, SEstudents are expected to be able to choose and implement thedevelopment process best suited to the reality of a softwaredevelopment company or sector.Projects are complex tasks, based on challenging questionsor problems, that involve students in design, problem-solving,and decision-making activities. Project-based learning (PBL) is a student-centered and learn-by-doing approach that orga-nizes learning around projects. PBL allows students to workrelatively autonomously over extended periods, culminatingin realistic products or presentations [6]–[8]. In SE education,PBL is one of the main successful methods broadly used [4],[9]–[11].Working on real projects also support students to acquireentrepreneurial skills, such as communication, project man-agement, and team-work. In this scenario, one of the chal-lenges regarding the adoption of projects based on real-worldproblems is to find stakeholders with whom the students cancooperate and who are able and willing to invest necessarytime and resources [12], [13]. Moreover, while entrepreneur-ship and PBL in SE education are burgeoning research topics,there seems to be very little work focusing on the pros andcons of involving external stakeholders in software engineeringeducation.This paper describes a case study integrating studentsfrom different programs in software engineering education,underpinned by project-based learning. In this study, wedivide development activities among students from differentprograms, at different levels of SE education, avoiding somepitfalls highlighted in the literature when involving externalstakeholders in academic projects. We present how this studywas designed and implemented in a large institution, in foursoftware engineering steps. The first step is mostly relatedto requirements gathering and design. Step two is dedicatedto information system development and implementation. Stepthree encompass integration tests and deployment processes.And step four is planned for support and maintenance activitiesfor the deployed systems. We investigate the evolution ofthe course over four semesters through the analysis of teamprojects, student reports, instructor notes, and structured feed-back. Our experience allowed us to unveil a rich set of lessonslearned, which are will help to refine the course and, hopefully,serve as guidance for those trying to include a similar approachin their curriculum.The remainder of this paper is organized as follows. Sec-tion II presents the main concepts of PBL. Section III describesthe SE course where the study was performed. Section IVdescribes the goals, method, and the study design for theexecution of this study. Section V presents how our PBL a r X i v : . [ c s . S E ] F e b pproach was implemented in a large institution. Section VIpresents the discussion of the lessons learned. Section VIIdiscusses related work. Finally, Section VIII concludes thispaper and indicates future developments.II. P ROJECT -B ASED L EARNING
Project-based learning (PBL), also known as problem-basedlearning, is a learn-by-doing approach to science educationthat focuses on helping students develop self-directed learningskills [14], [15]. In SE education, PBL is one of the main suc-cessful methods broadly used [4], [9]–[11]. PBL transcends theclassroom, involving students in the investigation of realisticproblems, learning by working on projects, discovering andfinding solutions as they progress along the path. Brender [16]describes PBL as an instructional model based on having stu-dents facing real-world issues and problems that they considersignificant, determining how to solve them and then, actingcollaboratively to create problem solutions. The instructor hasa less central role, acting as a mediator, and students take moreresponsibility for their learning. Proposals for interdisciplinaryactivities are favored and encouraged in PBL, which alsogives students the opportunity to work relatively autonomouslyover extended periods, culminating in realistic products orpresentations [6]–[8].Despite the benefits of PBL, there are some difficultiesand lessons learned in the related literature regarding theapplication of PBL and verification of its results [9], [12],[13], [17]–[19]. Project-based learning does not have a fixedstructure, making its implementation sometimes ad-hoc andopportunistic. Protocol definition and working procedures re-quire more time for preparation and operation. Teachers haveto manage stakeholders’ expectations and frustrations. On theother hand, the learning process is more flexible and involvesinteraction and cooperation among students, teachers, andother stakeholders. Moreover, the outcomes achieved at theend motivate learners and educators to create better projectsin the future [15], [20].III. PBL C
OURSE D ESCRIPTION
This PBL course relies on the cooperation of four differentprograms that encompass software engineering content, at alarge public educational institution: • A Professional and Technical Computer high schoolprogram (“PTC program”, henceforth) • An Information Systems bachelor undergraduate program(“IS program”, henceforth) • A Supervised Curricular Internship program (“SCI pro-gram”, henceforth) • A Junior Enterprise program (“JE program”, henceforth)PTC is a well-known established program that aims to trainhigh school technical professionals to perform activities in theIT area, acting as ethical, critical, and apt citizens before en-tering higher education. The content of this program is not ascomprehensive and in-depth as in an undergraduate educationmatrix, but includes the main concepts related to algorithms, object-oriented programming, database, and system analysiswith software engineering concepts.For the IS program, the SE course has 160 hours dividedin two subjects, Software Engineering I (SE-I) and Soft-ware Engineering II (SE-II), with 80 hours each. The coursesyllabus includes: software engineering concepts, softwareproduct requirements, life cycle, and software developmentparadigms, software quality, agile methods, detailed softwaredesign and design patterns, software architecture and structure,validation and verification, system implementation and tests,software maintenance and configuration management. Theobjectives of creating projects that involve students from PTCand IS programs are: (i) encourage teamwork by integratingstudents from the technical course with students from thehigher education course; (ii) apply the same projects in bothprograms allowing all involved students to benefit from PBLlearning; (iii) divide tasks to create realistic schedules allowingstudents in the IS program to have time to finish the softwarethey produce.The SCI program complements the process of teaching /learning through the application of scientific-technical knowl-edge in real situations during the exercise of a future profes-sion. Students must fulfill the minimum of 200 hours of aninternship to be eligible for graduation. The idea is to use theinternship to deploy systems that were successfully completedat the end of the IS program. This step aims to tackle oneof the frustrations mentioned by Stegh¨ofer et al. [12] whenuniversity staff is involved in the PBL process, with greatexpectation, and sees the product been discarded when theproject is finished. Among the goals of the SCI program, thefollowing stand out: • Enable and encourage students to increase professionaltraining. • Know the philosophy, guidelines, organization, and oper-ation of companies and institutions. • Improve interpersonal relationships and teamwork skills. • Exercise critical sense and creativity in the future profes-sion. • Participate in projects, research, and/or extension pro-grams within the scope of professional practice.It is important to highlight that not all projects will bedeployed, but just those that reach a quality level that isapproved by all stakeholders. Besides, students can choose todo their internships in other areas and/or companies. In thesecases, a system developed and approved by the stakeholderswould have to wait for another team that could take over thedeployment stage.Once students graduate they tend to move on, leaving thesystems they deploy behind. To give continuous support tosystems deployed by the SCI program, students from the JEprogram take control and keep supporting clients maintainingand evolving the system. Junior enterprises (JEs) are organiza-tions managed by undergraduate students and designed to betraining spaces for entrepreneurship and professionalization inconnection with the external community. The primary goal ofEs is to develop their members personally and professionallythrough business experience, carrying out projects and servicesin the area of expertise of the undergraduate course to whichthe junior company is linked. They offer low-cost servicesin different areas of knowledge, and students’ activities areguided by professors from the host institution. Thus, inaddition to reaching their own goal, the JEs contribute tothe development of entrepreneurship in their community. JEsmainly serve micro and small companies, which usually do nothave access to senior consultancy when facing managementdifficulties. Moreover, participating in a junior enterprise,as part of extra-curricular activities, complement rather thansubstitute entrepreneurship education [21].The course initiated in 2018 and since the beginning weadopted PBL principles as follows: • Project Based : There is a software development projectthat is central to the course. Most classroom activities andlectures are driven by the progression of the project. • Realistic : Each project represents a real-world problem,and we use institution employees acting as real customers(external stakeholders) during the course. • Evidence-based : The students have to deliver intermedi-ate products, such as text notes, diagrams, and sourcecode, mostly related to the software development lifecycle. All activities of the project are driven by meaning-ful questions that direct students into investigating andapplying the SE theory for the project. • Teamwork : The students have to work in teams, typicallyfrom 4 to 6 students in the PTC program e from 2 to 4students in the IS program.IV. C
ASE S TUDY D ESIGN
The PBL approach for this study is divided into four steps:(i) Requirements gathering, starting with PTC program; (ii)System development, moving to IS program; (iii) Systemdeployment, implemented by students enrolled in the SCIprogram, and (iv) Maintenance and support activities, imple-mented by members of the JE program. The stakeholders foreach project are distributed into the following groups: • Mediator: professor that coordinates the course. • Collaborators: professors responsible for subjects in-volved in the programs. • Clients: University staff members that have specific de-mands. • System Analysts: Students responsible for executing theproject, under supervision.As initial planning, the mediator meets with other professors(Collaborators) to define how to conduct and evaluate theprojects. Each professor must allocate part of their classesto explain the program and, during each semester, allowthe development of activities related to the project. Students,especially from the PTC program, have very little or no freetime at all. Therefore, the projects cannot be treated as extraactivities and must be carried out during the period of classes. This is imperative for the course to be successful and not tointerfere with other extracurricular activities.After meeting collaborators, the mediator makes contactwith university staff members to select potential clients andprojects. This selection depends mostly on the sector’s de-mands, which could benefit from the use of an informationsystem. The client’s availability is also important once theyneed to take part in project meetings and other events. Theselected clients are then briefed about how the PBL approachworks. Since this moment, it is important to emphasize that thegoal is to allow the students to participate in a real situationof system development, making clear that the final product,software in this case, is not the main objective.Once projects and clients are defined, it is time to discussthe idea with the students. It is important to motivate them,making them understand that they are going to play the roleof system analysts, discussing a real demand with real clients.Again, it has to be clear that they will be evaluated duringmonitoring activities as the projects evolve, independently ifthe final product will be deployed or not.After the initial clarification phase, students from the PTCprogram are divided into teams (4 to 6 members) and eachteam must appoint one of them to be the project manager.As in the industry, the manager’s leadership role is of greatimportance in motivating people and creating an effectiveworking environment, and we intend to encourage that amongstudents. After organizing the teams, it’s time to allocate aproject to each one. We established that each team, belongingto the same class, should have a distinct project. Differentprojects have different levels of difficulty and complexity, andthis has to be considered when guiding and evaluating teams.After project allocation, students will be able to workon them during classes, depending on the strategy definedby the collaborators. The theory learned in class will beused directly in the projects. Subjects related to softwareengineering and software development take part in this step.Once these subjects are completed, by the end of the semester,it is expected that the students deliver the artifacts presentedin Table I.
TABLE IA
RTIFACTS TO BE PRODUCED BY
PTC
PROGRAM ( STEP ONE ) ID Artifacts description
A1 Detailed description of the system.A2 Functional and non-functional requirements.A3 Use case diagrams (UCs), representing user’s interactions.A4 Entity Relationship Diagram (ERD) and Relational Model (RM).A5 System prototype with main screens (layout only).
The development of the first three artifacts demands knowl-edge of key concepts present in the syllabus of Systems Anal-ysis and Design subject. It requires the use of communicationskills and the ability to abstract, synthesize, and documentinformation received by the user (client), who is probably notan IT person. We consider this a new and profitable experiencehat the students would not have in a traditional SE course,in which the teacher plays the rule of a user. To serve asexample for the students, we provide real lists of requirementsand UML use case diagrams [22]. Artifacts A4 are relatedto the content of the Database subject. Students are asked toproduce entity relationship diagrams and relational models.Finally, to produce artifact A5 (prototype) they need conceptspresent in Systems Development subject. The deadlines of theprojects need to be synchronized with the subjects involved,allowing students to immediately apply the theory they learnin their projects and allowing teachers to assess students intheir subjects. Teams are also asked to write a text describingtheir work, what was accomplished, and what still needed tobe done. This documentation is the starting point for the nextteams of the IS program that will take over the projects.Along with the delivery of the mentioned artifacts, teamshad to produce a weekly report resuming their activities,including meetings, main difficulties, achievements, and anyother relevant information. Fig. 1 presents a template werecommended to write these reports, which are very importantfor monitoring activities. This template consists of a textdocument, with a table to specify dates and activities carriedout and / or scheduled. Besides, there is a space at the end tolist the main difficulties encountered.
Fig. 1. Template recommended for weekly reports.
In step two, students from the IS program, guided by pairprogramming techniques [23], [24], replace those from thePTC program in the System Analyst group. Similarly, profes-sors in the Collaborators group will be replaced by those fromthe IS program. They all have to be briefed about the projectsand the produced artifacts. It is expected that professors andstudents from both programs interact at this point. One cancriticize the fact that students from the IS program do notparticipate in the initial discussions about the project, gatheringrequirements. However, we understand that if the students hadto take part in all software development life cycle phases theyprobably would not be able to complete the software and reachthe deployment step. Besides, the opportunity to analyze andunderstand artifacts produced by another team is a positiveexperience that highlights the importance of the quality ofthe artifacts, e.g. , application of best practices in source code,concise and clear documentation, and detailed diagrams [25].Besides, during the development of a system, it is normal tomake changes and adaptations, meaning that students from the IS program are likely to update and improve the artifactsthey receive. As happened in step one, students use the theorylearned in class directly into their projects.To go to step three, it is expected that students finish theproject and present the outcome to the stakeholders. Projectseligible for deployment go to step three, in which the studentsinvolved enroll for the SCI program. The decision about whichprojects go to step three has to be made together with theagreement of all stakeholders. It is possible to have goodprojects that don’t go to step three because the students wantto do their internship with something else, and it is alsopossible to have projects that do not fulfill properly client’srequirements. The outcome of step three is the deployment ofthe system in one specific sector of the institution.In step four, once the information system is successfully de-ployed, all artifacts and technology developed are transferredto the members of the JE program, including repository access.From this point, the junior enterprise takes over maintenanceand support tasks.V. C
ASE S TUDY I MPLEMENTATION
This section describes how the study was implemented ina large institution, according to the methodology described inSection IV.
A. PTC-Program Implementation (step one)
This implementation step was initiated in the secondsemester of 2018, involving the following subjects of the PTCprogram: Systems Analysis and Design, Systems Develop-ment, and Database. Two classes were involved, class one(“PTC-A”, henceforth) with 30 students, and class 2 (“PTC-B”, henceforth) with 29 students. After contacting universitystaff members, we selected seven proposals for projects. Abrief description of the projects can be found in Table II.
TABLE IIS
ELECTED P ROJECTS
ID Brief description Sector
P1 Information system for soil analysis Soils LabP2 Production control and input distribution system FinancierP3 System for managing internships InternshipP4 Production control system for agribusiness AgribusinessP5 Medical care control NursingP6 Appointments control for the veterinary hospital Veterinary H.P7 Research Projects Support System a Research a This project was included later in step one.
Classes were then divided into groups (teams) and eachteam was assigned to a project. The decision on how the teamswould be composed was delegated to the students. Class PTC-A was divided into five teams with six students each. ProjectsP1, P2, P3, P5, and P6, from Table II, were assigned to theteams of PTC-A by a lot. Class PTC-B was divided into sixteams, one of them with four students and the others withfive students each. Projects from P1 to P6 were assigned toteams of class PTC-B, also by lot. Table III summarizes thatssignment of projects to teams. Only project P4, in class PTC-A, was not assigned to any team.
TABLE IIII
NITIAL A LLOCATION OF P ROJECTS TO T EAMS
Class Project Team
P1 T1 6P2 T2 6P3 T3 6PTC-A P4 - -P5 T4 6P6 T5 6Total of Students 30P1 T6 5P2 T7 5P3 T8 4PTC-B P4 T9 5P5 T10 5P6 T11 5Total of Students 29
Projects were monitored during the development of the arti-facts described in Table I. For each artifact, we describe nextthe main observations and feedback collected from classes,meetings, statements, weekly reports, and from the artifactsthemselves.
Artifacts A1 (system description) and A2 (requirements).
Mostteams (9 out of 11) described in their weekly reports that themain difficulty related to these artifacts was: “
To understandand document everything reported by the user ”. Team T1requested to change their project, arguing that they did not likeproject P1 and were not able to understand the requirements.We could notice that the students were not getting alongwell with the user of project P3. And this same team (T1)also refused to work with project P4, that was availablefor class PTC-A. Because of this unexpected situation, themediating teacher searched and introduced a new project (P7- see Table II), which was assigned to the team T1, with noobjections this time.
Artifact A3 (use cases).
During the development of the UMLuse cases, the main technical difficulty reported was relatedto the specification of alternate and exception flows in eachdiagram, as reported by team T6: “The main difficulties we had were (1) elaborating alterna-tive flows for normative requirements; (2) understandingexceptions in the use case specification.”(Team Report)
Most teams had similar reports. Moreover, three teams (T2,T6, and T8) reported relationship problems and disagreementsamong members. We had to reallocate two students from theiroriginal teams because of that. A student from the T6 teamswitched places with another from the T8.
Artifacts A4 (entity relationship and relational models).
Theseartifacts were developed while students learned how to use modeling tools such as
Astah and MySQL Workbench . Atthis stage, the main difficulties varied considerably, dependingon the project. For example, projects P2 and P6 had morecomplex entity relationships. Project P5 demanded a smallnumber of tables but some of them with a large number offields, as reported by team T4: “ We are facing some difficultieswith the size of the documents/entities ”. In general, studentsalso reported some difficulties to determine the type and sizeof the table fields, e.g. , when to use the type char instead of varchar or a blob instead of a text . Artifact A5 (system prototype).
To complete this artifact stu-dents had to exercise mainly their programming skills. Oneof the difficulties reported was related to task allocation forthe development of interfaces. We can see how team T2 dealtwith this problem in the following excerpt of their report: “Our group is satisfied with the results achieved so far. Insome tasks, we had greater participation of some members,and in other tasks of others. We left the less complex partsfor students with greater difficulty in programming.”(Team Report)
Team T1, which requested to change the project in the begin-ning, and also presented some internal relationship problems,summarized their participation as follows: “At the beginning of the project there were several dif-ficulties, however, we managed to fulfill our goals andall members helped equally in the programming. We arehappy with the final result.”(Team Report)
In short, all teams were able to deliver their prototypes, somevery good, others not so much, but the evaluation consideredthe evolution during the entire development process.
B. IS-Program Implementation (step two)
In this second step, students from the IS program replacethose from the PTC program inheriting their projects, and allartifacts produced so far. This step initiated in the first semesterof 2019, involving the following subjects of the IS program:Software Engineering, Object-Oriented Programming II, andDatabase. One class was involved (“IS-A”, henceforth) with10 students.Class IS-A was then divided into groups (pairs) and eachpair was assigned to a project. The decision on how thepairs would be composed was delegated to the students. Therewere five pairs in total. Students were then introduced to theprojects, and its artifacts, so they could decide which onesthey wanted to take over. Table IV presents the assignment ofprojects to pairs. The first pair (PR1) chose to continue projectP2, started by team T2 from the PTC program. Pair PR2 choseproject P5, started by team T4, while pair PR3 chose the sameproject, but with artifacts developed by team T10. Pair PR4chose project P6, started by team T5, and pair PR6 also choseP6 but started by team T11.The first two weeks were intended to allow pairs to knowtheir projects in more detail. They were encouraged to interact https://astah.net/ NITIAL A LLOCATION OF P ROJECTS TO P AIRS IN THE IS PROGRAM
Class Pair Project
PR1 P2 by T2PR2 P5 by T4IS-A PR3 P5 by T10PR4 P6 by T5PR5 P6 by T11 with the user and also with the students from the PTC program,to better understand systems requirements and artifacts. Duringthis period, PR2 requested to change their project, stating thatthe artifacts they chose initially are no so good and that thesource code was too complicated. They decided to move toproject P2 by team T7.With exception of PR1, all others complained about thequality of source code and commentaries they received, statingthat many things had to be rebuilt, as we can see in this excerptfrom the first report of pair PR4: “We will keep the information present in the documents aswell as the text information on each screen of the program.However, we have to refactor classes into proper packages,as well as separate the images and put all interfaces andbuttons to work.”(Pair Report)
PR5 argued that refactoring was not worth and decided torewrite the source code from scratch: “After analyzing the code, it was verified that it is notfeasible to modify the existing code, and it was thereforedecided to create a new one. The layout and functionalityfor the time being, will be the same.”(Pair Report)
Summarizing, students from the IS program were satisfied withdocumentation, diagrams, and requirements gathered in stepone, but the artifacts related to source code and database tableswere difficult for them to understand and maintain. Only PR1praised the source code they inherited.
C. SCI-Program Implementation (step three)
In this third step, stakeholders of each project get togetherto evaluate and select which ones are eligible for deployment.This step was initiated in the second semester of 2019. Fromthe five projects of step two (IS program), three of them wereselected: P2 (pair PR1), P2 (pair PR2), and P5 (pair PR3). Aspresented in Table II, project P2 was designed for the Financialdepartment of the institution, and, at this point, there weretwo solutions available for this sector. We had at least threemeetings to decide which solution would be deployed. Theemployees involved could not make the decision, and it wasnecessary to establish specific criteria in order to choose. Inthe end, we decided to continue with P2 (PR1) and P5 (PR3).To deploy P2, students initially faced a problem withthe computer desktop available in the financial department.When the system was executed, the screens always appearedtruncated, cut, and it was necessary to adapt the layout tosupport older operating systems and monitors. In addition to this problem, there was a period in which the sector wasinvolved in a deadline and couldn’t support the students in theinternship and, because of that, the deployment was delayed.The deployment of P5 inside the infirmary of the institutiondemanded a series of changes in the database tables, giventhe issues that arose when users started entering data. Itwas necessary to go back a few steps in the specificationto adapt to the system. The participation and engagement ofthe responsible nurse were essential for the final validation ofthe system. To install the database in one of the institution’sMySQL servers, the students also had to interact with employ-ees in the information technology department. This was theirfirst experience of deploying an internal information systemdeveloped by students in a teaching project. All protocolswere new, for teachers, students, and employee members ofIT department.
D. JE-Program Implementation (step four)
When classroom lessons were interrupted, due to the pan-demic of COVID-19, we had to stop our PBL course beforecreating the junior enterprise that would receive, store, main-tain, and support the systems deployed. The institution hasa specific regulation for the creation of companies linked toundergraduate courses, and all preparations were already beingmade to formalize and complete this step. The intention is toresume this initiative, where it left off as soon as face-to-faceactivities can return safely.VI. L
ESSONS L EARNED
In this section, we provide an overview of the lessons wehave learned from applying our PBL approach. We describethese lessons according to the seven recurring risk themes de-scribed in [12] when involving external stakeholders in projectcourses: student ability, outcome, expectation, engagement,context, feedback, and misalignment.
Student Ability.
This risk is associated with the knowledge,skills, and abilities of the students. In our approach, we workedwith two different groups of students: we started with highschool level students from a professional technical coursein the PTC program and then continued with undergraduatestudents for the rest of the development process. The stu-dents of the technical course do not see the contents of thesubjects with the same depth as the undergraduate students.For example, subjects involved in the PTC program do notencompass Scrum [26] and Kanban [27] agile fundamentals,which are present in the undergraduate syllabus. For thisreason, the dunning level must be different, and teachers wereable to evaluate them separately since these different groups ofstudents do not work together. Students in the PTC programstart in step one, and after that, the projects continue withundergraduate students in the following steps. The output ofone step is the input for the next one, and the different teamsdo not have to interact, avoiding problems that could arise fromthe heterogeneity of the students involved. Overall, all groupsof students could benefit from the learning process. Someof them who showed no interest in the theoretical classes,efore the PBL course, took on a leadership role in the projectactivities and automatically improved their performance.
Lesson
In addition to the technical abilities required of students,we understand that communication issues play an importantrole here. The effective participation of users is fundamentalto the evolution of the projects, and communication skills arealso required of students, which represents a major challengefor most of them, in all steps of the project implementation.Students had important lessons about how to conduct meetingsand gather relevant information. Besides, unexpected problemsin this field may occur, as we presented in Section V-A, whenteachers had to intervene and reallocate students and tasks dueto internal and external relationships and communication prob-lems. It is important to show students that a good relationshipwith team members and external stakeholders is key to suc-cess. Moreover, a very recent study with Stack Overflow jobsdemonstrated that communication, collaboration, and problem-solving are the most demanded soft skills that IT companieslook for in candidates [28].
Lesson
This risk is associated with the desired results for alllenses: students, external stakeholders, and teachers. At first,students have a certain fear of not being able to reach theend of the project with a satisfactory outcome, as it is a realdemand, with real users. External stakeholders, potential users,in this case, are not concerned about grades or been evaluated,they create instead an expectation related to the possibility ofhaving an information system that will help them with theirdaily activities. For the teachers, the priority is on studentlearning and completion of the related SE subjects.As highlighted before, the expectation around the outcomeof a project is twofold, it feeds and increases stakeholdersmotivations, but, in the same way, it can create a feelingof disappointment if the system is not deployed at the end.We were able to experience this feeling, since of the sevensectors/projects involved, only two went through the deploy-ment step. However, how the activities and meetings withexternal stakeholders were conducted, making it clear, fromthe beginning, what the main objectives were, contributed tothe relief of frustration, mitigating this threat. It was possible toevidence this fact when projects ended, and people responsiblefor sectors that were not contemplated with the deployment ofa system stated that they are willing to participate again in thenext editions of the PBL implementation.
Lesson
Another point to be considered was the feeling of ownershipthat students demonstrated regarding their systems/projects. We often hear from them: “
My system... ”, “
My code... ”, “
Myproduct... ”, suggesting that the outcome of projects would betheir property. During one of the monitoring activities, onestudent came with the following question: “
If the system looksgood, can we sell it afterward? ”. This question raised someconcerns, on the behalf of teachers, about the legal impli-cations of using a software developed by students. We thenrealized that we should have spent more time discussing in-stitutional regulations, ownership, software license, and termsof use with students and external stakeholders.
Lesson
This risk is associated with the aims and mo-tivations for: students, external stakeholders, and teachers.Initially, students are more concerned about academic achieve-ments and having good grades. External stakeholders arewilling to help with the educational approach, but they alsoexpect to benefit from this interaction to enhance their dailyactivities and processes. Teachers expect students to learn froma fruitful experience of developing an information system andif this process yields something useful, even better.As the projects evolved, we observed that students gotinvolved with the demands of the sectors, and a desire todeliver a product that was useful grew. Playing the roleof clients, external stakeholders presented some unrealisticdemands which were difficult for the students to understand.And some demands, although realistic, needed filtering tobe developed in time. Furthermore, it is difficult to mediatethe interaction between students and external stakeholdersensuring that these interactions are constructively aligned withthe course objectives [29]. To tackle these problems, themediating teacher had to act and often propose design changesto make projects viable within the deadline and the purposeof learning through projects. In the end, we could achievesatisfactory results in managing expectations, but it took muchmore time and effort from the mediator than expected.
Lesson
This risk is associated with the effort, time, orresources the external stakeholder can invest in the course.In our study, we initially took some time to select potentialprojects to participate in. We interviewed external stakeholdersinvolved to measure their availability and level of engagement.Nevertheless, we noticed that some dedicated themselves morethan others during the activities, for many different andunforeseen reasons. In project P6, for example, the externalstakeholder responsible told the students that he would beable to attend meetings and other monitoring activities onlyon Tuesday mornings. This clearly interfered with the progressof P6 activities. But this time constraint was established onlyafter the beginning of the course. It was not detected duringproject selection interviews. This was an indication that ourinterviews should be better structured and guided by clearerranking criteria. esson
This risk is associated with the institution or studyprogram related issues that are beyond the teacher’s control.To mitigate this threat, we adapted our implementation toinstitution deadlines, calendars, pedagogical course design andregulations, and other known events scheduled to happen.Some things though are hard to predict, e.g. , when we hadtwo project outcomes, from different groups, approved fordeployment in the same sector (Financial department - ProjectP2), and it was necessary to elaborate a new set of criteriain order to choose, as we presented in Section V-C. Anotherunexpected situation was the step four suspension due to theCOVID-19 pandemic, as we described in Section V-D. Whocould see that coming?
Lesson
This risk is associated with the interaction betweenexternal stakeholders and students, which is crucial for thesuccess of the course but hard to observe and control. Themediating teacher must monitor the students’ activities, al-though it is necessary to give them some independence duringactivities that require communication with those who are inthe role of customers/users. In Section V-A we showed thatteam T1 requested to change their project, and we had tocreate a new project (P7) for them. That was due to com-munication issues between team T1 and the user responsiblefor project P1. The teacher could act quickly in this case, butmore disagreements may likely have occurred in other teamsand projects without being reported by students or externalstakeholders. We invested in scheduling regular meetings andworkshops as monitoring activities to detect relationship prob-lems and provide valuable feedback to students. We believethat the communication issues involved are part of the realexperience we wanted to provide to students because theywill probably go through similar situations when they leavethe academy and go to industry. We could also observe somestudents who previously did not show interest starting to havea leadership position in the project. Overall, the interactionbetween students and external stakeholders provided valuablefeedback and contributed considerably to projects.
Lesson
This risk is associated with possible misalign-ment considering the goals of the different parties and thecourse. While the expectations of the external stakeholders,students, and of the teachers on their own can be perfectlyreasonable, compared to each other they can be misaligned.Students often expected clear-cut answers and solutions, whileexternal stakeholders may have unrealistic demands and nu-anced descriptions. To address potential misalignment of ex- pectations we invested in clear communication even beforeprojects start.One particular situation was observed during the devel-opment of projects. Sometimes students asked a practicalquestion to a collaborating teacher, inside the classroom, andthen asked the same question to the teacher mediator, gettingdifferent answers and solutions. If we consider for examplethe database, there are many ways we can arrange tablesand constraints among tables to solve the same problem.Students got confused when they got different solutions fromteachers. And then we realized that, in our approach, weshould have scheduled some moments in which mediating andcollaborating teachers could discuss technical issues relatedto the projects, without sharing them with the students orexternal stakeholders because it probably would influence theirdecisions. The benefit would be only to align teachers’ speech.
Lesson
VII. R
ELATED W ORK
Existing projects have evaluated the application of PBLmethods to promote SE education and focused on differentmethods [6], [7], [10], [11], [30]. However, to our knowledge,none of them divide activities among students from differentprograms, at different levels of SE education, and have aconsistent plan to avoid pitfalls highlighted in the literaturewhen involving external stakeholders in academic projects.Benedetto and Navon [25] present an approach to exploitingteam shuffling dynamics in a formal software design course toconvey the importance of SE concepts, making students awareof the practical value of design-related activities in a systemdevelopment process. They conducted an empirical study inwhich they analyzed students performance during softwaredesign classes. Evidence collected by the qualitative analysisof predefined one-on-one interviews showed that students hada better understanding of the concepts taught with the teamshuffling approach than before.Our work also benefits from this idea of team shuffling whenundergraduate students receive the artifacts produced in stepone by other teams, and have to take over ongoing projects.Souza et al. [4] perform an opinion survey study to eval-uate the students’ perception of the adoption of PBL in SEeducation. They use questionnaires to collect responses of 49undergraduate students divided into two samples: 32 enrolledin an introductory SE course that uses PBL, and 17 studentsenrolled in a SE course that adopts a traditional teacher-centered learning method (non-PBL). The PBL principlesadopted include projects based on real-world problems, inwhich the instructors play the role of customers during thecourse. Among the results, in general, students agree that itis important to use practical software development projects inthe context of SE education, and that two of the most recurringnegative aspects pointed by the non-PBL sample was the lackof orientation in activities and the lack of classroom activitiesto support the development of projects.urden et al. [13] report a case study on how the integrationof entrepreneurial experiences into a software engineeringproject course can benefit students to increase their team-work skills and competencies. They discuss how to implemententrepreneurial experiences that focus on taking action andmanaging resources in agile software projects, making it possi-ble to other SE educators to understand and even adopt specificcourse design aspects. Regarding SE education, one of thebiggest challenges they encountered lies in finding appropriatestakeholders with whom the students can collaborate and whoare able and willing to invest necessary resources.We tackle the challenges they describe when we use uni-versity staff members to play the role of clients, and our studyshowed that it is possible to avoid known problems related tothe involvement of external stakeholders in academic projects.Delgado and Aponte [9] analyze the evolution of a PBLcourse in SE for undergraduate students. They monitor soft-ware project repositories and the students’ feedback during aperiod of six semesters, to investigate how the adoption of aPBL method affects students’ grades and their project activ-ities. Among the benefits, they conclude that the monitoredstudents improved their grades and were able to add morefunctionalities in their applications. On the other hand, oncethe students had more time for improving the features in theirapplications, at the expense of close monitoring of internalcode quality, it resulted in an increase in technical debt.Stegh¨ofer et al. [12] develop a model that allows analyzingthe involvement of external stakeholders in university courses.They obtain insights from past course instances and use themto identify potential risks and benefits in future projects.They apply the model in eight courses in SE programs, eachprogram using a different strategy to select the stakeholders.One of these strategies is the use of university employeesacting as customers. After applying the model, they show thatthe students tend to take the projects more seriously whenexternal stakeholders are involved, and that guest lectures andsupervision can also decrease the workload for the teachers.However, there are challenges that affect students negatively.The authors observe that: (i) students are frustrated whenstakeholders use different terminology than teachers; (ii) stake-holders are frustrated when students focus more on academicachievements than on fulfilling their needs; (iii) teachers arefrustrated when students do not take the insights expectedfrom the experience. Specifically in the case when universityemployees are used, they highlight that a plan for softwaremaintenance is needed to enable its use. Therefore, the uni-versity’s IT staff must be involved before preparing the courseproject description. And users from different areas might haveunrealistic requirements that can confuse the students.VIII. F
INAL R EMARKS
This paper described a case study integrating students fromdifferent SE programs and involving external stakeholders,underpinned by project-based learning. In this study, wedivided activities of a software development process amongthe students, at different levels of SE education, and used a consistent plan to avoid some pitfalls highlighted in theliterature when involving external stakeholders in academicprojects. We presented how this study was designed andimplemented in a large institution, in four steps, summarizedas follows: (i) step one started in the second semester of2018 and it was mostly dedicated to requirements gatheringand design; (ii) step two initiated in the first semester of2019 and it was dedicated to information system developmentand implementation; (iii) step three started in the secondsemester of 2019 and it was intended for integration tests anddeployment processes; (iv) and step four, which was suspendeddue to the coronavirus pandemic, encompass support andmaintenance activities for the deployed systems. The casestudy implementation had the participation of 59 students froma professional technical course in step one, working in teams,and 10 undergraduate students from a Bachelor program inInformation Systems in the following steps, guided by pairprogramming techniques. Although we use different programswith students having different levels and learning objectives,each program takes part separately in one of the four stepsdescribed in our approach, avoiding problems that could arisefrom the heterogeneity of the students involved.We followed the idea of team shuffling when undergraduatestudents received the outcomes produced in step one by otherteams. It is important to highlight some benefits of thisapproach: (i) since the end of project activities, in each step,has to be aligned with the end of the subjects involved in theprogram, if the students had to take part in all developmenttasks they probably would not be able to complete the softwareand reach the deployment step; (ii) the opportunity to analyzeand understand artifacts produced by others stands out theimportance of the quality of the artifacts, e.g. , application ofbest practices in source code, concise and clear documentation,and detailed diagrams. In addition, the need to communicatewith external stakeholders is present in all steps, and not justduring the requirements gathering.The involvement of external stakeholders in project courseswas considered positive and produced very good results, butit came with some challenges, such as misalignment betweenstakeholders and students or difficulties to manage expecta-tions. We designed and implemented our PBL approach toovercome these known challenges and provide a beneficialexperience for the students in SE education. Overall, the feed-back from stakeholders and students exceeded expectations,although it increased the workload of teachers. We were ableto distill a set of lessons learned, some of which are echoedin related literature. We expect that at least some of them willbe useful for anyone implementing a similar course.As a consequence of this study, we plan to institutionallyformalize the PBL course improvement process by definingspecific outcomes and measurements. Moreover, the creationof the junior enterprise for the Bachelor program in Informa-tion Systems suspended due to the pandemic crisis, in stepfour, will be used to meet other external community demandsfor software development, in addition to maintenance andsupport routines for the institution’s internal systems.
EFERENCES[1] M. Feathers,
Working Effectively with Legacy Code . Prentice Hall PTR,2004.[2] A. Fox and D. Patterson,
Engineering Software as a Service:An Agile Approach Using Cloud Computing . Strawberry CanyonLLC, 2013. [Online]. Available: https://books.google.com.br/books?id=3kqjmwEACAAJ[3] M. R. Marques, A. Quispe, and S. F. Ochoa, “A systematic mappingstudy on practical approaches to teaching software engineering,” in
IEEEFrontiers in Education Conference (FIE) , 2014, pp. 1–8.[4] M. Souza, R. Moreira, and E. Figueiredo, “Students perception on theuse of project-based learning in software engineering education,” in
XXXIII Brazilian Symposium on Software Engineering (SBES) . As-sociation for Computing Machinery (ACM), 2019, p. 537–546.[5] M. Ardis, D. Budgen, G. W. Hislop, J. Offutt, M. Sebern, and W. Visser,“SE 2014: Curriculum guidelines for undergraduate degree programs insoftware engineering,”
Computer , vol. 48, no. 11, pp. 106–109, 2015.[6] P. Blumenfeld, E. Soloway, R. Marx, and J. Krajcik, “Motivatingproject-based learning: Sustaining the doing, supporting the learning,”
Educational Psychologist , vol. 26, pp. 369–398, 2011.[7] N. Erdogan and T. D. Bozeman,
Models of Project-Based Learning forthe 21st Century . SensePublishers, 2015, pp. 31–42.[8] M. Shuto, H. Washizaki, K. Kakehi, Y. Fukazawa, S. Yamato, andM. Okubo, “Learning Effectiveness of Team Discussions in VariousSoftware Engineering Education Courses,” in
IEEE 29th Conferenceon Software Engineering Education and Training (CSEE&T) , 2016, pp.227–231.[9] D. Delgado, A. Velasco, J. Aponte, and A. Marcus, “Evolving a project-based software engineering course: A case study,” in
IEEE 30th Con-ference on Software Engineering Education and Training (CSEE&T) ,2017, pp. 77–86.[10] C. R. Rupakheti, M. Hays, S. Mohan, S. Chenoweth, and A. Stouder,“On a pursuit for perfecting an undergraduate requirements engineeringcourse,” in
IEEE 30th Conference on Software Engineering Educationand Training (CSEE&T) , 2017, pp. 97–106.[11] M. Marques, S. F. Ochoa, M. C. Bastarrica, and F. J. Gutierrez,“Enhancing the student learning experience in software engineeringproject courses,”
IEEE Transactions on Education , vol. 61, no. 1, pp.63–73, 2018.[12] J. Stegh¨ofer, H. Burden, R. Hebig, G. Calikli, R. Feldt, I. Hammouda,J. Horkoff, E. Knauss, and G. Liebel, “Involving external stakeholdersin project courses,”
ACM Trans. Comput. Educ. , vol. 18, no. 2, pp. 8:1–8:32, 2018.[13] H. Burden, J. Stegh¨ofer, and O. H. Svensson, “Facilitating en-trepreneurial experiences through a software engineering project course,”in . IEEE / ACM, 2019, pp. 28–37.[14] D. Boud and G. Feletti,
The Challenge of Problem-based Learning .Kogan Page, 1998. [Online]. Available: https://books.google.com.br/books?id=z0c9AAAAIAAJ[15] S. Sahyar, R. Sani, and T. Malau, “The effect of problem based learning(PBL) model and self regulated learning (SRL) toward physics problemsolving ability (PSA) of students at senior high school,”
AmericanJournal of Educational Research , vol. 5, no. 3, pp. 279–283, 2017.[16] W. Bender,
Project-Based Learning: Differentiating Instruction forthe 21st Century . SAGE Publications, 2012. [Online]. Available:https://books.google.com.br/books?id=ZtmzMNZYoXUC[17] A. M. C. A. Oliveira, S. C. dos Santos, and V. C. Garcia, “PBL inteaching computing: An overview of the last 15 years,” in
IEEE Frontiersin Education Conference (FIE) . IEEE Computer Society, 2013, pp.267–272.[18] M. Jazayeri, “Combining mastery learning with project-based learningin a first programming course: An experience report,” in .IEEE Computer Society, 2015, pp. 315–318.[19] G. H. S. Alexandre, S. C. dos Santos, A. N. Rodrigues, and P. B. Souza,“Applying and managing PBL - an experience in information systemseducation,” in . SciTePress, 2018, pp. 57–67.[20] H. Jung, W. Jun, and L. Gruenwald, “A design and implementationof web-based project-based learning support systems,” in
First Interna- tional Conference on The Human Society and the Internet - InternetRelated Socio-Economic Issues . Springer-Verlag, 2001, p. 354–368.[21] J. Almeida, A. D. Daniel, and C. Figueiredo, “The future of man-agement education: The role of entrepreneurship education and juniorenterprises,”
The International Journal of Management Education , pp.100–318, 2019.[22] S. Sengupta and S. Bhattacharya, “Formalization of UML use casediagram-a z notation based approach,” in
International Conference onComputing Informatics , 2006, pp. 1–6.[23] L. Williams and R. Kessler,
Pair Programming Illuminated . USA:Addison-Wesley Longman Publishing Co., Inc., 2002.[24] K. Beck and C. Andres,
Extreme Programming Explained: EmbraceChange (2nd Edition) . Addison-Wesley Professional, 2004.[25] J. I. Benedetto and J. Navon, “Exploiting group shuffling dynamics toconvey the importance of good software design,” in
ACM/IEEE 42ndInternational Conference on Software Engineering: Software Engineer-ing Education and Training (ICSE-SEET) . Association for ComputingMachinery, 2020, p. 193–196.[26] K. S. Rubin,
Essential Scrum: A Practical Guide to the Most PopularAgile Process , 1st ed. Addison-Wesley Professional, 2012.[27] E. Brechner,
Agile Project Management with Kanban , 1st ed. USA:Microsoft Press, 2015.[28] J. E. Montandon, C. Politowski, L. L. Silva, M. T. Valente, F. Petrillo,and Y.-G. Gu´eh´eneuc, “What skills do IT companies look for innew developers? a study with Stack Overflow Jobs,”
Information andSoftware Technology , vol. 1, pp. 1–6, 2020.[29] J. Biggs, “Enhancing teaching through constructive alignment,”
HigherEducation , vol. 32, pp. 347–364, 10 1996.[30] F. V. Binder, R. Albuquerque, S. Reinehr, and A. Malucelli, “Innovationand active learning for training mobile app developers,” in