Walter W. Schilling

Assessing the effectiveness of video feedback in the computing field

Engineering students exhibit a wide array of learning styles across the perception, input, organization, processing, and understanding dimensions. To improve students performance in the classroom, many techniques have been developed to address these variances. The computing fields, however, tend to have a large percentage of students who are visual learners. These students learn best by seeing, and they can perform very well in the classroom with the appropriate usage of teaching styles. However, when it comes to providing feedback to students on submitted assignments, the main method employed is the written comment, which is not conducive to visual learners. This method is most prevalent in the academic community because overall, it is the simplest form of feedback that a faculty member can provide to students. However, written feedback is often highly ineffective at improving student performance, as many students simply do not read the comments because the students feel they are not relevant to their performance. This paper presents an assessment of an alternative method for providing feedback to students: video feedback. In lieu of written feedback, students are provided feedback for software engineering exercises through the use of a short video made via video capture. The video captures in multimedia format the instructors perceptions and actions when grading a given assignment. The video includes both aural commentary as the assignment is assessed, as well as dynamic visuals of the grading process, demonstrating failures and improvements that can be made in the submitted assignment. The article describes the pedagogical foundation for the technique, specifics of the technique used, student perceptions of the technique, and an assessment of the learning gains from using such a method in a junior level class. In general, students are show to prefer the technique versus traditional grading, and an improvement in overall outcomes for the course is shown to exist as well.

A methodology for quantitative evaluation of software reliability using static analysis

This paper proposes a methodology for quantitative evaluation of software reliability in updated COTS or Open Source components. The model combines static analysis of existing source code modules, limited testing with execution path capture, and a series of Bayesian Belief Networks. Static analysis is used to detect faults within the source code which may lead to failure. Code coverage is used to determine which paths within the source code are executed as well as their execution rate. A series of Bayesian Belief Networks is then used to combine these parameters and estimate the reliability for each method. A second series of Bayesian Belief Networks then combines the module reliabilities to estimate the net software reliability. A proof of concept for the model is provided, as the model is applied to five different open-source applications and the results are compared with reliability estimates using the STREW (Software Testing and Early Warning) metrics. The model is shown to be highly effective and the results are within the confidence interval for the STREW reliability calculations, and typically the results differed by less than 2%. This model offers many benefits to practicing software engineers. Through the usage of this model, it is possible to quickly assess the reliability of a given release of a software module supplied by an external vendor to determine whether it is more or less reliable than a previous release. The determination can be made independent of any knowledge of the developers software development process and without any development metrics.

Improving Student Understanding with Video Grading

It is known that students exhibit different learning styles. Good instructors adapt their teaching style to target the appropriate style(s) for their students, and by doing so achieve significant improvements in student outcomes. The same approach, however, does not hold true for submitted assignments. Due to logistics, most submitted assignments are graded in the same fashion: providing numeric feedback (i.e. a grade) and written comments. This article describes a different approach, an approach which is designed to aid visual and audible learners to obtain better feedback from submitted assignments.

Analyzing the impact of asynchronous multimedia feedback on novice computer programmers

For many engineering students, freshman programming represents one of the hardest courses for them to master. Unlike other science fields, few students are routinely exposed to programming in the K12 system. This can make the freshman programming course daunting. However, in the field of software engineering, success in this area is vital, as success in nearly all future courses requires mastery of this skillset. In the engineering field, we find that many students are visual learners. These students learn best by seeing, and they can perform very well in the classroom with the appropriate usage of teaching styles. However, when it comes to providing feedback to students on submitted assignments, the main method employed is the written comment, which is not conducive to visual learners. From a faculty members standpoint, this makes sense, as it is the simplest form of feedback. However, written feedback is often ineffective at improving student performance, as many students simply do not read the comments because the students feel they are not relevant to their performance. This can be compounded in the freshman year, as students are still learning what is meant to be an effective college student. At higher levels, an alternative feedback mechanism, namely asynchronous multimedia feedback, has shown great promise. In lieu of written feedback, students are provided feedback for software engineering exercises through the use of a short video made via video capture. The video captures in multimedia format the instructors perceptions and actions when grading a given assignment. The video shows, in real time, what the instructor saw, whether it is a program crashing or the successful operation of the program. Furthermore, it provides the instructor the ability to potentially fix simple blatant errors and see the instructors debugging strategy. The article describes the pedagogical foundation for the technique, specifics of the technique used, student perceptions of the technique, and an assessment of the learning gains from using such a method in an introductory freshman programming course. In general, students are show to prefer the technique versus traditional grading, and a statistically significant improvement in overall outcomes for the experimental course is shown to exist. A statistically significant correlation between the watching of videos and outcomes is also shown.