Mikel Villamañe

Dealing with common problems in engineering degrees' Final Year Projects

After the implementation of the Bologna Process, the Final Year Project (FYP) is a mandatory course in all bachelor degrees in Spanish higher education system. Although the FYP is new for many degrees, it was already established in the previous education system for engineering degrees. The authors of this paper are lecturers in a bachelor degree in Computer Science at three different colleges of the University of the Basque Country UPV/EHU. In the last few years, they have supervised more than 80 FYPs and they have all faced similar problems during the supervision process. This paper presents a study conducted to determine if the observed problems are just a perception of the authors or whether they are general problems in the elaboration of FYPs. In the study, 60 former students and 27 lecturers took part. Four main categories of problems have been identified and a proposal to overcome them is presented.

Supporting complex assessment processes

Assessment is a key element in the teaching-learning process. Every formal education implies some kind of assessment, and the results of this assessment contribute to the quality improvement in that process. Currently, most assessment scenarios are very complex and require tools to support them. Establishing a good assessment approach requires continuous decision-making processes to guarantee fair assessments. In this paper the TraMeA assessment support system is presented centering on its characteristics that can help in those decision making processes. The proposal is exemplified with real examples derived from the evaluation of Final Year Projects.

Tramea, a tool to support the development of Final Year Projects. Usability evaluation

Carrying out and monitoring Final Year Projects are challenging processes for all the actors involved: lecturers and students. Those projects exhibit problems regarding their evaluation, supervision and the lack of appropriate communication channels between the actors. The use of information and communication technologies can help to overcome those problems. This paper presents TraMeA, a system developed to support both lecturers and students and lighten the problems mentioned above. The usability evaluation conducted on the tool and its results are also described.

Recommending Dimension Weights and Scale Values in Multi-rubric Evaluations

Rubrics are scoring tools that lay out the specific expectations for an assignment. They are very appropriate tools for formative assessment as they have proved to be adequate to reduce subjectivity in the evaluation process. When the evaluation entails several tasks, a rubric for each task should be defined. However, computing the final score using rubrics is not always a simple task. On the one hand, each task has its own relevance in the final grade. On the other hand, the score of each rubric depends on the performance levels achieved in each dimension and the importance or weight of each dimension. Determining the most appropriate weight for each task, dimension and performance level is complex. This paper presents a recommender for settling those values in a multi-rubric evaluation process.

Rating monitoring as a means to mitigate rater effects and controversial evaluations

Assessment is a key aspect of any instructional process, as it is the main mean of determining the competences of the students. Nowadays, the assessment and scoring carried out by groups of reviewers, namely evaluation of final year project or thesis works or even peer evaluation, are becoming more and more frequent. However, the assessment and scoring of a work in such scenarios can be affected by each raters thinking processes, knowledge level and personal preferences among other issues. These idiosyncrasies are known as rater effects and can dramatically affect the evaluation process. Although many works point out that the use of certain evaluation instruments, e.g., evaluation rubrics, can increase the fairness and impartiality of the evaluation, rater effects may be still present and remarkably affect the scoring. Furthermore, some works might present controversy on their assessment, i.e., the evaluators of a certain work might strongly disagree on its quality. Therefore, the identification of the rater effects and controversial evaluations is crucial to be able to take remediation actions and to guarantee a fair evaluation. However, this identification process is often hard for scoring leaders. Consequently, tools that help leaders in this process are necessary. This paper presents the visualizations used by RaMon (a system for monitoring raters and controversial evaluations) to help the monitoring process, along with the support it provides to take remediation actions.

RubricVis: enriching rubric-based formative assessment with visual learning analytics

The implementation of the Bologna Process has implied, in many Spanish universities, the introduction of continuous evaluation processes, in which formative assessment has a great relevance. In such kinds of scenarios, rubrics provide many benefits that can remarkably be improved using Visual Learning Analytics techniques. This paper presents some of the visualization capabilities provided by RubricVis, a system that uses Visual Learning Analytics techniques to enhance rubric-based formative assessment. The objective of these visualizations is to enrich the feedback for both students and teachers in such evaluation environments. This feedback will help users to understand the learning and evaluation processes in order to improve them.