Frank Musekamp

Assessing Engineering Competencies: The Conditions for Educational Improvement.

Low-stakes assessment is supposed to improve educational practice by providing feedback to different actors in educational systems. However, the process of assessment from design to the point of a final impact on student learning outcomes is complex and diverse. It is hard to identify reasons for substandard achievement on assessments, let alone the means that should be undertaken to improve the educational setting. Furthermore, it is difficult to show a causal link between educational reforms and change in test achievement over time. This paper examines the potential impact that low-stakes testing initiatives have in engineering higher education on educational improvement by examining two case studies. It discusses how the design of constructs and particular assessment conditions may foster or limit educational improvement and outlines conditions to ensure that low-stakes assessment designs achieve educational improvement.

Student motivation in low-stakes assessment contexts: an exploratory analysis in engineering mechanics

The goal of this paper is to examine the relationship of student motivation and achievement in low-stakes assessment contexts. Using Pearson product-moment correlations and hierarchical linear regression modelling to analyse data on 794 tertiary students who undertook a low-stakes engineering mechanics assessment (along with the questionnaire of current motivation and the ‘Effort Thermometer’), we find that different measures of student motivation (effort, interest, sense of challenge, expectancy of success and anxiety to fail) showed atypical correlation patterns. The nature of the correlations further varies depending on the type of test booklet used by students. The difficulty of the early items in the assessment were positively correlated with ‘anxiety’ and ‘success’, but negatively correlated with ‘interest’. In the light of our findings, we suggest that future research should systematically explore (taking into account testing conditions like test booklet design and test-item format) the implications of student motivation for achievement in low-stakes assessment contexts. Until the consequences of these processes are better understood, the validity of assessment data generated in low-stake conditions in the higher education sector will continue to be questioned. With a greater understanding of these processes, steps could be taken to correct for student motivation in such settings, thus increasing the utility of such assessments.

Competence Modeling and Measurement in Engineering Mechanics

Since the implementation of the Bologna process, there has been more attention to the quality of engineering study courses and to necessity of measuring the extent to which these courses facilitate and support the acquisition of engineering competence of students. So far it is not possible to assess this in a reliable and valid way because of missing models of competence. The foundation of all competence diagnostics is (1) a model of competencies which need to be developed, and (2) the subsequent development and application of adequate test instruments. This chapter proposes a competence model as the basis for an assessment of learning results in mechanical engineering, and places it within the current state of empirical educational research. Hypotheses on the competence structure and on the levels of mechanical engineering students will be derived from the model. The outcome can be used as a step forward for more transparency of the teaching processes and how teaching might be optimized by using the results of assessments.