Theo Bastiaens

A Five-Dimensional Framework for Authentic Assessment

Authenticity is an important element of new modes of assessment. The problem is that what authentic assessment really is, is unspecified. In this article, we first review the literature on authenticity of assessments, along with a five-dimensional framework for designing authentic assessments with professional practice as the starting point. Then, we present the results of a qualitative study to determine if the framework is complete, and what the relative importance of the five dimensions is in the perceptions of students and teachers of a vocational college for nursing. We discuss implications for the framework, along with important issues that need to be considered when designing authentic assessments.

The Impact of Intrinsic Motivation on E-Learning in Authentic Computer Tasks

Students with high intrinsic motivation often outperform students with low intrinsic motivation. However, little is known about the processes that lead to these differences. In education based on simulations or authentic electronic learning environments, this lack of insight is even more clear. The present study investigated what students actually did in an electronic learning environment that was designed as a game-like realistic simulation in which students had to play the role of a junior consultant. The results show that students with high intrinsic motivation did not do more, rather they tended to do different things. Analysis of log files showed that the increased curiosity that students with high intrinsic motivation have, resulted in proportionally more explorative study behaviour. However, the learning outcomes of students with high intrinsic motivation were not better.

The Boundary Approach of Competence: A Constructivist Aid for Understanding and Using the Concept of Competence

Although competence is an important concept in human resource development and education, there is no theoretical framework for competence. This article focuses on the development of such a theoretical framework. It proposes the boundary approach ofcompetence, an aid to support human resource managers and educationalists in thinking about the concept of competence and in defining it properly. Here, the concept of competence is being explored by focusing on its dimensions and by identifying differences with related terms. The boundary approach of competence heavily depends on a constructivist point ofview. This holds that the quest for one absolute meaning of competence is being abandoned and that instead competence definitions are being valued against their degree of viability. This article proposes three variables for enhancing viability: people, goal and context.

New Learning Design in Distance Education: The impact on student perception and motivation

Many forms of e‐learning (such as online courses with authentic tasks and computer‐supported collaborative learning) have become important in distance education. Very often, such e‐learning courses or tasks are set up following constructivist design principles. Often, this leads to learning environments with authentic problems in ill‐structured tasks that are supposed to motivate students. However, constructivist design principles are difficult to implement because developers must be able to predict how students perceive the tasks and whether the tasks motivate the students. The research in this article queries some of the assumed effects. It presents a study that provides increased insight into the actual perception of electronic authentic learning tasks. The main questions are how students learn in such e‐learning environments with “virtual” reality and authentic problems and how they perceive them. To answer these questions, in two e‐learning programs developed at the Open University of the Netherlands (OUNL) designers’ expectations were contrasted with student perceptions. The results show a gap between the two, for students experience much less authenticity than developers assume.

The surplus value of an authentic learning environment

This article presents a study that provides insight in the effects of an authentic electronic learning environment on student performance and experiences. It is expected that learning in an authentic learning environment results in higher performance and improves intrinsic motivation of students. The results of this study showed, contrary to what was expected, that student who worked in an authentic environment did not perform better than students who worked in a less authentic environment. Moreover, the reported experiences with the learning environments did not differ between both groups.

The training of peer assessment skills to promote the development of reflection skills in teacher education

Effective assessment approaches, based on constructivist views, receive special attention in current innovations in higher education. These assessment approaches promote integration of assessment and instruction, seeing the student as an active person who shares responsibility, reflects, collaborates and conducts a continuous dialogue with the teacher. Assessment has no longer purely the function of crediting students with recognized certificates but is above all valuable for the monitoring of students’ progress and to support them in improving their learning activities. The emphasis shifts to a representation of assessment as a tool for learning (Arter, 1996; Boud, 1990, 1995; Dochy & McDowell, 1997). Assessment is an important issue in the current developments towards more studentcentered learning. Students are more responsible for their own learning process and are increasingly regarded as active participants in instructional activities. An assessment approach has to be chosen that is in alignment with the learning goals of students. To realize the implementation of assessment as a learning tool, a number of changes are desirable on different levels in the organization of institutions in higher education such as the level of the student, the level of the teachers, and the management level. In this article the level of student teachers is addressed. Specifically, the role of student teachers as assessors of their own work and that of peers is investigated. There is an increasing demand for self and peer assessments in teacher training colleges, because these forms of assessment fit in well with the latest view on the education

Authentic assessment, student and teacher perceptions: the practical value of the five‐dimensional framework

This study focused on determining the facets of assessment authenticity by exploring the perceptions of both students and teachers of vocational education and training. It elaborates on a theoretical five‐dimensional framework (5DF) that differentiates between five dimensions and several sub‐elements of authenticity. This framework led to the development of a questionnaire for examining if the facets of the 5DF are recognised by students and teachers in practice. Reliability and factor analysis as well as readability scores were used. Teachers recognised both the dimensions and the sub‐elements as facets that determine assessment authenticity. In the eyes of the students, four of the five dimensions (task, physical context, form and result/criteria) determine authenticity, while students do not perceive the social context as a characteristic of assessment authenticity, nor do they differentiate the several sub‐elements. Implications for using the 5DF to develop or evaluate authentic assessments are discussed.

Authenticity is in the eye of the beholder: student and teacher perceptions of assessment authenticity

In vocational education and training (VET) in the Netherlands, learning and working are integrated from the start. Authentic assessments are used during competence‐based VET curricula to achieve correspondence between learning and working. The premise behind this study is that authenticity is subjective and that perceptions of assessment authenticity influence student learning for the assessments. It examines whether students and teachers differ in their perceptions of the authenticity of various assessment characteristics. Subsequently this study investigates whether freshman and senior students, who differ in their amount of practical experience, differ in their perceptions of assessment authenticity. The main findings were that teachers rated most assessment characteristics as more authentic than students did, while freshman and senior students did not differ in their perception of authenticity. Implications deal with communicating about and developing authentic assessment in the eyes of both students and teachers to stimulate students’ professional skills development during a VET curriculum

Development of the Teacher Feedback Observation Scheme: evaluating the quality of feedback in peer groups

Research suggests that feedback is an essential element in learning. This study focuses on feedback that teachers provide in reciprocal peer groups to improve their performance in the classroom. The Teacher Feedback Observation Scheme (TFOS) was developed to identify feedback patterns, which approaches feedback as a multidimensional process. The TFOS helps acquire insights into the effectiveness of feedback, and provides information regarding the situations in which possible interventions can be undertaken if feedback is declining and becoming ineffective. This may especially be necessary when the communication of feedback is mediated by information and communications technology (ICT). The TFOS was piloted using videotaped sessions of three face-to-face groups, as well as one virtual group, using discussion wikis. All four groups of teachers used the Video Intervision Peer-coaching (VIP) procedure. The findings reveal that feedback in the virtual group was less effective than it was in the face-to-face groups. In addition, ineffective feedback patterns in the face-to-face groups transitioned into more effective feedback patterns. The TFOS appears to be adept at identifying feedback patterns in peer groups.

Assessing an electronic performance support system for the analysis of jobs and tasks

In this research project the concept of ‘Electronic Performance Support Systems’ will be introduced. An Electronic Performance Support System (EPSS) is an integrated computerised environment that supports and occasionally monitors employees while doing their jobs. In general an EPSS contains the following four components: tools (to perform a job), information (needed to do the job correctly), advice (for the difficult parts of the job) and training (to extend the employees knowledge and skills). In this study the effectiveness of an EPSS for instructional designers is reported.