Esther Winther

Workplace-based competence measurement: developing innovative assessment systems for tomorrow’s VET programmes

As a consequence of the large-scale assessment studies (TIMMS; PISA) in compulsory schooling, attention is now being given to the modelling and measurement of competencies in initial vocational education and training. This new output-led perspective of teaching/training and learning/working processes demands new approaches to research. Using the concept of the curriculum–instruction–assessment triad this paper argues that competencies in the fields of business and commerce education can be determined and compared internationally via a large-scale assessment. Empirical results are presented that demonstrate the possibilities of running such an international study in VET.

Measurement of vocational competences: an analysis of the structure and reliability of current assessment practices in economic domains

BackgroundBoth fostering and measuring action competence remain central targets of vocational education and training research; adequate measurement approaches clearly are prerequisites for international, large-scale assessments. For the German Chamber of Commerce and Industry, competence assessments of industrial managers rely mainly on final examinations that attempt to measure not just knowledge but also action competence. To evaluate this test instrument, this article considers two questions: (1) Can the test assess action competence with validity, and (2) how reliable are the corresponding assessment results?MethodsThe study relied on statistical procedures (e.g., IRT scaling), applied empirically to a sample of 1,768 final examinations.ResultsAs a result the current examination appears neither adequate nor accurate as an instrument to capture action competence.ConclusionsWe conclude that several improving steps have to be undertaken to improve the economic assessment.

Assessing Tomorrow’s Potential: A Competence Measuring Approach in Vocational Education and Training

Adequate measurement of action competence remains a central target of vocational education and training research; adequate measurement approaches in the vocational domain clearly are a prerequisite for accountable systems to authorize access to professional activities, as well as for future large-scale assessments. For the German Chamber of Commerce and Industry, competence assessments in the area of business and commerce rely mainly on final examinations that attempt to measure not just knowledge but also action competence. To evaluate and improve a test instrument, this chapter considers two questions: (1) how valid and reliable was the original test-format, and (2) how valid and reliable are the corresponding assessment results of a recently developed prototype? The study relies on statistical procedures (e.g., IRT scaling), applied empirically to a sample of 1768 final examinations of industrial managers in the original format, and to 479 industrial managers taking a prototype new format. The advanced prototype version appears as a more valid and accurate instrument to capture action competence. We conclude that several practical steps can be undertaken to improve current assessment practices in the area of business and commerce.

Designing Competence Assessment in VET for a Digital Future

This chapter gives an overview of the design, implementation, and evaluation of a technology-based assessment (TBA) measuring trainee competencies in the field of economics. Current research on “Industry 4.0” or “digital natives” demonstrates how quickly technological advances change the working world and how important information and communications (ICT) skills already are in the workplace. Digitalization not only entails new requirements for working but also has far-reaching implications for vocational learning and vocational educational design. In line with the idea of triadic assessment (Pellegrino (2012) The design of an assessment system focused on student achievement: A learning sciences perspective on issues of competence, growth and measurement. In: Bernholt, Neumann and Nentwig (eds) Making it tangible. Learning outcomes in science education. Waxmann, Munster, pp 79–107; Pellegrino et al., (2003) Knowing what students know: the science and design of educational assessment. National Academy Press, Washington, DC), this leads to a growing need for appropriate assessments to be aligned with curriculum and instruction. Hence, we introduce an instrument that aims to bridge the gap between work reality and assessment, using a prototype of an authentic workplace simulation. More specifically, this technology-based tool visualizes work and business processes in an industrial enterprise and requires test-takers to act and make decisions in computer-based tasks (e.g., editing text files and tables, answering emails, or dealing with daily test statistics). In order to reflect digitalized workplace processes, technological aspects of real work environments must be didactically modeled and installed as usable features. Thus, the usability of the installed features is crucial, as technical authenticity constitutes a basic requirement of valid assessment. For the purpose of testing the usability of this simulation, a think-aloud study of the user interface was conducted with a total of N = 37 business education students.

Large-Scale Assessment of Vocational Education and Training

Our study is part of a European endeavor to measure effects of workplace learning by an international large-scale assessment in the fields of business and administration until 2020. We demonstrate how job-typical workplace tasks can be identified on a European level and, then, assessed at a web-based virtual workplace, which provides a broad spectrum of authentic tasks that typically have to be worked out at real workplaces and form the basis of assessment. By a field study – as a first step – we proved our assumptions, won data on the fulfillment of typical actions of clerks and used item response theory for evaluating a corresponding competence model as basis for a large-scale assessment of the effects of workplace learning.