Viola Katharina Klotz

Measurement of vocational competences: an analysis of the structure and reliability of current assessment practices in economic domains

BackgroundBoth fostering and measuring action competence remain central targets of vocational education and training research; adequate measurement approaches clearly are prerequisites for international, large-scale assessments. For the German Chamber of Commerce and Industry, competence assessments of industrial managers rely mainly on final examinations that attempt to measure not just knowledge but also action competence. To evaluate this test instrument, this article considers two questions: (1) Can the test assess action competence with validity, and (2) how reliable are the corresponding assessment results?MethodsThe study relied on statistical procedures (e.g., IRT scaling), applied empirically to a sample of 1,768 final examinations.ResultsAs a result the current examination appears neither adequate nor accurate as an instrument to capture action competence.ConclusionsWe conclude that several improving steps have to be undertaken to improve the economic assessment.

Using Digital Media to Assess and Promote School and Adult Education Teacher Competence

Classrooms are informational environments that are highly complex in nature. Teachers need to make use of many different information sources in the classroom in order to adapt to these environments, perform successfully, and develop competence. Hence, effective design research in teacher education has to take this high complexity into account too, for instance by using video technologies. In the chapter presented here, we describe a research program that uses the advantages of digital media to assess and promote teacher competence. A web-based learning and testing environment was designed and implemented in several studies with school and adult education teachers. We successfully developed a test instrument to measure the pedagogical/psychological knowledge of teachers in the two chosen educational contexts via the web-based environment. Results of several intervention studies indicated that the web-based learning and testing environment with integrated video case studies is a powerful tool to promote taking on new perspectives and applying knowledge among school and adult education teachers. Implications for research, practice, and the required web-based infrastructure are discussed.

Assessing Tomorrow’s Potential: A Competence Measuring Approach in Vocational Education and Training

Adequate measurement of action competence remains a central target of vocational education and training research; adequate measurement approaches in the vocational domain clearly are a prerequisite for accountable systems to authorize access to professional activities, as well as for future large-scale assessments. For the German Chamber of Commerce and Industry, competence assessments in the area of business and commerce rely mainly on final examinations that attempt to measure not just knowledge but also action competence. To evaluate and improve a test instrument, this chapter considers two questions: (1) how valid and reliable was the original test-format, and (2) how valid and reliable are the corresponding assessment results of a recently developed prototype? The study relies on statistical procedures (e.g., IRT scaling), applied empirically to a sample of 1768 final examinations of industrial managers in the original format, and to 479 industrial managers taking a prototype new format. The advanced prototype version appears as a more valid and accurate instrument to capture action competence. We conclude that several practical steps can be undertaken to improve current assessment practices in the area of business and commerce.