Miriam Toepper

Valid Competency Assessment in Higher Education

The aim of the 15 collaborative projects conducted during the new funding phase of the German research program Modeling and Measuring Competencies in Higher Education—Validation and Methodological Innovations (KoKoHs) is to make a significant contribution to advancing the field of modeling and valid measurement of competencies acquired in higher education. The KoKoHs research teams assess generic competencies and domain-specific competencies in teacher education, social and economic sciences, and medicine based on findings from and using competency models and assessment instruments developed during the first KoKoHs funding phase. Further, they enhance, validate, and test measurement approaches for use in higher education in Germany. Results and findings are transferred at various levels to national and international research, higher education practice, and education policy.

An Overview of Assessment Practices

On a global scale, attempts to specify conceptually academic learning outcomes have been made mostly in OECD countries. Definitions of learning outcomes can be used not only as an orientation for accrediting degree courses and universities; in some countries, they are used also as a basis for the development of assessment methods. The following will provide an overview of various key approaches and projects in this field.

Adapting and Validating the Collegiate Learning Assessment to Measure Generic Academic Skills of Students in Germany: Implications for International Assessment Studies in Higher Education

Starting in 2015, a German research team from the program Modeling and Measuring Competencies in Higher Education (KoKoHs), in collaboration with the US Council for Aid to Education (CAE), adapted and validated the Collegiate Learning Assessment (CLA+) for the German language and cultural context to measure generic higher-order cognitive skills of university students and graduates in Germany. In this chapter, the conceptual and methodological background, the framework of the adaptation and validation study, as well as preliminary results are presented. Finally, findings are discussed critically, and future challenges and perspectives are explored.

Kompetenzen von Studierenden – Ergebnisse des „KoKoHs“-Programms im Kontext der nationalen und internationalen Assessmentpraxis

Ausgehend von aktuellen Entwicklungen im Hochschulbereich und daraus resultierenden Anforderungen wird in diesem Beitrag die nationale und internationale Assessmentpraxis skizziert und vor diesem Hintergrund zentrale Ergebnisse des „KoKoHs“-Programms dargestellt. Abschliesend werden weitere theoretische, methodologische und empirische Herausforderungen sowie Entwicklungsperspektiven fur Hochschulpolitik und -praxis aufgezeigt.

Demands in Higher Education

The field testing of the models and instruments at higher education institutions throughout Germany and the analyses of the data gathered on students’ competencies and their determinants pointed to a number of common challenges that institutions, policy-makers, practitioners, and students face with regard to different study phases of higher education today.

Development Perspectives for Higher Education Research and Practice

In the 24 collaborative projects of the KoKoHs program, comprising more than 70 individual research projects, objective test instruments were developed for assessing the competencies of students and graduates and tested in higher education practice (see table in section 8). The KoKoHs assessments provide empirically substantiated information on students’ competency levels and insight into the effectiveness of teaching-and-learning opportunities in higher education. KoKoHs has offered means to address major challenges of higher education in a more effective manner and, in the long term, contributed to improving structural, organizational, and individual conditions for teaching and learning at higher education institutions.

Competency Orientation in Higher Education

In recent years, higher education, particularly in Germany, has been marked by fundamental structural and programmatic changes after the implementation of the Bologna reform and the orientation towards competencies in the national standards for education, for instance, set forth by the Standing Conference of the Ministers of Education and Cultural Affairs in Germany. The quality of academic education and its social returns have received increasing attention, and endeavors to develop and implement fair assessments of students’ learning outcomes have revealed diverse challenges. In Germany and internationally, it is becoming increasingly important to define and assess in an objective, reliable, and valid way students’ learning outcomes in higher education, for which suitable instruments are urgently needed.

Modeling and Measuring Competencies in Germany

In order to meet the challenges facing higher education policy and practice, and initiate the necessary fundamental assessment-research, the German Federal Ministry of Education and Research (BMBF) established the Modeling and Measuring Competencies in Higher Education (KoKoHs) research program in 2011. The program was part of the BMBF funding priority Research on Science and Higher Education aimed at building a capacity for empirical research in higher education.

Überblick zu nationalen und internationalen Assessments

Zur begrifflichen und konzeptionellen Prazisierung dessen, was unter akademischen Lernergebnissen zu verstehen sei, finden sich – weltweit betrachtet – die meisten Initiativen in OECD-Landern. Solche Definitionen werden nicht nur als Orientierungsrahmen bei Akkreditierungen von Studiengangen und Hochschulen genutzt, sondern bilden in einigen Landern auch eine Basis bei der Entwicklung von Assessmentverfahren. Im vorliegenden Kontext sollen einige zentrale Entwicklungen und Projekte aus diesem Bereich skizziert werden.

Konzeptioneller und methodischer Rahmen zur Analyse des Forschungsstandes

Die vorliegende Forschungssynthese dokumentiert den Stand der nationalen und internationalen Forschung zur Erfassung von akademisch vermittelten Kompetenzen bei Studierenden und Hochschulabsolventen fur die Jahre 2010 bis 2014. Zu seiner Erstellung wurde im Sommer/Herbst 2014 – Winter 2015 eine breit angelegte Dokumentenanalyse durch Literatur- und Datenbankrecherchen (inkl. qualitativen Inhaltsanalysen) durchgefuhrt. Um eine systematische Vorgehensweise zu gewahrleisten, wurde der Recherche eine Auswahl von national und international relevanten hochschul- und bildungsforschungsspezifischen sowie disziplinubergreifenden Literatur- und Projektdatenbanken zugrunde gelegt; auf nationaler Ebene z.B. Fachinformationssystem Bildung (FIS Bildung), die Literaturdatenbank des Deutschen Instituts fur Internationale Padagogische Forschung (DIPF), auf internationaler Ebene z.B. die Education Resources Information Center (ERIC)- Literaturdatenbank des US-amerikanischen Institute of Education Sciences (IES).