Esther Care

Assessment and Teaching of 21st Century Skills

Rapidand seemingly acceleratingchanges in the economies of developed nations are having a proportional effect on the skill sets required of workers in many new jobs. Work environments are often technology-heavy, while problems are frequently ill-defined and tackled by multidisciplinary teams. This book contains insights based on research conducted as part of a major international project supported by Cisco, Intel and Microsoft. It faces these new working environments head-on, delineating new ways of thinking about 21st-century skills and including operational definitions of those skills. The authors focus too on fresh approaches to educational assessment, and present methodological and technological solutions to the barriers that hinder ICT-based assessments of these skills, whether in large-scale surveys or classrooms. Equally committed to defining its terms and providing practical solutions, and including international perspectives and comparative evaluations of assessment methodology and policy, this volume tackles an issue at the top of most educationalists agendas.

A Framework for Teachable Collaborative Problem Solving Skills

In his book “Cognition in the Wild”, Hutchins (1995) invites his readers to scan their immediate environment for objects that were not produced through collaborative efforts of several people, and remarks that the only object in his personal environment that passed this test was a small pebble on his desk. In fact, it is remarkable how our daily lives are shaped by collaboration. Whether it is in schools, at the workplace, or in our free time, we are constantly embedded in environments that require us to make use of social skills in order to coordinate with other people. Given the pervasiveness of collaboration in everyday life, it is somewhat surprising that the development of social and collaborative skills is largely regarded as something that will occur naturally and does not require any further facilitation. In fact, groups often fail to make use of their potential (Schulz-Hardt, Brodbeck, Group performance and leadership. In: Hewstone M, Stroebe W, Jonas K (eds) Introduction to social psychology: a European perspective, 4th edn, pp 264–289. Blackwell, Oxford, 2008) and people differ in the extent to which they are capable of collaborating efficiently with others. Therefore, there is a growing awareness that collaborative skills require dedicated teaching efforts (Schoenfeld, Looking toward the 21st century: challenges of educational theory and practice. Edu Res 28:4–14, 1999). Collaborative problem solving has been identified as a particularly promising task that draws upon various social and cognitive skills, and that can be analysed in classroom environments where skills are both measurable and teachable.

The Relationship Between Vocational Interests, Self-Efficacy, and Achievement in the Prediction of Educational Pathways:

The influence of vocational interest, self-efficacy beliefs, and academic achievement on choice of educational pathway is described for a cohort of Australian students. Participants were 189 students aged 14—15 years, who were considering either academic or applied learning pathways and subject choices for the final 3 years of secondary school. Using Holland’s interest model within a social cognitive career theory (SCCT) framework, logistic regression analyses indicated that all three constructs were significant predictors of pathway and subject selection and enrolment. The best predictive model for students with strong Realistic interests was an interaction of self-efficacy and interest. For Investigative students, both self-efficacy and achievement were best predictors and for Artistic, Social, and Conventional, achievement was the best predictor of future course enrolment. The results of this research offer partial support for the theory of Lent, Brown, and Hackett in that a variable pattern of associations between vocational interest, self-efficacy, and achievement emerged across the Holland interest themes.

Developmental assessment: lifting literacy through professional learning teams

Outcomes and findings from an evidence‐based approach to targeting primary school students’ developmental reading comprehension levels for effective learning are described. Nineteen schools participated in a literacy assessment project designed to monitor and improve the reading comprehension achievement levels of their students. The project integrated a developmental approach to learning and teaching, information derived from standardised reading comprehension assessments, and professional development for teachers. Reading comprehension achievement across the schools increased at a higher rate than typically expected. Teacher discourse about teaching and learning changed from discrete skill and resource focused, to developmentally focused. The centrality of a professional learning team approach to change for the student, teacher and school is discussed.

Language Ability and Verbal and Nonverbal Executive Functioning in Deaf Students Communicating in Spoken English

The internal use of language during problem solving is considered to play a key role in executive functioning. This role provides a means for self-reflection and self-questioning during the formation of rules and plans and a capacity to control and monitor behavior during problem-solving activity. Given that increasingly sophisticated language is required for effective executive functioning as an individual matures, it is likely that students with poor language abilities will have difficulties performing complex problem-solving tasks. The aim of this study was to investigate the relationship between language ability and verbal and nonverbal executive functioning in a group of deaf students who communicate using spoken English, as measured by their performance on two standardized tests of executive function: the Delis-Kaplan Executive Function System (D-KEFS) 20 Questions Test and the D-KEFS Tower Test. Expressive language ability accounted for more than 40% of variability in performance on the D-KEFS 20 Questions Test. There was no significant relationship between language ability and performance on the D-KEFS Tower Test. There was no relationship between language ability and familiarity with the specific problem-solving strategies of both D-KEFS Tests. Implications of the findings are discussed.

Assessment of Collaborative Problem Solving in Education Environments

ABSTRACT The Assessment and Teaching of 21st Century Skills (ATC21STM) project undertook a research and development plan that included conceptualization of 21st century skills and assessment task development. Conceptualization focused on the definition of 21st century skills. This article outlines the particular case of one of these skills, collaborative problem solving, from its definition and identification of subskills, to development of a method for its assessment. The theories contributing to construct definition are described. These have implications for the particular clusters of subskills that are of interest for educational assessment. An approach to assessment task creation is illustrated through the deconstruction of a well-known reasoning task, and its re-development to sample both the cognitive and social aspects of collaborative problem solving. The assessment tasks are designed to generate formative feedback for teachers in order to identify levels of ability within and between their students and support tailoring of instruction differentially for improvement.

Collaborative Problem Solving Tasks

This chapter outlines two distinct types of collaborative problem solving tasks – content-free and content-dependent – each allowing students to apply different strategies to solve problems collaboratively. Content-free tasks were developed to emphasise the enhancement of inductive and deductive thinking skills. Content-dependent tasks allow students to draw on knowledge gained through traditional learning areas or subjects within the curriculum. The collaborative problem solving framework emphasises communication for the purpose of information gathering, identification of available and required information, identification and analysis of patterns in the data, formulation of contingencies or rules, generalisation of rules, and test hypotheses. Characteristics of tasks which were identified as appropriate for eliciting collaborative problem solving processes are reported and illustrated by exemplar items.

Automatic Coding Procedures for Collaborative Problem Solving

This chapter examines the procedure followed in defi ning a scoring process to enable the reporting of individual student results for teachers to use in the classroom. The procedure begins with the identifi cation of task features that match elements of the skills frameworks, and is followed by the generation of simple rules to collect data points to represent these elements. The data points are extracted from log fi les generated by students engaged in the assessment tasks and consist of the documentation of each event, chat and action from each student. The chapter includes examples of the process for defi ning and generating global and local (task specifi c) indicators, and examples of how the indicators are coded, scored and interpreted. The development of coding and scoring of data generated when students engage in collaborative problem solving tasks is described. The data generated are captured in a process stream data fi le. Patterns of these data are coded as indicators of ele- ments defi ned in the conceptual framework outlined in Hesse et al. ( 2015 ; Chap. 2 ) and the relative complexity of indicators is used in a scoring process. The scored data are then used to calibrate the tasks. The calibrations form the basis of interpre- tation and these are used in forming reports for students and teachers. Figure 6.1 summarises the entire process from task development to the reporting of student ability based on a developmental framework.

The ATC21S Method

The ATC21STM project followed a research and development plan that consisted of five phases: conceptualisation, hypothesis formulation, development, calibration and dissemination. (The acronym ATC21STM has been globally trademarked. For purposes of simplicity the acronym is presented throughout the chapter as ATC21S.) Within the conceptualisation phase, the project focused on the definition of twenty-first century skills. This chapter outlines the selection and conceptualisation of the skills to be assessed. It describes how this led to the development of hypothesised learning progressions which portrayed how the skills might vary across more and less adept individuals. Assessment tasks were commissioned to be developed from a mixture of commercial agencies and universities. The tasks were then subjected to concept checking, cognitive laboratories, pilot studies and calibration trials. The final stage of the process is dissemination, which includes the development of scoring, reporting and teaching in the Assessment and Teaching of 21st Century Skills system.

The Factor Structure of Expressed Preferences for School Subjects.

This study reports the factor structure of preferences for 31 school subjects expressed by 952 females and 862 males enrolled in Year 10 in 30 Australian schools. Within each sex, six principal factors were rotated to an oblique solution. The rotated factor structure of the preferences was interpreted within the framework of Hollands (1973) typology of persons and environments (Realistic, Investigative, Artistic, Social, Enterprising, Conventional — RIASEC). The data were consistent with Hollands model at the level of interpretation in terms of RIASEC themes. The correlations between the factors revealed apparent sex differences in the relationships between interest themes that were not predictable from Hollands hexagonal model of the relationships. These results suggest a srong association between interests in school subjects and occupational interests. They also provide an empirical basis for classifying school subjects in terms of RIASEC interest themes.