Natalie Simper

Assessing teamwork in undergraduate education: a measurement tool to evaluate individual teamwork skills

Effective teamwork skills are essential for success in an increasingly team-based workplace. However, research suggests that there is often confusion concerning how teamwork is measured and assessed, making it difficult to develop these skills in undergraduate curricula. The goal of the present study was to develop a sustainable tool for assessing individual teamwork skills, with the intention of refining and measuring these skills over time. The TeamUp rubric was selected as the preliminary standardised measure of teamwork and tested in a second year undergraduate course (Phase One). Although the tool displayed acceptable psychometric properties, there was concern that it was too lengthy, compromising student completion. This prompted refinement and modification leading to the development of the Team-Q, which was again tested in the same undergraduate course (Phase Two). The new tool had high internal consistency, as well as conceptual similarity to other measures of teamwork. Estimates of inter-rater reliability were within a satisfactory range, although it was determined that logistical issues limited the feasibility of external evaluations. Preliminary evidence suggests that teamwork skills improve over time when taught and assessed, providing support for the continued application of the Team-Q as a tool for developing teamwork skills in undergraduate education.

Effects of Concept Mapping on Creativity in Photo Stories

This research tested the use of concept map planning to support the development of creativity in photo stories, hypothesizing that skills taught to support organization would improve creativity. Concept maps are a type of graphic organizer, used to represent an ordering of ideas with nodes and linking words that form propositional statements. They were applied to story development under the assumption that the creative development of narrative elements requires organization. Undergraduate education students were randomly assigned to either concept map or text-based planning groups for the production of digital photo stories. Creativity was operationalized using dimensions of novelty, interest, clarity, and ability to be understood. A multivariate effect of planning method was found for these dimensions favoring the concept map group, due largely to greater clarity in the photo stories. Clarity in photo stories adds to their utility as creative products. This study represents first steps toward empirically assessing concept maps for a creative purpose, and further investigation, with a larger sample in an authentic context, over a longer period of planning time is recommended.

Development of the Transferable Learning Orientations Tool: Providing Metacognitive Opportunities and Meaningful Feedback for Students and Instructors.

This study encapsulates the development and testing of the Transferable Learning Orientations (TLO) tool. It is a triangulated measure built on select scales from the Motivated Strategies for Learning Questionnaire (MSLQ), together with multiple-choice items adapted from the lifelong learning VALUE rubric, and an open-ended response for each dimension. Select scales from the MSLQ were tested in a range of undergraduate courses, and the TLO (version one) was developed and piloted in a first-year engineering course. Minor refinements were made, and the TLO (version two) was retested with second-year undergraduates. The TLO is designed to engage students in meta-cognitive processes and provide meaningful feedback to students. The dimensions are outcome motivation, learning belief, self-efficacy, transfer and organisation. Results from the second-year group were more consistent and reliable than the first-year group, suggesting that context is an important factor. The scales demonstrate acceptable reliability, and the moderate correlations between scale scores and rubric ratings provide support for concurrent validity. We recommend the TLO be tested with broader populations to confirm psychometric properties and that it be implemented longitudinally to investigate the development of learning skills and changes in orientations over time.

Formative feedback and scaffolding for developing complex problem solving and modelling outcomes

ABSTRACT This paper discusses the use and impact of formative feedback and scaffolding to develop outcomes for complex problem solving in a required first-year course in engineering design and practice at a medium-sized research-intensive Canadian university. In 2010, the course began to use team-based, complex, open-ended contextualised problems to develop problem solving, communications, teamwork, modelling, and professional skills. Since then, formative feedback has been incorporated into: task and process-level feedback on scaffolded tasks in-class, formative assignments, and post-assignment review. Development in complex problem solving and modelling has been assessed through analysis of responses from student surveys, direct criterion-referenced assessment of course outcomes from 2013 to 2015, and an external longitudinal study. The findings suggest that students are improving in outcomes related to complex problem solving over the duration of the course. Most notably, the addition of new feedback and scaffolding coincided with improved student performance.