Cynthia Weston

Analyzing Interview Data: The Development and Evolution of a Coding System

This paper describes the process used by a research team to develop a coding system for analyzing data from interview transcripts and situates the process within approaches to qualitative analysis. Successive versions of the coding scheme illustrate its development over several years; the role of team members and verification in this evolution are discussed. Several lessons emerge from our experience: a) coding is not what happens before analysis, but constitutes an important part of the analysis; b) a team builds codes and coding builds a team through the creation of a shared understanding of the phenomenon; and c) collaborative qualitative research requires a kind of rigor that an independent researcher might not be aware of or need.

Building a metacognitive model of reflection

An increased value is being placed on quality teaching in higher education. An important step in developing approaches to better instruction is understanding how those who are successful go about improving their teaching. Thus, several years ago we undertook a program of research in which the concept of “reflection” provided the frame of reference. We envisaged reflection as a process of formative evaluation, and also saw links between reflection and metacognition. What we have documented and analyzed in detail are the reflective processes of six university professors in their day-to-day planning, instructing and evaluating of learners. The result is a metacognitive model and coding scheme that operationalize the process of reflection. Both provide a language for describing reflection and therefore a way to think about how to improve teaching. In this paper, we describe the research and the model and the contributions they make to our understanding of teacher thinking in higher education.

A model for understanding formative evaluation in instructional design

The model elaborated here provides a common language for analyzing and understanding the literature on formative evaluation by identifying four components: who participates, what roles can be taken, what techniques can be used, and in what situations these can occur. Premised on the design process, intentional decisions about these components must be made after establishing goals of the instruction and considering the constraints. The model was validated by analyzing 11 instructional design texts. The analysis revealed many assumptions embedded in the language that is used to talk about formative evaluation and highlighted what is emphasized and what is not addressed about the process. The model provides a decision-making template for designing an effective formative evaluation.

Designing instruction for the context of online learning

Many individuals are creating instruction that will be used for online learning. This paper reviews four generic categories of instructional principles: Instructional Design (pedagogical issues), Subject Matter (content issues), Language (semantic and syntactic issues), and Presentation (physical issues), that should be used as guidelines when designing instructional materials. Nine current considerations for the context of online learning are drawn from the literature and experience and we show how each relates to these existing guidelines. We conclude the four generic categories of instructional principles apply equally to the design of instruction for the context of online learning and most issues that people talk about when designing online instruction are simply elaborations of these existing principles. A job aid is provided to help designers systematically consider essential principles when designing online instruction.

Reflection on Teaching: Types and Goals of Reflection

We describe an elaboration of an empirical cognitive model of reflection (McAlpine, Weston, Beauchamp, Wiseman, & Beauchamp, 1999). Specifically, we explain 2 fundamental constructs in the model that were previously less well understood: goal and type of reflection.

A reflective learning framework to evaluate CME effects on practice reflection

Introduction: The importance of reflective practice is recognized by the adoption of a reflective learning model in continuing medical education (CME), but little is known about how to evaluate reflective learning in CME. Reflective learning seldom is defined in terms of specific cognitive processes or observable performances. Competency‐based evaluation rarely is used for evaluating CME effects. To bridge this gap, reflective learning was defined operationally in a reflective learning framework (RLF). The operationalization supports observations, documentation, and evaluation of reflective learning performances in CME, and in clinical practice. In this study, the RLF was refined and validated as physician performance was evaluated in a CME e‐learning activity. Methods: Qualitative multiple‐case study wherein 473 practicing family physicians commented on research‐based synopses after reading and rating them as an on‐line CME learning activity. These comments formed 2029 cases from which cognitive tasks were extracted as defined by the RLF with the use of a thematic analysis. Frequencies of cognitive tasks were compared in a cross‐case analysis. Results: Four RLF cognitive processes and 12 tasks were supported. Reflective learning was defined as 4 interrelated cognitive processes: Interpretation, Validation, Generalization, and Change, which were specified by 3 observable cognitive tasks, respectively. These 12 tasks and related characteristics were described in an RLF codebook for future use. Discussion: Reflective learning performances of family physicians were evaluated. The RLF and its codebook can be used for integrating reflective learning into CME curricula and for developing competency‐based assessment. Future research on potential uses of the RLF should involve participation of CME stakeholders.

Zones: Reconceptualizing Teacher Thinking in Relation to Action.

The authors of this article value the framework provided by the conceptions and approaches to teaching literature, yet have come to feel it does not fully represent the complexity of the relation between teacher thinking and action. Drawing on a socio‐cognitive perspective, they studied lecturer interviews collected over a semester to examine how their thinking related to action. They identified four zones of thinking that represent a spectrum of specificity of thinking in relation to action. Each represents conceptually distinct thinking, having a particular characteristic and use, yet their boundaries are fluid. The authors propose that the zones encompass the space from conceptions through to action, which enables the reconceptualization of the relation between thinking and action, thus making a substantial contribution to the present understanding of this relationship.

Concept mapping to support university academics’ analysis of course content

The authors’ goal in working with university academics is to support an intellectual process of close examination of instructional decisions, making explicit the rationale and intentionality underlying those decisions. Subject matter understanding is the primary point of reference in this process. The focus of the research described here is the use of an unstructured form of concept mapping to support academics in the analysis of course content as the first step in a course design process. While some academics with whom the authors have worked have been initially skeptical about concept mapping, the large majority of them, in the end, report that they value the process and what they gained from it. The findings show that the concept mapping process provided an alternate means to rethink course content, one that highlighted relationships among concepts, encouraged a view of the course as an integrated whole, and frequently provided the occasion to make explicit the types of thinking required in the course.

A web-based tutorial to enhance student learning of activity analysis.

Background. This paper describes the impact on learning of a web-based tutorial for the application of activity analysis, with occupational therapy students, at McGill University, Montreal, Quebec. This tutorial offers unique, interactive instructional strategies allowing for self-directed higher cognitive and reflective learning, which has not been possible, to date, in current web-based technology. Scope. Through repeated practice, students collaboratively (in dyads) apply concepts of activity analysis, and receive immediate feedback by comparing their answers to a standard. Methods and Results. Comparisons of performance outcomes on summative exams before and after implementation of the tutorial are made, suggesting a decrease in variance scores indicating fewer students are falling below the class average. These results are attributed to opportunities to practice activity analysis and to receive immediate feedback. Practice Implications. Web-based instruction can impact student learning if the instructional strategies ensure coherence with all other instructional components, match the learning outcomes, facilitate self-directed and collaborative learning, and allow for practice and feedback.

The influence of participants in formative evaluation on the improvement of learning from written instructional materials

Formative evaluation in instructional design is widely advocated as a means to improve instructional materials through tryouts with learners and experts and revision based on this feedback. Research on formative evaluation has sought to prove its effectiveness and to provide guidance as to which methods of collecting feedback and which sources of feedback are most effective in improving learning. It is difficult to determine from the research whether aspects of tryout or of revision are responsible for improved learning. This paper describes a systematic investigation of both tryout and revision to more clearly determine the influence of different formative evaluation participants (experts, learners and revisers) on the improvement of learning from written instructional materials.It was found that revised versions incorporating learner feedback had the most impact on improving learning from the materials. It was also found that revisers have a far more powerful impact on formative evaluation outcomes than was previously supposed, in terms of how they mediate and incorporate the feedback they are given. The results refute contentions that any revision is better than none since versions revised without learner feedback did not improve learning. Implications are discussed.