Alenoush Saroyan

Evaluating University Teaching: Time to take stock

In the chapter that Dunkin and Barnes wrote for the Handbook of Research on Teaching (1986), they made the observation that evaluation and the improvement of teaching have always been linked concepts in higher education. This assertion is questioned and it is argued that since student course ratings are the predominant way of evaluating teaching in most universities, the link between evaluation and improvement is at best, tentative. It is argued that in order for the evaluation to lead to improvement, there is a need to have a measure of all the elements that together make teaching what it is. This paper attempts to extend the existing conceptualisation of teaching and proposes a model that shows the underlying processes that influence action. The model highlights the significance of conceptions and belief structures, knowledge and the context within which the instructional event takes place. The paper concludes by suggesting activities and instruments which may have greater potential in providing information about the quality of teaching and are more likely to lead to change in thinking about teaching and subsequently practice.

Variations in lecturing styles

Many factors contribute to the way a lecture is conceptualized and delivered. These include ones philosophy and beliefs about teaching, knowledge of pedagogical principles, availability of resources, and the realities surrounding the instructional situation. In this paper three types of lecturing styles are described and their differences are highlighted in the context of current conceptions of teaching and pedagogical principles. The three lectures are subsequently characterized as content-driven, context-driven, and pedagogy-driven. Evaluation data suggest that the more pedagogically oriented the lecture, the higher it is rated by students.

Evidence-based practice: a review of theoretical assumptions and effectiveness of teaching and assessment interventions in health professions

Health care professionals are expected to use a systematic approach based on evidence, professional reasoning and client preferences in order to improve client outcomes. In other words, they are expected to work within an evidence-based practice (EBP) context. This expectation has had an impact on occupational therapy academic programs’ mandates to prepare entry-level clinicians who demonstrate competence in the knowledge, skills and behaviors for the practice of evidence-based occupational therapy. If the EBP approach is to be entrenched in the day to day practice of future clinicians, a pedagogically sound approach would be to incorporate EBP in every aspect of the curriculum. This, however, would require a comprehensive understanding of EBP: its basis, the principles that underpin it and its effectiveness in promoting core professional competencies. The existing literature does not elucidate these details nor does it shed light on how requisite competencies for EBP are acquired in professional education in general and in occupational therapy education in particular. Drawing from educational psychology and EBP in the health professions, this paper provides a critical review of the evidence that supports EBP and the effectiveness of EBP teaching and assessment interventions in professional heath sciences programs and offers suggestions for the design of EBP instruction, grounding recommendations in educational theory for the health professions.

An analysis of guidelines for expert reviewers

Forty eight sets of recommendations identified by their authors as practical heuristics for the evaluation and revision of instructional materials were reviewed and consolidated. These guidelines were extracted from professional journals, book chapters and independent publications. Initially, all items were compiled, and then sorted into three categories according to their specific focus: content (subject matter), design, and presentation. In a second sort, identical items were eliminated and semantically equivalent items were grouped together. Three independent judges performed the same tasks for reliability. The outcome is a comprehensive list of 67 items, representing all of the reviewed guidelines. This instrument could be a successful aid in identifying deficiencies of instructional materials in the areas of content, design and presentation.

Differences in expert practice: a case from formative evaluation

A clear case has been made in the literature regarding the role of the learner in formative evaluation of instructional materials. In contrast, research on the role of the expert has been limited. This paper describes formative evaluation by instructional designers and subject matter experts, highlighting their task interpretation, their focus on text features, and their strategies. Content analysis of protocols based on think-aloud data suggest that instructional designers take on the role of a generalist, use a comparative method of review, and appear to be directed by the heuristics of the Instructional Systems Design Model. In contrast, subject matter experts approach the task as a specialist, use a sequential method of review, and seem to be directed by the domain knowledge. These differences lead to the identification of different problems and the generation of different revision recommendations. The implications of findings are discussed.

Assessing the impact of a faculty development workshop: A methodological study

Background: Faculty development programs have been used widely in the health professions. Although many activities have been described in the literature, few studies exist that have measured the impact of such programs. Purpose: This pilot study was designed to test a methodology to measure change in attitudes, cognitive learning, and teaching behaviors following a workshop on small group teaching. Methods: Instruments were designed to measure attitudes, cognitive learning, and teaching behaviors. Study participants included 10 faculty members who had attended the workshop between 1988 and 1993 and 10 control participants. Over a 3‐month period, the three instruments were employed to collect data. Results: The instruments developed for this study detected differences between the two groups. The experimental group exhibited more small‐group teaching skills and greater knowledge about small‐group teaching than did the control group. Differences existed in the use of and attitudes toward this teaching method...

Agency matters: academic developers’ quests and achievements

The paper provides an analysis of the concepts of agency, compliance, and resistance, drawing on evidence provided in the Belgian, Norwegian, and Polish cases in this Special Issue, as well as the Canadian context. Using the Cultural Historical Activity Theory framework as a lens, it highlights conditions that foster agency, those that may invoke compliance or resistance, and those that lead to fruitful collaboration.

Creation of an evidence-based practice reference model in falls prevention: findings from occupational therapy

Purpose: This study attempted to capture the evidence-based practice (EBP) behaviours of expert occupational therapy (OT) clinicians in order to develop a reference model of EBP in falls prevention. Methods: Expert clinicians participated in the creation of a clinical vignette through focus group discussions. Using the vignette as the stimulus case, the same clinicians answered questions that reflected the EBP process. Validation of original responses and data synthesis occurred through a second focus group. This validation process resulted in the elaboration of a tree structure EBP decision model. Results: Findings show that clinicians are not expert evidence-based practitioners. Although some of the experts’ clinical decisions were based on a combination of professional experience and research evidence, clinicians relied primarily on clinical experience for more complex aspects of decision-making. When explicitly instructed to answer questions corresponding to the five EBP steps, experts were compelled to think about the use of evidence and could proceed through the EBP process. Conclusions: The model represents the expert clinical decisions in each of the EBP steps and illustrates what aspects of the decision-making process are in line with EBP versus aspects that are driven primarily by experience. This research has the potential to assist clinicians working in prevention of falls in geriatric rehabilitation who can use the model as a practice framework to guide them through the EBP process. Implications for Rehabilitation Experienced clinicians can proceed through the steps of the evidence-based practice process with guidance and scaffolding. Clinical experience is a major factor in decision-making in falls prevention. Models of expert performance provide useful insights into decision-making outcomes and evidence-based practise behaviours of experienced clinicians in falls prevention.

Describing medical student curiosity across a four year curriculum: An exploratory study

Abstract Background: Intellectual curiosity can be defined as a desire for knowledge that leads to exploratory behavior and consists of an inherent and stable trait (i.e. trait curiosity) and a variable context-dependent state (i.e. state curiosity). Although intellectual curiosity has been considered an important aspect of medical education and practice, its relationship to medical education has not been empirically investigated. The purpose of this exploratory study was to describe medical students’ intellectual curiosity across a four-year undergraduate program. Methods: We employed a cross-sectional design in which medical students, across a four-year undergraduate program at McGill University, completed the Melbourne Curiosity Inventory as a measure of their state and trait intellectual curiosity. A Mixed Models ANOVA was used to compare students across year of training. Results: Four hundred and two out of 751 students completed the inventory (53.5%). Trait curiosity was significantly higher than state curiosity (M = 64.5, SD = 8.5 versus M = 58.5, SD = 11.6) overall, and within each year of training. Conclusions: This study is the first to describe state and trait intellectual curiosity in undergraduate medical education. Findings suggest that medical students’ state curiosity may not be optimally supported and highlight avenues for further research.

The InforMed Professor: Clinical Instruction of Breast Disease Diagnosis and Management

This paper describes a learning environment in the domain of breast disease and discusses some of the issues in designing a system cooperatively with content, pedagogical, and computer programming experts. The environment uses curricular content, presented in a non-linear fashion, from third and fourth year medical school; all aspects of the curriculum are available simultaneously to enhance the integration of declarative and procedural knowledge necessary in skilled clinical performance. The system represents an interactive, authentic environment where learners construct knowledge, develop higher order cognitive skills such as diagnosis, reasoning, and decision making, apply knowledge and skills, interact with patients, and receive appropriate feedback. The design of the learning environment is based on the theory of cognitive apprenticeship, thus the type and level of information and feedback provided by the system is dependent up on the learners input. The system also allows learners to evaluate themselves by comparing their actions and decisions to those of experts in the field. The collaboration of experts from different disciplines in designing this environment has resulted in a greater degree of sophistication in ways in which the project has been conceptualized and reified. By focusing all experts on the learning goals of the environment and by cooperatively assisting each type of expert to extend their knowledge of the other areas, we have been able to create a system that incorporates the key theoretical principles from each area. The result is a learning environment that emphasizes meaningful learning of knowledge and skills that learners can apply when they are confronted with similar situations in the real world.