Thomas Loveday

Student evaluation of courses : what predicts satisfaction?

The main goals of course evaluations are to obtain student feedback regarding courses and teaching for improvement purposes and to provide a defined and practical process to ensure that actions are taken to improve courses and teaching. Of the items on course evaluation forms, the one that receives the most attention and consequently the most weight is the question, ‘Overall, I was satisfied with the quality of this course.’ However, no attention has been placed on examining the predictors of students being ‘satisfied with the quality of this course’ overall. This study attempts to address this gap. The findings show that while student characteristics and reasons for enrolling in a course are predictors of overall satisfaction, it is the evaluation questions that predict the majority of the variation in course satisfaction. The findings also reveal that faculty‐selected optional questions are stronger predictors of overall satisfaction than compulsory questions.

The Capability of Static and Dynamic Features to Distinguish Competent From Genuinely Expert Practitioners in Pediatric Diagnosis

Children’s Hospital at Westmead, Sydney, Australia Objective: The authors describe the development of a new, more objective method of distinguishing experienced competent nonexpert from expert practitioners within pediatric intensive care. Background: Expert performance involves the acquisition and use of refined feature-event associations (cues) in the operational environment. Competent nonexperts, although experienced, possess rudimentary cue associations in memory. Thus, they cannot respond as efficiently or as reliably as their expert counterparts, particularly when key diagnostic information is unavail- able, such as that provided by dynamic cues. Method: This study involved the application of four distinct tasks in which the use of relevant cues could be expected to increase both the accuracy and the efficiency of diagnostic performance. These tasks included both static and dynamic stimuli that were varied systematically. A total of 50 experienced pediatric intensive staff took part in the study. Results: The sample clustered into two levels across the tasks: Participants who performed at a consistently high level throughout the four tasks were labeled experts, and participants who performed at a lower level throughout the tasks were labeled competent nonexperts. The groups differed in their responses to the diagnostic scenarios presented in two of the tasks and their ability to maintain performance in the absence of dynamic features. Conclusion: Experienced pediatricians can be decomposed into two groups on the basis of their capacity to acquire and use cues; these groups differ in their diagnostic accuracy and in their ability to maintain performance in the absence of dynamic features. Application: The tasks may be used to identify practitioners who are failing to acquire expertise at a rate consistent with their experience, position, or training. This information may be used to guide targeted training efforts.

An objective approach to identifying diagnostic expertise among power system controllers

Objective: The present study investigated whether performance across a range of cue-based cognitive tasks differentiated the diagnostic performance of power control operators into three distinct groups, characteristic of novice, competence, and expertise. Background: Despite its increasing importance in the contemporary workplace, there is little understanding of the cognitive processes that distinguish novice, competent, and expert performance in the context of remote diagnosis. However, recent evidence suggests that cue acquisition and utilization may represent a mechanism by which the transition from novice to expertise occurs. Method: The study involved the application of four distinct cue-based tasks within the context of power system control. A total of 65 controllers, encompassing a range of industry experience, completed the tasks as part of an in-service training program. Results: Using a cluster analysis, it was possible to extract three distinct groups of operators on the basis of their performance in the cue-based tasks, and these groups corresponded to differences in diagnostic performance. Conclusion: The results indicate assessments of the capacity to extract and utilize cues were able to distinguish expert from competent practitioners in the context of power control. Application: Assessments of the capacity to extract and utilize cues may be used in the future to distinguish expert from nonexpert practitioners, particularly in the context of remote diagnosis.

Cue Utilization and Broad Indicators of Workplace Expertise

Where occupational performance outcomes are difficult to measure, there is a tendency to associate expertise with years of experience and/or previous occupational position. Although useful, these indicators represent composite constructs that embody a number of different variables, only some of which may be strongly associated with the transition to expertise. In identifying an alternative measure of expertise, it is necessary to consider the cognitive processes associated with expert performance, in particular, the role of cue utilization. The present study, conducted in the context of software engineering, was designed to test the relationship between cue utilization and self, peer, and error management indicators of expertise. The results indicated that participants who exhibited relatively higher levels of cue utilization were significantly more likely to self-report engaging in behaviors associated with expert decision making, to be nominated as an expert by their peers, and to demonstrate superior error management when developing solutions to development problems.

Pattern Recognition as an Indicator of Diagnostic Expertise

Expertise is typically associated with high levels of experience in a domain. However, high levels of experience do not necessarily mean that operators are capable of performing at the level of expertise. Based on evidence that pattern-recognition is the foundation of expert diagnostic performance, two studies investigated the utility of distinguishing competent from expert practitioners using measures of the component tasks of pattern-recognition. In two dissimilar domains, performance across the tasks clustered into two levels, reflecting competence and expertise. Performance on the tasks was only weakly correlated with years of experience in the domain. The significance of these results is discussed in relation to assessment and training evaluation.

Trait-based cue Utilization and initial skill acquisition: implications for models of the progression to expertise

The primary aim of this study was to examine the role of cue utilization in the initial acquisition of psycho-motor skills. Two experiments were undertaken, the first of which examined the relationship between cue utilization typologies and levels of accuracy following four simulated, power-off landing trials in a light aircraft simulator. The results indicated that higher levels of cue utilization were associated with a greater level of landing accuracy following training exposure. In the second study, participants’ levels of cue utilization were assessed prior to two 15 min periods during which they practiced take-offs and landings using a simulated unmanned aerial vehicle (UAV). Consistent with Study 1, the outcomes of Study 2 revealed a statistically significant relationship among levels of cue utilization and the number of trials to criterion on the take-off task, and the proportion of successful trials during both take-off and landing. In combination, the results suggest that the capacity for the acquisition and the subsequent utilization of cues is an important predictor of skill acquisition, particularly during the initial stages of the process. The implications for theory and applied practice are discussed.

Expert and competent non-expert visual cues during simulated diagnosis in intensive care.

The aim of this study was to examine the information acquisition strategies of expert and competent non-expert intensive care physicians during two simulated diagnostic scenarios involving respiratory distress in an infant. Specifically, the information acquisition performance of six experts and 12 competent non-experts was examined using an eye-tracker during the initial 90 s of the assessment of the patient. The results indicated that, in comparison to competent non-experts, experts recorded longer mean fixations, irrespective of the scenario. When the dwell times were examined against specific areas of interest, the results revealed that competent non-experts recorded greater overall dwell times on the nurse, where experts recorded relatively greater dwell times on the head and face of the manikin. In the context of the scenarios, experts recorded differential dwell times, spending relatively more time on the head and face during the seizure scenario than during the coughing scenario. The differences evident between experts and competent non-experts were interpreted as evidence of the relative availability of task-specific cues or heuristics in memory that might direct the process of information acquisition amongst expert physicians. The implications are discussed for the training and assessment of diagnostic skills.

Positive Affect Is Associated With Reduced Fixation in a Realistic Medical Simulation

Objective: This study extends previous research by exploring the association between mood states (i.e., positive and negative affect) and fixation in practicing anesthetists using a realistic medical simulation. Background: The impact of practitioner emotional states on fixation is a neglected area of research. Emerging evidence is demonstrating the role of positive affect in facilitating problem solving and innovation, with demonstrated implications for practitioner fixation. Method: Twelve practicing anesthetists (4 females; Mage= 39 years; SD = 6.71) were involved in a medical simulation. Prior to the simulation, practitioners rated the frequency they had experienced various positive and negative emotions in the previous three days. During the simulation, the patient deteriorated rapidly, and anesthetists were observed for their degree of fixation. After the simulation, practitioners indicated the frequency of these same emotions during the simulation. Results: Nonparametric correlations were used to explore the independent relationships between positive and negative affect and the behavioral measures. Only positive affect impacted the likelihood of fixation. Anesthetists who reported more frequent recent positive affect in the three days prior to the simulation and during the simulation tended to be less fixated as judged by independent raters, identified a decline in patient oxygen saturation more quickly, and more rapidly implemented the necessary intervention (surgical cricothyroidotomy). Conclusion: These findings have some real-world implications for positive affect in patient safety. Application: This research has broad implications for professions where fixation may impair practice. This research suggests that professional training should teach practitioners to identify their emotions and understand the role of these emotions in fixation.

The Relationship between pre-flight decision-making and cue utilization

This study examined the relationship between measures of cue utilization and decision selection in the context of pre-flight decision-making. Fifty one licensed pilots participated in an on-line assessment of cue utilization using the program EXPERTise. The results revealed a relationship between cue utilization and decision selection in a pre-flight decision scenario. Overall, the results suggest a role for assessments of cue utilization in the evaluation of pilots’ capacity for effective weather-related decision-making.

Cue Utilization, Perceptions, and Experience in the Interpretation of Weather Radar Returns

This study was designed to examine the role of cue utilization, perceptions, and measures of operational experience in the interpretation of a scenario involving the interpretation of weather radar returns. A total of 47 qualified pilots completed EXPERTise 2.0, an online assessment of cue utilization in the context of weather radar systems. They also completed a scenario involving the interpretation of weather radar returns which required an assessment as to whether they could continue the flight safely in the absence of a change in track or altitude. Consistent with research in other domains, the results revealed a relationship between performance and cue utilization. No relationships were evident on the basis of flight experience nor the inclination to use or trust weather radar systems. The results provide the basis for a tool that might be employed to assess pilots’ cue utilization, thereby enabling more targeted approaches to pilot training and weather radar system design.