Kristina N. Bauer

Self-Assessment of Knowledge: A Cognitive Learning or Affective Measure?

We conducted a meta-analysis to clarify the construct validity of self-assessments of knowledge in education and workplace training. Self-assessment’s strongest correlations were with motivation and satisfaction, two affective evaluation outcomes. The relationship between self-assessment and cognitive learning was moderate. Even under conditions that optimized the self-assessment–cognitive learning relationship (e.g., when learners practiced self-assessing and received feedback on their self-assessments), the relationship was still weaker than the self-assessment–motivation relationship. We also examined how researchers interpreted self-assessed knowledge, and discovered that nearly a third of evaluation studies interpreted self-assessed knowledge data as evidence of cognitive learning. Based on these findings, we offer recommendations for evaluation practice that involve a more limited role for self-assessment. ........................................................................................................................................................................

The effects of technical difficulties on learning and attrition during online training.

Although online instruction has many potential benefits, technical difficulties are one drawback to the increased use of this medium. A repeated measures design was used to examine the effect that technical difficulties have on learning and attrition from voluntary online training. Adult learners (N = 530) were recruited online and volunteered to participate in a 4-hr training program on using computer spreadsheets. Technical difficulties were inserted in some of the training modules in the form of error messages. Using multilevel modeling, the results indicated that the presence of these technical difficulties impaired learning, such that test scores were lower in modules where trainees encountered technical difficulties than in modules where they did not encounter technical difficulties. Furthermore, the effect on learning was greater among trainees who eventually withdrew from the course than among trainees who completed the course. With regards to attrition, pretraining motivation provided a buffer against dropping out, especially when trainees encountered technical difficulties. Learning also predicted attrition from the subsequent module, such that attrition was higher among trainees with low test scores in the previous module. The current study disentangles some of the implications of technical difficulties and suggests that organizations should provide trainees with the technical support required to overcome technical difficulties in training. Furthermore, the findings contribute to our theoretical understanding of the implications of interruptions on performance in online training.

Gamification of task performance with leaderboards

The use of leaderboards is a common approach to the gamification of employee performance, but little is known about the specific mechanisms and mediating processes by which leaderboards actually affect employee behavior. Given the lack of research in this domain, this study proposes goal-setting theory, one of the most well-established motivational theories in psychology, as a framework by which to understand these effects. In this study, a classic brainstorming task is gamified with a leaderboard in order to explore this. Participants were randomly assigned to four classic levels of goal-setting (do-your-best, easy, difficult and impossible goals) plus a leaderboard populated with initials and scores representing identical goal-setting conditions. The presence of a leaderboard was successful in motivating participants to performance levels similar to that of difficult and impossible goal-setting, suggesting participants implicitly set goals at or near the top of the leaderboard without any prompting to do so. Goal commitment, a common individual difference moderator in goal-setting theory, was also assessed and behaved similarly in the presence of the leaderboard as when traditional goals were provided. From these results, we conclude that goal-setting theory is valuable to understand the success of leaderboards, and we recommend further exploration of existing psychological theories, including goal-setting, to better explain the effects of gamification. Goal-setting theory was offered as an explanatory framework for leaderboards.An experiment found addition of a leaderboard on a task increased performance.Leaderboards performed similarly to traditional difficult and impossible goals.Individual goal commitment moderated the success of leaderboards as with goals.Goal-setting and other psychological theories should be explored in gamification.

Psychological Theory and the Gamification of Learning

Research on the gamification of learning currently lacks a sturdy theoretical foundation on which to build new knowledge. In this chapter, we identify and explore several theories from the domain of psychology to provide this foundation. This includes the theory of gamified instructional design, classic conditioning theories of learning, expectancy-based theories, goal-setting theory, and self-determination theory. For each theory (or family of theories), we describe the theory itself, relate it to gamification research, and identify the most promising future research directions given that basis. In exploring these theories, we conclude that gamification is not a “new” instructional technique per se but is instead a new combination and presentation of classic motivational techniques. This combination may provide unique value over other approaches, but this is an unresolved empirical question. We conclude by making specific recommendations for both gamification researchers and practitioners to best advance the study of gamification given this sturdy theoretical basis.

How to Avoid the Dark Side of Gamification: Ten Business Scenarios and Their Unintended Consequences

The problems that may arise from gamification have been largely ignored by researchers and practitioners alike. At the same time, use of gamification in recruitment, onboarding, training, and performance management are on the rise in organizations as businesses turn toward technology to meet their objectives. This chapter investigates drawbacks of using elements of games in each of these applications through a series of scenarios describing different gamified interventions. For each scenario, a discussion follows regarding potential problems with the intervention, how psychological science may explain this, how these errors can be avoided, as well as future directions for gamification research. Employee motivation is noted as a critical concern in gamification, and classic theories of motivation are utilized to help explain why some interventions may fail to motivate desired behavior. For training design, a popular area for gamification, practitioners are urged to consider the intended training outcomes before designing a training program with gaming elements.

Using adaptive difficulty to optimize videogame-based training performance: The moderating role of personality.

While videogames can easily be tailored to provide “adaptive training,” little research exists examining whether this benefit enhances training outcomes. The current study investigated three task difficulty manipulations and the moderating role of personality. Participants engaged in six 7-minute missions in a videogame-based training task. Openness to experience and neuroticism, but not conscientiousness, interacted with task difficulty condition such that trainees higher in these traits performed better over the course of training in the adaptive condition. These results suggest that adaptive training can result in the greatest performance improvement when the trainees personality is suited to the instructional environment.

Work-Life Job Analysis: Applying a Classic Tool to Address a Contemporary Issue

This paper draws from the work-life and human resources research literatures and introduces Work-Life Job Analysis (WLJA) as a tool to leverage work-life balance within organizations. WLJA is based upon the AET (Arbeitwissenschaftliches Erhebungsverfahren zur Tatigkeitsanalyse) a German-developed job analysis procedure used to analyze stress and strain. WLJA is comprised of a questionnaire to analyze the context of the work and a task analysis to assess the work performed in order to identify areas of work-life enrichment and conflict. We provide a theoretical and research basis for the tool, provide step-by-step instructions for application, and illustrate the procedure by analyzing a tenure-track faculty position at a university. As shown by the results of our application, WLJA can be used to enhance a position and to intervene where work-life conflict exists. WLJA is a strategic and scientifically-based tool for work-life intervention that enables psychologist-managers to accommodate specific work-life...