Kurt Steuck

Knowledge and Performance: Tracking the Development of Expertise:

Performance in complex tasks may not be a monotonic function of experience. In several domains, an inverted U-shaped function has been observed, with increased error rates being associated with intermediate levels of experience. This contradicts the idea of a linear relationship between the development of expertise and performance levels. A simple monotonic increase in similarity to some ideal knowledge representation may not adequately describe changes in knowledge over time. Crosssectional research on people with varying levels of expertise at complex tasks supports the idea of distinct stages in the development of expertise and helps account for an increase in errors as people develop new skills. We conducted a longitudinal study of ninth-grade biology students’ understanding of certain ecology concepts. Students’ mental representations of these concepts were measured three times during a 12-week course. Over the semester, similarity to the knowledge representation of a good student was more predictive of final grades than was similarity to the teachers’ expert representation, as was predicted. According to the non-monotonic theory of knowledge development, expert representations are less powerful predictors of student performance because they do not adequately model student knowledge of the domain. These data support the non-monotonic theory of knowledge development by revealing the superiority of the good student representation over the teacher representation.

Lessons Learned from Large-Scale Implementation of Adaptive Tutoring Systems: The Fundamental Skills Training Project

The panel will present a large-scale ITS implementation project that spanned seven years. The emphasis of the research was on the measurement and evaluation of tutor effectiveness, and lessons learned related to the design of tutoring systems and their integration with 9th grade classroom instruction. Three aspects of evaluation of tutoring systems will be discussed including design issues, implementation issues, and effectiveness issues. An introductory overview and summary of overall lessons learned from research with three tutors will also be presented. The panel will be informal and time will be allocated for comments and questions from the audience in-between and after the presentations. References and short papers on the tutors will be available.