Peter Pirolli

The structure of design problem spaces

It is proposed that there are important generalizations about problem solving in design activity that reach across specific disciplines. A framework for the study of design is presented that (a) characterizes design as a radial category and fleshes out the task environment of the prototypical cases; (b) takes the task environment seriously; (c) shows that this task environment occurs in design tasks, but does not occur in every nondesign task; (d) explicates the impact of this task environment on the design problem space; and (e) demonstrates that, given the structure of the information-processing system, the features noted in the problem spaces of design tasks will not all occur in problem spaces where the task environment is vastly different. This analysis leads to the claim that there are a set of invariant features in the problem spaces of design situations that collectively constitute a design problem space. Protocol studies are reported in which the problem spaces of three design tasks in architecture, mechanical engineering, and instructional design are explored and compared with several protocols from nondesign problem-solving tasks.

Motivating the Notion of generic design within information-processing theory: the design problem space

The notion of generic design, although it has been around for 25 years, is not often articulated; such is especially true within Newell and Simons (1972) information-processing theory (IPT) framework. Design is merely lumped in with other forms of problem-solving activity. Intuitively, one feels there should be a level of description of the phenomenon that refines this broad classification by further distinguishing between design and nondesign problem solving. However, IPT does not facilitate such problem classification. This article makes a preliminary attempt to differentiate design problem solving from nondesign problem solving by identifying major invariants in the design problem space.

A Theory of the Measurement of Knowledge Content, Access, and Learning

Abstract : We develop an approach to the measurement of knowledge content, knowledge access and knowledge learning. This approach has two elements: First we describe a theoretical view of cognition, called the Newell-Dennett framework, which we see as being particularly favorable to the development of a measurement approach. Then, we describe a class of measurement models, based on Rasch modeling, which we see as being particularly favorable to the development of cognitive theories. Knowledge content and access are viewed as determining the observable actions selected by an agent in order to achieve desired goals in observable situations. To the degree that models within the theory fit the data at hand, one considers measures of observed behavior to be manifestations of intelligent agents having specific classes of knowledge content and varying degrees of access to that knowledge. Although agents, environment, and knowledge are constitutively defined (in terms of one another), successful application of our theory affords separation of parameters associated with the person from those associated with the environment. We present and discuss two examples of measurement models developed within our approach that address the evolution of cognitive skill, strategy choice and application, and developmental changes in mixtures of strategy use.

A Cognitive Model and Computer Tutor for Programming Recursion

This paper aims at finding the optimal combination of written instruction and on-line practice for learning a new computer application. Experimental subjects learned commands for an electronic spreadsheet by reading brief user-manual descriptions and working training problems on-line. The form of the training problems was varied within subjects in order to control how much independent problem solving subjects engaged in while learning any given command. There were three forms of practice: (1) Pure Guided Practice, in which subjects were told exactly what keystrokes to type to solve the problems; (2) Pure Problem Solving Practice, in which subjects solved problems without guidance; and (3) Mixed Practice, in which the first problem for a command was presented in Guided Practice form and two others in Problem Solving form. The spacing of the training problems was also manipulated; the problems pertaining to a given command were either Massed (i.e., presented consecutively), or Distributed (i.e., separated by other instructional material). After a 2-day delay, subjects solved new problems on the computer without referring to the instructional materials. The results indicate that problem solving was a more difficult form of training than guided practice, but it produced the best performance at test. Distributing the spacing of training problems during training also improved performance at test. The results have clear pragmatic implications for the design of interactive tutorial manuals as well as implications for cognitive models of skill acquisition.

The Instructional Design Environment: technology to support design problem solving

The problems of efficiently producing effective instruction in areas such as industry, military, and vocational education are exacerbated by several complex factors: increasingly rapid change in technology; substantial variation in training needs and target populations within large organizations; inefficient mechanisms for performing formative evaluations; and a lack of means for efficiently incorporating new models of design into practice. In the context of these increasingly difficult problems, we present the Instructional Design Environment (IDE), a hypermedia system for designing and developing instructional material, including texts, interactive video disk, and intelligent tutoring systems. The representation of design analyses and specifications, and the design activities of IDE users are not constrained by any particular model of instructional design, but can be tailored to suit a wide range of such models. Although the system is continually evolving and exists in several forms, (Swartz and Russell, 1989) we discuss how the features of IDE suggest how computer-based design environments may provide ways simplifying the design problems for technical training in rapidly changing areas.

Measuring Learning Strategies and Understanding: A Research Framework

We present a framework for measurement and diagnosis using knowledge-based models. Based on formulations of knowledge-level analysis, symbol-level analysis, constructivist learning, and situated cognition, we describe the possible frames of reference and fundamental measurement approaches that may be adopted in the analysis of cognition and learning. In general, these frames of reference define how individuals, contexts, knowledge, and activity are constitutively defined. We then present several measurement models extending the objective measurement approach of Rasch models to the analysis of learning strategies and knowledge development. The structure and parameter estimates of such models can then be used in the specification of probabilistic belief networks that can perform on-line student modelling.

Towards a Unified Model of Learning to Program

This paper outlines a Soar model of learning to program. The model is intended to address program writing, program comprehension, and the processing of instruction in standard didactic forms as well as more complex interactive forms. The model reveals fine-grained regularities underlying the acquisition and development of programming skill. Recent work on the model suggests that it can be extended to address individual differences in learning strategies and the comprehension of instruction.