Hendrik J. Vos

Applications of Bayesian Decision Theory to Sequential Mastery Testing.

The purpose of this paper is to formulate optimal sequential rules for mastery tests. The framework for the approach is derived from Bayesian sequential decision theory. Both a threshold and linear loss structure are considered. The binomial probability distribution is adopted as the psychometric model involved. Conditions sufficient for sequentially setting optimal cutting scores are presented. Optimal sequential rules will be derived for the case of a subjective beta distribution representing prior true level of functioning. An empirical example of sequential mastery esting for concept-learning in medicine concludes the paper.

A COMPENSATORY APPROACH TO OPTIMAL SELECTION WITH MASTERY SCORES

A Bayesian approach for simultaneous optimization of test-based decisions is presented using the example of a selection decision for a treatment followed by a mastery decision. A distinction is made between weak and strong rules where, as opposed to strong rules, weak rules use prior test scores as collateral data. Conditions for monotonicity of optimal weak and strong rules are presented. It is shown that under mild conditions on the test score distributions and utility functions, weak rules are always compensatory by nature.

A simultaneous approach to optimizing treatment assignments with mastery scores

A model for simultaneous optimization of combinations of test-based decisions in psychology and education is proposed using Bayesian decision theory. The decision problem addressed consists of a combination of a placement and a mastery decision. Weak and strong decision rules are distinguished. As opposed to strong rules, weak rules are allowed to take prior test scores in the series of decisions into account. The introduction of weak rules makes the placement-mastery problem a multivariate decision problem. Conditions for optimal rules to take monotone forms are derived. Results from an empirical example of instructional decision making are presented to illustrate the differences between a simultaneous and a separate approach.

Applications of Bayesian decision theory to intelligent tutoring systems

The purpose of this paper is to consider some applications of Bayesian decision theory to intelligent tutoring systems. In particular, it will be indicated how the problem of adapting the appropriate amount of instruction to the changing nature of students capabilities during the learning process can be situated within the general framework of Bayesian decision theory. Two basic elements of this approach will be used to improve instructional decision making in intelligent tutoring systems. First, it is argued that in many decision-making situations the linear loss model is a realistic representation of the losses actually incurred. Second, it is shown that the psychometric model relating observed test scores to the true level of functioning can be represented by Kelleys regression line from classical test theory. Optimal decision rules will be derived using these two features.

A Bayesian sequential procedure for determining the optimal number of interrogatory examples for concept learning

The purpose of this paper is to derive optimal rules for sequential decision-making in intelligent tutoring systems. In a sequential mastery test, the decision is to classify a student as a master, a nonmaster, or to continue testing and administering another item. The framework of Bayesian sequential decision theory is used; that is, optimal rules are obtained by minimizing the posterior expected losses associated with all possible decision rules at each stage of testing and using techniques of backward induction. The main advantage of this approach is that costs of testing can be taken explicitly into account. The sequential testing procedure is demonstrated for determining the optimal number of interrogatory examples for concept-learning in the Minnesota adaptive instructional system. The paper concludes with an empirical example in which, for given maximum number of interrogatory examples for concept-learning in medicine, the appropriate action is indicated at each stage of testing for different number-correct score.

Simultaneous optimization of decisions using a linear utility function

The purpose of this article is to simultaneously optimize decision rules for combinations of elementary decisions. With this approach, rules are found that make more efficient use of the data than could be achieved by optimizing these decisions separately. The framework for the approach is derived from Bayesian decision theory. To illustrate the approach, two elementary decisions (selection and mastery decisions) are combined into a simple decision network. A linear utility structure is assumed. Decision rules are derived both for quota-free and quota-restricted selection–mastery decisions in case of several subpopulations. An empirical example of instructional decision making in an individual study system concludes the article.

Contributions of minimax theory to instructional decision making in intelligent tutoring systems

The purpose of this paper is to formulate decision rules for adapting the appropriate amount of instruction to learning needs in intelligent tutoring systems. The framework for the approach is derived from minimax decision theory (minimum information approach), i.e. optimal rules are obtained by minimizing the maximum expected loss associated with each possible decision rule. The binomial model was assumed for the conditional probability of a correct response given the true level of functioning, whereas threshold loss was adopted for the loss function involved. A simple decision rule is given for which only the minimum true level of functioning required for being a ‘true master’ and the value of the loss ratio have to be specified in advance by the decision-maker. The procedures are demonstrated for the problem of determining the optimal number of interrogatory examples for concept-learning in the Minnesota Adaptive Instructional System (MAIS). The Bayesian decision component assumed in the MAIS and the minimax strategy are compared with each other in terms of their weak and strong points. An empirical example of determining the optimal number of interrogatory examples for concept-learning in medicine concludes the paper.

Adapting the Amount of Instruction to Individual Student Needs.

ABSTRACT The purpose of this paper is to formulate optimal sequential decision rules for adapting the appropriate amount of instruction to learning needs. The framework for the approach is derived from Bayesian decision theory. It is assumed that three actions (namely master, partial master, and nonmaster) are open to the decision‐maker. The procedures are demonstrated for the problem of determining the optimal number of interrogatory examples for concept learning. It is shown that the optimal sequential decision rules take into account improvements in learning. An empirical example of computer‐based instructional decision making for concept learning in medicine concludes the paper. ∗The author is indebted to Wim J. van der Linden for his valuable comments and to Jan Gulmans for providing the data for the empirical example.

A lattice representational definition of a hierarchy of instructional processors usable in educational courseware

The basic “recognize-act-recognize-end” cycle can be recognized in elementary as well as in more advanced forms of CAI. This article attempts to offer a unifying formal framework in which different elaborations of this cycle (embodied in a “processor”) can be placed. Three different levels of elaboration are distinguished which can be considered to be situated into the nodes of a lattice of models of the instructional process. A formal definition of such a framework can serve at least two functions. In the first place a uniform and precise definition of various elaborations can be given and new elaborations can be created in a logically funded way. Secondly, such a framework can support the modelling of instructional processes and the stimulation of student behavior. Thus, pre-testing of courseware could become feasible. Aspects of the framework have been used to implement two prototypes of support systems for the development of CAI courseware.

A minimax procedure in the context of sequential testing problems in psychodiagnostics

The purpose of this paper is to derive optimal rules for sequential testing problems in psychodiagnostics. In sequential psychodiagnostic testing, each time a patient is exposed to a new treatment, the decision then is to declare this new treatment effective, ineffective, or to continue testing and exposing the new treatment to another random patient suffering from the same mental health problem. The framework of minimax sequential decision theory is proposed for solving such testing problems; that is, optimal rules are obtained by minimizing the maximum expected losses associated with all possible decision rules at each stage of testing. The main advantage of this approach is that costs of testing can be explicitly taken into account. The sequential testing procedure is applied to an empirical example for determining the effectiveness of a cognitive-analytic therapy for patients suffering from anorexia nervosa. For a given maximum number of patients to be tested, the appropriate action is indicated at each stage of testing for different numbers of positive reactions to the cognitive-analytic therapy. The paper concludes with a simulation study, in which the minimax sequential strategy is compared for the anorexia nervosa example with other procedures that exist for similar classification decision problems in the literature in terms of average number of patients to be tested, classification accuracy and average loss.