James B. MacQueen

A MODIFIED DYNAMIC PROGRAMMING METHOD FOR MARKOVIAN DECISION PROBLEMS

Abstract : At the beginning of each period a system is in a certain state. Depending on the state, choice of an action determines the income for that period and the transition probabilities for moving to the next state. The problem is to choose the action at the beginning of each period which will maximize future total discounted income. A modified dynamic programming method for this problem is described. The method gives improved error control by showing how to compute upper and lower bonds on the optimal return which are, respectively, monotone decreasing and monotone increasing, and converge to the optimal return. The convergence appears to be quite rapid.

Some time-invariant stopping rule problems

Let X 1, X 2, … be an i.i.d. sequence. We consider three stopping rule problems for stopping the sequence of partial sums each of which has a time-invariance for the payoff that allows us to describe the optimal stopping rule in a particularly simple form, depending on one or two parameters. For certain distributions of the Xn , the optimal rules are found explicitly. The three problems are: (1) stopping with payoff equal to the absolute value of the sum with a cost of time , (2) stopping with payoff equal to the maximum of the partial sums with a cost of time , and (3) deciding when to give up trying to attain a goal or set a record . For each of these problems, the corresponding problems repeated in time, where the objective is to maximize the rate of return, can also be solved.

A K-Means Cluster Analysis Computer Program With Cross-Tabulations and Next-Nearest-Neighbor Analysis

A cluster analysis computer program is presented which uses the K-Means algorithm to obtain partitions of multivariate data which have low within-class variance. The program provides a somewhat novel form of a next-nearest neighbor analysis, convenient cross tabulations for variables other than the ones used in the clustering, an efficient form of variable clustering, and approximate randomization tests for a variety of relationships.