Alan Jessop

Minimally biased weight determination in personnel selection

The derivation of weights from preference statements is subject to difficulties, some of which are due to the unreliability of the judgement of the decision maker. To overcome this Jaynes’ principle of maximum entropy has been invoked and may be applied either to weights or to the linear weighted scores of the candidates in a selection problem. When candidates are relatively few the two strategies give different styles of interaction. These are discussed and illustrated by application to a problem of personnel selection.

Entropy in multiattribute problems

Concepts involving entropy have intermittently been used in consideration of multiattribute problems. This paper discusses the main approaches that have been proposed. It is argued that the analogies drawn with information theory and between weights and probabilities are to some degree unsatisfactory and a reconsideration is given. The result is a new suggestion in the spirit of Jaynes’ maximum entropy methodology, but now in two parts: ask those questions that are likely to be most discriminating (minimize entropy) but process the answers obtained conservatively (maximize entropy). In contrast to many previous methods, it is the distribution of the utilities of alternatives rather than of the weights given to attributes that is the focus of attention. Copyright

Blockmodels with maximum concentration

There are many circumstances in which binary relations are defined between pairs of objects: in sociology there are social relations between people; in business there are trading relations between firms; in design there are functional dependencies between components. In all of these the clustering of objects into densely interconnected blocks reveals something of the structure of the system. In this paper a criterion is presented which permits the construction of blocks to be formulated as a quadratic programme. The method is applied to two illustrative cases: the pattern of elective choices by MBA students and the performance assessment of British universities. The method is shown to give results which are readily interpreted and, for the purpose of performance ranking, leads to a more realistic description of achievement.

Sensitivity and robustness in selection problems

When using a simple linear multiattribute utility model for the selection of a preferred alternative the result depends in part on the weighting of the attributes. Given the difficulties which may make weight determination problematic it is natural to seek to justify a choice by showing that it is insensitive to weight imprecision. To do this some view must be taken of what constitutes justification. Three such views are defined and tested on a real data set: choose that alternative which maximises the number of plausible scenarios in which it is best (sensitivity); maximises the difference between best and second best (robustness); minimises the difference between a chosen candidate and the best under different weighting schemes (risk aversion). Application of these different criteria identify a small number of alternatives, one of which is readily chosen. Some quite loose weight restrictions are applied, the clarity of the recommendations indicating that further judgmental input may not be needed.

Using Imprecise Estimates for Weights

In multi-attribute decision problems the decision to differentiate between alternatives will be affected by the precision with which weights are specified. Specifications are imprecise because of the uncertainty characteristic of the judgements on which weights are based. Uncertainties are from two sources, the accuracy with which judgements are articulated and the inconsistency when multiple judgements are made and must be reconciled. These uncertainties are modelled using probabilistic weight estimates integrated by the Dirichlet distribution. This ensures the consistency of the estimates and leads to the calculation of significance of the differences between alternatives. A simple plot of these significant differences helps in the final decision whether this is selection or ranking. The method is used to find weight estimates in the presence of both types of uncertainty acting separately and together.

A measure of competitiveness in leagues: a network approach

Many sports are played competitively in a league format. Final positions are based on the aggregations of the points won at each game. Issues of promotion, relegation and much else will depend on the position in the league. However, the results may also be seen to constitute a network of inter-team relations in which the links represent the degree to which a pair of teams have similar performance. This idea is taken as the basis for the construction of a systemic measure of competitiveness in the league. The basis for the model is the construction of a blockmodel on a network of binary relations. The method is illustrated by application to nine seasons of the English soccer Premier League.

Pattern language: A framework for learning

Pattern languages (PLs) were devised for use in architecture and urban design as ways of providing useful access to design knowledge both for the expert designer and for the lay user. They have subsequently found use in the software community for technical and organisational issues. The paper reports some applications of PLs: (1) as a vehicle for making explicit knowledge that would otherwise remain tacit; (2) as an aid to teaching; (3) as a means of presenting statistical methods to managers.

Exploring structure: A blockmodel approach

The problem of network-based disaggregation as an aid to structuring design problems was first proposed by Christopher Alexander. In this paper a new algorithm is described based on the principle of the blockmodel commonly found in the social sciences. The method is simpler than that of Alexander (and that of Elms). The consequences of this simplicity are explored by comparative evaluation using two problems and all three methods. The inherent complexity of the problem is examined in terms of the small world hypothesis, indicating that relatively little interaction between problem components may greatly increase the complexity of the network and thereby increase the arbitrary basis of the structure expressed.

Prioritisation of an IT budget within a local authority

This paper describes the prioritisation of an IT budget within a department of a local authority. The decision problem is cast as a simple multiattribute evaluation but from two perspectives. First, as an exercise in group decision making. Here the emphasis is on a shared process wherein the object is to obtain consensus. The use of an explicit evaluation framework and the ability to interact with the evaluation data in real time via a simple spreadsheet model were found to improve the decision making. Second, the prioritisation is made analytically. The motivation is to determine the degree to which the rankings are the result of the structural characteristics of the projects themselves rather than of the differences in importance attached to the achievement of the goals represented by the project attributes. Three methods are used: Monte Carlo simulation of ranks, cluster analysis based on attributes and an approach based on entropy maximisation. It is found that in the case studied the structure inherent in the data is high and so the results of the analyses are robust. Finally, a procedure is suggested for the appropriate use of these analyses via a facilitator to aid prioritisation decisions.

Multiple Attribute Probabilistic Assessment of the Performance of Some Airlines

Organisations are increasingly ranked according to their performance. This is usually done by some third party for general consumption. Two considerations then become important: first, that those making the ranking wish to appear to be doing so as objectively as possible; second, that it is desirable that the rationale for the relative performance measures are transparent and easily appreciated by the non-expert consumer.