Andras Pete

Optimal Team and Individual Decision Rules in Uncertain Dichotomous Situations

In this paper, we consider the problem of determining the optimalteam decision rules in uncertain, binary (dichotomous) choice situations. We show that the Relative (Receiver) Operating Characteristic (ROC) curve plays a pivotal role in characterizing these rules. Specifically, the problem of finding the optimal aggregation rule involves finding a set ofcoupled operating points on the individual ROCs. Introducing the concept of a “team ROC curve”, we extend the method of characterizing decision capabilities of an individual decisionmaker (DM) to a team of DMs. Given the operating points of the individual DMs on their ROC curves, we show that the best aggregation rule is a likelihood ratio test. When the individual opinions are conditionally independent, the aggregation rule is a weighted majority rule, but with different asymmetric weights for the ‘yes’ and ‘no’ decisions. We show that the widely studied weighted majority rule with symmetric weights is a special case of the asymmetric weighted majority rule, wherein the competence level of each DM corresponds to the intersection of the main diagonal and the DMs ROC curve. Finally, we demonstrate that the performance of the team can be improved by jointly optimizing the aggregation rule and the individual decision rules, the latter possibly requiring a shift from the isolated (non-team) optimal operating point of each DM.

Optimization of detection networks with multiple event structures

Considers a multilevel hierarchical decision network faced with a distributed binary detection problem with partial information at the individual decisionmaker (DM). The partial information is modeled by different local events at the DMs, and these local events are probabilistically related to one another. Solution to this generalized hypothesis testing problem is obtained using the optimal control approach, where the optimization criterion is the expected decision cost of the network. The impacts of variations in the correlation of events at two communicating nodes on the aggregated expertise of the network and on the overall decision cost are illustrated via a numerical example. >

Distributed detection in teams with partial information: a normative-descriptive model

A hierarchical team faced with a binary detection problem, wherein decision makers (DMs) have access to different subsets of noise-corrupted information about the true state of the environment, is considered. A normative model is developed that aggregates the individual expertise of DMs at different levels of the hierarchy. The resulting team expertise is characterized in the form of a team receiver operating characteristic (ROC) curve, thereby replacing the team by an equivalent single decision-making node. The normative model is tested against human teams in a laboratory experiment. The team objective is to minimize the cost of errors in the final decision at the primary DM, where the cost structure and the information structure are treated as independent variables. Discrepancies between normative predictions and experimental results are attributed to inherent limitations and cognitive biases of humans. >

An overview of decision networks and organizations

The paper summarizes recent results on both binary and M-ary distributed hypothesis testing problems with decision makers (DMs) organized in structured decision networks. The general problem of finding an optimal organizational structure and decision strategy for such networks is formulated as a functional optimization problem. A normative model to study the effect of interactions between task structure and organizational design on the performance of hierarchical organizations is presented. A binary signal detection model is considered to illustrate the joint impact of organizational design and of task environment on the organizational decision performance. The concept of a congruent organizational structure (i.e., a structure that achieves centralized performance with minimal communication) is introduced, and a graph decomposition algorithm to synthesize congruent structures is discussed.

Optimization of decision networks in structured task environments

This paper considers the problem of determining the optimal distributed decision strategy for a team of decision-makers (DMs) arranged in an arbitrary acyclic organizational structure and controlling a complex structured process. Each DM has access to uncertain and partial information about the task environment and can control only a portion of it. We present an influence diagram model of the joint task-organization system and formulate the optimal team strategy in terms of a set of coupled hypothesis testing tasks at DMs. Specifically, we show that the scope of a local decision task is determined by the interaction of the task structure (what can be measured) and of the information access structure of the organization (who can measure what); while the control structure (who can influence what event) has an impact on the locally perceived costs associated with the decision options available. Theoretical results are illustrated via a numerical example, and connections to existing decision models are discussed.

Structural reconfiguration and informal coordination in administrative organizations

This paper considers the problem of determining the optimal design of public organizations in terms of maximizing their reliability against institutional failures. To capture both the individual and the system-level aspects of organizational decisionmaking, first we present an analytical model that characterizes the optimal decision behavior of a single decision maker (unit, agent, in general: DM) in the context of a binary decision task. In this sense, reliability of a DM against the two possible error types: implementation of the wrong policy (error of comission, Type I error) and failure to act when it is necessary (error of omission, Type II error) are interpreted as the result of a particular decision strategy. Individual expertise is represented in the form of a Relative Operating Characteristic (ROC) curve that, in turn, depicts the necessary trade-off between the two errors when selecting an appropriate decision strategy. Component decisions are then combined along the lines of organizational structure which is described using a graph formalism. We show that the task of finding the best organizational design involves a joint optimization over structure and strategy, and implement the normative model in the context of a detailed example. Our numerical results suggest that when DMs coordinate their decision rules, there is little difference in the performance of various organizational structures.

Distributed binary detection with different local hypotheses

The authors consider a generalized distributed binary hypothesis-testing problem with a hierarchical team. In this problem, the subordinate decision-makers (DMs) transmit their opinions on their own local hypotheses, which are only probabilistically related to the global hypotheses at the primary DM. It is shown that the normative decision strategies of all DMs are coupled likelihood-ratio tests, but the decision thresholds are also a function of the joint probability distribution of hypotheses at all DMs. To assess the discrimination capabilities of a team, the authors introduce the concept of a team relative (receiver) operating characteristic curve. The concept was tested on teams of humans using a hypothetical medical diagnosis task. Potential human biases leading to discrepancies between the normative predictions and experimental results were identified. These form the basis for a normative-descriptive model currently under development.<<ETX>>

Team Relative Operating Characteristic: A Measure of Team Expertise in Distributed Detection Tasks 1

Abstract We consider a generalized distributed binary hypothesis-testing problem within a hierarchical team. In this problem, the subordinate decisionmakers (DMs) transmit their opinions on their own local hypotheses, which are only probabilistically related to the global hypotheses at the primary DM. In a recent paper, we have shown that the normative decision strategies of all DMs are coupled likelihood-ratio tests, but the decision thresholds are also a function of the joint probability distribution of hypotheses at all DMs. In order to assess the discrimination capabilities of a team, we have introduced the concept of a “Team Relative (Receiver) Operating Characteristic (ROC) curve”. In this paper, we extend earlier results in the literature, and show that the area under the team ROC curve can be used as a measure of expertise of teams. To predict the expertise of actual teams, the normative model is tested on teams of humans using a hypothetical medical diagnosis task. Potential human biases leading to discrepancies between the normative predictions and experimental results are identified. A normative-descriptive model is developed to capture the cognitive biases of human DMs. The model provides excellent predictions with respect to the individual and team ROC operating points and the confidence estimates.

Structural adaptation versus strategy coordination in decisionmaking organizations

This paper presents an analytical model to study the joint impact of organizational structure and message format on the performance of decision-making systems. Unlike the traditional formulation of M-ary hypothesis testing with dependent measurements, we explicitly consider the structure of a task environment, and determine the optimal subtask of each member in the organization. It is shown that the decision accuracy improves if the organization adapts its structure to that of the task; however, the efficiency of such reconfiguration depends on the communication constraints.<<ETX>>

TEAM RELATIVE OPERATING CHARACTERISTIC: A MEASURE OF TEAM EXPERTISE IN DISTRIBUTED DETECTION TASKS1

We consider a generalized distributed binary hypothesis-testing problem within a hierarchical team. In this problem, the subordinate decisionmakers (DMs) transmit their opinions on their own local hypotheses, which are only probabilistically related to the global hypotheses at the primary DM. In a recent paper, we have shown that the normative decision strategies of all DMs are coupled likelihood-ratio tests, but the decision thresholds are also a function of the joint probability distribution of hypotheses at all DMs. In order to assess the discrimination capabilities of a team, we have introduced the concept of a “Team Relative (Receiver) Operating Characteristic (ROC) curve”. In this paper, we extend earlier results in the literature, and show that the area under the team ROC curve can be used as a measure of expertise of teams. To predict the expertise of actual teams, the normative model is tested on teams of humans using a hypothetical medical diagnosis task. Potential human biases leading to discrepancies between the normative predictions and experimental results are identified. A normative-descriptive model is developed to capture the cognitive biases of human DMs. The model provides excellent predictions with respect to the individual and team ROC operating points and the confidence estimates.