Daniel Serfaty

Resource allocation and performance evaluation in large human-machine organizations

A methodology is presented for mapping the processes comprising a mission onto the capable resources within the organization such that the completion time of the terminal process in the mission is minimized. The authors then build a Petri net simulation directly from the output of the mapping algorithm to perform sensitivity analyses on the solution. This capability enables the analyst to study in an interactive way variations in the performance of the organization as a function of its workload capacity, the expertise distribution of its members, the task requirements, and the communication network linking the different resources in the organization. >

Sequential revision of belief: an application to complex decision making situations

This research addresses the process of sequential revision of beliefs or judgments in complex situations. The task domain, military command and control, provides decision makers with opportunities to revise their tactical judgments as streams of information flow in for their consideration. A contrast-inertia model is proposed that describes subjects sequential revision of beliefs exhibited by subjects and a resulting order-effect that is observed when subjects attempt to integrate pieces of confirming and disconfirming evidence. Two experiments were conducted to test the predictions of the contrast-inertia model and to investigate various aspects of the order effect. The experiments manipulated the initial starting position or anchor against which subjects contrast new evidence to revise their beliefs. Results form both experiments showed strong recency and order effects when subjects integrated inconsistent pieces of evidence sequentially, regardless of the initial anchor. Moreover, the contrast-inertia model fit the experimental data very well and confirmed the basic assumptions predicting an order effect.

Evaluating Expertise in a Complex Domain–Measures Based on Theory

Human factors practitioners are often concerned with defining and evaluating expertise in complex domains where there may be no agreed-upon expertise levels, no single right answers to problems, and where the observation and measurement of real-world expert performance is difficult. This paper reports the results of an experiment in which expertise was assessed in an extremely complex and demanding domain–military command decision making in tactical warfare. The hypotheses of the experiment were: 1) command decisionmaking expertise can be recognized in practice by domain experts; 2) differences in the command decisionmaking expertise of individuals can be identified even under conditions that do not fully replicate the real world; and 3) observers who are not domain experts can recognize the expert behaviors predicted by a mental-model theory about the nature of expertise. In the experiment, the expertise of military officers in developing tactical plans was assessed independently by three “super-expert” judges, and these expertise-level ratings were correlated with independent theory-based measures used by observers who were not domain experts. The results suggest that experts in a domain have a shared underlying concept of expertise in that domain even if they cannot articulate that concept, that this expertise can be elicited and measured in situations that do not completely mimic the real world, and that expertise measures based on a mental-model theory can be used effectively by observers who are not experts in the domain.

From Team Structure to Team Performance: A Framework

This paper presents a theoretical framework that links the organizational structure of a team to the teams performance through intervening factors such as the need for coordination, the need for communication, the extent to which the teams mission can be pre-planned, and the teams mutual awareness of each others tasks. We suggest that the organizational structure of a team interacts with the nature of the teams mission (in particular, the interdependence among the tasks to be performed) to generate the need for coordination in order to successfully accomplish the mission. The need for coordination and the extent to which the mission can be pre-planned drive the need for communication during the mission. The efficiency of that communication is, in turn, affected by factors such as the teams level of mutual awareness. The paper presents several innovative measures for components of the suggested framework, and summarizes empirical evidence for the framework.

Analysis of organizational processes in adaptive command and control architectures

The relationship between overall team workload, the accuracy with which team members could estimate each others workload, and the teams performance was examined under three different organizational structures for a joint task force command and control team. The study used a new workload questionnaire that asked team members to rate not only their own workload but also the workload of the team as a whole and the workload of each of the other individuals on the team. It was hypothesized that ability to accurately estimate the workload of others in the team might be associated with higher team performance. Results show that performance was highest under the organization where overall team workload was lowest. Surprisingly, however, the variability in individual workload was highest in this organization, and the accuracy of mutual workload estimation was lowest. We conclude that more complex hypotheses are needed about the relationship between the accuracy of mutual workload perceptions in a team, team processes, and team performance.

The effects of team size on team coordination

The authors investigate the evolution of team coordination as teams expand in a dynamic, distributed, decision-making environment. The foundation of the Team Size Experiment (TEASE) is a distributed computer testbed where subjects performed simulations first as individuals, and later as part of a dyad and yet again as a triad. Time pressure was manipulated as an independent variable on all teams, and as a second variable the content of the decision makers information display was varied for dyads and triads only. Preliminary experimental results focus on the adaptation of different size teams to time stress. The average performance over all experiment conditions of triads was superior to the performance of individuals who did slightly better than dyads. The triad success is attributed in part to a clear division of responsibility, while the dyad had greater coordination uncertainty due to a shared resource responsibility. The performance improvement of triads over dyads suggests that duties performed on cooperative tasks must be clearly defined among team members to assure successful team coordination.<<ETX>>

Collaborative Tools for Counter-Terrorism Analysis

One of the major challenges in counter-terrorism analysis involves connecting the relatively few and sparse terrorism-related dots embedded within massive amounts of data flowing into the governments intelligence and counter-terrorism agencies. Information technologies have the potential to empower intelligence agencies or analysts with the ability to find pertinent data faster, conduct more efficient and effective analysis, share information with others if necessary, relay concerns to the appropriate decision-makers, and ultimately put the data into a form that allows senior decision-makers to understand and act on it so that they can anticipate and preempt terrorist plots or attacks from occurring. Advanced collaboration among multiple analysts or tools is one such crucial technology. In this paper, we introduce NEMESIS (network modeling environment for structural intervention strategies), a collaborative environment to integrate and share information among different counter-terrorism analysis tools. Two component tools, ASAM (adaptive safety analysis and monitoring system) and ORA (organizational risk analysis), are described in this paper. The functionality of these two tools, along with the NEMESIS collaboration is illustrated via a real world example gleaned from open sources

Collaboration and modeling tools for counter-terrorism analysis

One of the major challenges in counter-terrorism analysis today involves connecting the relatively few and sparse terrorism-related dots embedded within massive amounts of data flowing into the governments intelligence and counter-terrorism agencies. Information technologies have the potential to empower analysts with a superior ability to process and analyze the data, disseminate and share it, and ultimately put the data into a form that allows senior decision-makers to understand and act on it so that they can anticipate and ultimately preempt terrorist plots or attacks from occurring. Advanced collaboration among multiple analysts or tools is one such crucial technology. We introduce NEMESIS, a collaborative environment to integrate and share information among different modeling tools. Two component-modeling tools, ASAM System and ORA, are described in this paper. The functionality of these two tools along with the NEMESIS system is illustrated via a real world example gleaned from open sources.

Designing the Best Team for the Task: A Method that Combines Algorithms, Heuristics, and Expert Judgment

This paper describes a systematic, formal, quantitative approach for designing teams. The method develops a quantitative hierarchical mission or goal structure that specifies the parallelism, sequence, and structure of the tasks that must be accomplished in order to complete the teams mission. Using that quantitative task structure, we apply a systems engineering approach that describes organizational performance criteria as a multi-variable objective function to be optimized. We work closely with subject matter experts to develop the task structure and to specify the criteria to be optimized for the team (e.g., the distribution of workload among team members or the time needed to complete the mission). This process is iterative, with team designs produced, reviewed by domain experts, and modified. The method is currently being applied to two organizational design problems for Navy teams, and shows promise as a tool for human factors practitioners dealing with team design and organizational redesign issues.

Normative Design of Project-Based Adaptive Organizations

Over the past few years, mathematical and computational models of organizations have attracted a great deal of interest in various fields of scientific research (see Lin & Carley, 1993 for review). The mathematical models have focused on the problem of quantifying the structural (mis)match between organizations and their tasks. The notion of structural congruence has been generalized from the problem of optimizing distributed decision-making in structured decision networks (Pete, Pattipati, Levchuk, & Kleinman, 1998) to the multi-objective optimization problem of designing optimal organizational structures to complete a mission, while minimizing a set of criteria (Levchuk, Pattipati, Curry, & Shakeri, 1996, 1997, 1998). As computational models of decision-making in organizations began to emerge (see Carley & Svoboda, 1996; Carley, 1998; Vincke, 1992), the study of social networks (SSN) continued to focus on examining a network structure and its impact on individual, group, and organizational behavior (Wellman & Berkowitz, 1988). Most models, developed under the SSN, combined formal and informal structures when representing organizations as architectures (e.g., see Levitt et al., 1994; Carley & Svoboda, 1996). In addition, a large number of measures of structure and of the individual positions within the structure have been developed (Roberts, 1979; Scott, 1981; Wasserman & Faust, 1994; Wellman, 1991).