Neelam Naikar

Designing teams for first-of-a-kind, complex systems using the initial phases of cognitive work analysis: case study.

We present a technique for team design based on cognitive work analysis (CWA). We first develop a rationale for this technique by discussing the limitations of conventional approaches for team design in light of the special characteristics of first-of-a-kind, complex systems. We then introduce the CWA-based technique for team design and provide a case study of how we used this technique to design a team for a first-of-a-kind, complex military system during the early stages of its development. In addition to illustrating the CWA-based technique by example, the case study allows us to evaluate the technique. This case study demonstrates that the CWA-based technique for team design is both feasible and useful, although empirical validation of the technique is still necessary. Applications of this work include the design of teams for first-of-a-kind, complex systems in military, medical, and industrial domains.

Evaluating Design Proposals for Complex Systems with Work Domain Analysis

In this paper we propose a new framework for evaluating designs based on work domain analysis, the first phase of cognitive work analysis. We develop a rationale for a new approach to evaluation by describing the unique characteristics of complex systems and by showing that systems engineering techniques only partially accommodate these characteristics. We then present work domain analysis as a complementary framework for evaluation. We explain this technique by example by showing how the Australian Defence Force used work domain analysis to evaluate design proposals for a new system called Airborne Early Warning and Control. This case study also demonstrates that work domain analysis is a useful and feasible approach that complements standard techniques for evaluation and that promotes a central role for human factors professionals early in the system design and development process. Actual or potential applications of this research include the evaluation of designs for complex systems.

Work Domain Analysis for Training-System Definition and Acquisition

Training-needs analysis is critical for defining and procuring effective training systems. However, traditional approaches to training-needs analysis are not suitable for capturing the demands of highly automated and computerized work domains. In this article, we propose that work domain analysis can identify the functional structure of a work domain that must be captured in a training system, so that workers can be trained to deal with unpredictable contingencies that cannot be handled by computer systems. To illustrate this argument, we outline a work domain analysis of a fighter aircraft that defines its functional structure in terms of its training objectives, measures of performance, basic training functions, physical functionality, and physical context. The functional structure or training needs identified by work domain analysis can then be used as a basis for developing functional specifications for training systems, specifically its design objectives, data collection capabilities, scenario genera...

Use of Cognitive Work Analysis Across the System Life Cycle: From Requirements to Decommissioning:

Cognitive Work Analysis (CWA) is a systems-based approach to the analysis, modelling, and design of complex sociotechnical systems that is particularly useful when working with real-time work domains in which operator adaptation and flexibility may be needed (Rasmussen, Pejtersen, & Goodstein, 1994; Vicente, 1999). In this paper we argue that CWA can be used not only for design, with which it is most commonly associated, but also throughout the system life cycle. We present a table that shows the five phases of CWA crossed with different steps and activities in the system life cycle, and in the cells of the table we indicate how a particular phase of CWA informs the system life cycle activity in question. We illustrate this discussion with material from our own work using CWA in air defence environments, such as the use of work domain analysis in the tender evaluation for Australias AEW&C system. CWA not only leverages and coordinates some previous human engineering techniques, but it also adds important analytic products that have been absent from previous techniques.

Crossing the boundaries of safe operation: An approach for training technical skills in error management

Robust error management within the cockpit is crucial to aviation safety. Crew resource management (CRM) focuses on non-technical skills for error management but the training of technical skills for error detection and error recovery is also a potentially valuable strategy. We propose a theoretical basis for training technical skills in error management as well as a cognitively oriented technique for analysing accidents and incidents to identify specific training requirements. To evaluate the strengths and limitations of this new approach, we present a case study of its application to the F-111, a strike aircraft in the Royal Australian Air Force. This case study demonstrates that the new training approach is both feasible and useful, although an empirical validation of the approach is still necessary. In addition, the case study highlights the limitations of the current F-111 simulator for training technical skills for error detection and error recovery.

The Use of Work Domain Analysis for the Design of Training Systems

In this paper we argue that specifications for training equipment must be based on statements of mission-system functionality. To develop a good description of functionality is a difficult technical challenge, and the methodology of Work Domain Analysis has been developed for that purpose. However, a Work Domain Analysis does not fully specify the devices that are needed for training. Other forms of analysis and inference are needed to resolve issues of crititicality, instructional functions, implementation of functions in a training device, and fidelity of training device features. In this paper we explain the means of moving from functional requirements as developed by a Work Domain Analysis to specifications for training equipment.

Analysing Activity for Future Systems

Future systems pose special challenges that preclude many standard forms of work analysis for identifying the cognitive demands of those systems. Yet identifying the cognitive demands of future systems early during development is critical for system effectiveness. In this paper we demonstrate that Cognitive Work Analysis offers an approach for capturing the cognitive demands of future systems. Specifically, we illustrate the kinds of cognitive models that result from conducting a Cognitive Work Analysis of a future system called Airborne Early Warning and Control (AEW&C). These models characterise the cognitive activity of AEW&C in terms of: the work domain, decision making, and mental strategies. We also discuss actual and potential applications of these analyses for team and interface design and we show how we have applied this approach to design a team for AEW&C.

Temporal coordination control task analysis for analysing human-system integration

Evaluating whether prospective air defense systems will effectively integrate the activities of humans and computers is a pressing issue for developers and purchasers of such systems. We need effective ways of modeling human-system integration while such systems are under development and while the possibilities for action are underspecified. We present a Cognitive Work Analysis-based analytic framework under development for characterising air defense control tasks and also for characterising the larger framework in which such control tasks must be coordinated. The challenge is to produce a formative rather than descriptive or normative model, within which broad logical and temporal constraints can be represented, so revealing possibilities for action as events unfold.

Using Cognitive Work Analysis to Develop a Capability for the Evaluation of Future Systems

Evaluating complex human-machine systems is a multidimensional problem that requires the analyst to specify the characteristics of the system to be evaluated and to specify the method of evaluation to be used in determining its effectiveness. Whereas current approaches using task analysis techniques are used widely for evolutionary systems, other approaches, such as those from within the Cognitive Work Analysis (CWA) framework, may be better suited to revolutionary systems. This paper presents ongoing work directed at developing a human-in-the-loop simulation capability for revolutionary, or novel, systems that aims to evaluate the impact of a subsystems performance (e.g. radar detection range) on the overall systems purpose (e.g. the safe transportation of troops). Two aspects of the development of this capability are presented: 1) the use of Work Domain Analysis and Control Task Analysis to represent systems for the purpose of providing measures of performance for human-in-the-loop experiments performed in support of system evaluation and 2) criteria for judging the utility of a CWA approach for system evaluation compared to other more traditional techniques.

Work Domain Analysis for Identification of Training Needs and Training-System Design

Training needs analysis is critical for designing effective training systems. However, traditional techniques for analysing training needs are inadequate for capturing the demands of modern workplaces, where the primary role of workers is to adapt on-line to cope with unpredictable contingencies. By identifying the functional structure of a work domain, which is independent of particular events, work domain analysis provides a framework for informing the design of training systems that will allow workers to be trained to deal with unpredictable situations. To illustrate this argument, we outline a work domain analysis of a fighter aircraft and show how it may be used for identifying the training needs of fighter pilots and for deriving functional specifications for a training system for this work domain.