Andreas Gregoriades

Scenario-based assessment of nonfunctional requirements

This paper describes a method and a tool for validating nonfunctional requirements in complex socio-technical systems. The system requirements analyzer (SRA) tool validates system reliability and operational performance requirements using scenario-based testing. Scenarios are transformed into sequences of task steps and the reliability of human agents performing tasks with computerized technology is assessed using Bayesian belief network (BN) models. The tool tests system performance within an envelope of environmental variations and reports the number of tests that pass a benchmark threshold. The tool diagnoses problematic areas in scenarios representing pathways through system models, assists in the identification of their causes, and supports comparison of alternative requirements specifications and system designs. It is suitable for testing socio-technical systems where operational scenarios are sequential and deterministic, in domains where designs are incrementally modified so set up costs of the BNs can be defrayed over multiple tests.

Workload prediction for improved design and reliability of complex systems

This paper describes a method and a tool for analysing and predicting workload for the design and reliability of complex socio-technical systems. It concentrates on the need to assess workload early in the design phase to prevent systems failures. This is a continuation of our previous work on workload assessment. The method is supported by a tool that enables scenario-based validation of prospective socio-technical systems designs such as command and control rooms of military vessels. The approach combines probabilistic measures of human performance with subjective estimates of workload. The causal relationships of performance shaping factors (PSF) are modelled in a Bayesian belief network (BBN) and used to assess the agents operational performance and reliability. Workload for each agent is calculated based on demand placed upon agents in terms of behavioural response to tasks, communications and interactions between humans and technology. The approach uses scenarios to stress test prospective system designs, where each scenario is modelled as a sequence of events. Reliability is expressed in terms of human error and is dynamically assessed throughout test scenario executions using BBN technology. The innovation beyond traditional reliability analysis relies to the use of dynamic and static estimates of reliability inputs for better informed assessment. This method enables identification of performance bottlenecks to be addressed by the designer early in the design phase. A case study is presented that demonstrates the use of the method and tool for the design of the command and control room of a military vessel.

A socio-technical approach to business process simulation

This paper describes a socio-technical approach to business process redesign through the investigation of complex interactions and dependencies among humans and IT systems of organisations. The focus is on the need to assess business process performance early in the redesign process, to prevent organisational failures. The method is based on human performance quantification and is supported by a tool that enables scenario-based evaluation of prospective organisational processes through simulation. The approach combines probabilistic and subjective measures of tasks and communication acts in business processes to quantify business performance in terms of cycle time. The approach models business processes as a set of scenarios of sequential activities where the dependencies between actors, IT systems and tasks are explicitly defined. Business process performance assessment is achieved through a systematic walkthrough of the process model using these scenarios. Human performance constitutes an important parameter to business process performance and is modelled based on Human Performance Shaping Factors (PSF) and assessed using Bayesian Belief Networks (BBN). Process cycle time is calculated using aggregates of task and communication completion times, and calibrated using performance estimates of each of the agents in the scenario. The method enables trade-off analysis among candidate process models and identification of performance bottlenecks early in the design phase. A radiology process improvement case study is presented that demonstrates the use of the method and the tool.

Automating Scenario Analysis of Human and System Reliability

The system reliability analyzer tool for analyzing the reliability of system designs is described and its use illustrated in a system engineering case study of a naval command and control system. The performance of systems consisting of human operators and technology components is assessed by Bayesian nets, which calculate error probabilities from inputs of agent properties and environmental conditions. The tool tests scenarios representing the system design and its operational behavior, which is modeled as cycles of command and control tasks. The tool indicates weak points in the scenario sequence and assesses the reliability of one or more system designs with a set of operational scenarios and a variety of environmental conditions

Unifying business objects and system dynamics as a paradigm for developing decision support systems

Due to the market-driven nature of modem organisations, it is important that they can easily adapt to changing business needs. In order to be able to do so, organisations need to employ information systems that exhibit the important characteristic of adaptability. Change, however, is risky because it encompasses unpredictable behaviours. Organisations, in order to minimise this risk, employ decision support systems (DSS) techniques that enable predictions to be made. This paper describes a simulation methodology, based on the combination of business objects and system dynamics that assists organisations in predicting future behaviours. The methodology eliminates the need for duplicate models of enterprise operation and simulation, and introduces a framework that enables the unification of the two in a single model.

Scenario advisor tool for requirements engineering

This study investigates the usefulness of a scenario advisor tool which was built to help requirements engineers to generate sufficient sets of scenarios in the domain of socio-technical systems. The tool provides traceability between scenario models and requirements and helps to generate new scenarios and scenario variations. Through two series of evaluation sessions, we found that the scenario advisor tool helped users to write more sound scenarios without any domain knowledge, and to generate more variations on existing scenarios by providing specific scenario-generation hints for each scenario component. The tool should improve the reliability of requirements elicitation and validation.

Human-Centered Safety Analysis of Prospective Road Designs

One of the most important issues in road safety management is the lack of reliable methods for predicting the likelihood of accidents. Road safety assessment systems have been developed; however, these systems only employ historical or retrospective analyses, and the human factor element is weak or missing. Effective safety management requires both holistic and prospective viewpoints, with human factors having an intrinsic role. The main goal of this paper is to contribute toward that need through the application of Bayesian belief networks and road traffic simulation for validating the safety requirements of prospective road designs. The theoretical platform of the method is the concepts of human performance and mental workload and how these affect accident likelihood. This paper presents a novel method and a tool that integrates these two mature technologies, for assessing the safety performance of road designs before they are developed. A case study is included that illustrates the application of the method and tool.

Assessing critical success factors for military decision support

This paper outlines the application of case-based reasoning and Bayesian belief networks to critical success factor (CSF) assessment for parsimonious military decision making. An important factor for successful military missions is information superiority (IS). However, IS is not solely about minimising information related needs to avoid information overload and the reduction of bandwidth but it is also concerned with creating information related capabilities that are aligned with achieving operational effects and raising operational tempo. Moreover, good military decision making, should take into account the uncertainty inherent in operational situations. Herein, we illustrate the development and evaluation of a smart decision support system (SDSS) that dynamically identifies and assesses CSFs in military scenarios and as such de-clutters the decision making process. The second contribution of this work is an automated configuration of conditional probability tables from hard data generated from simulations of military operational scenarios using a computer generated forces (CGF) synthetic environment.

Towards a user-centred road safety management method based on road traffic simulation

One of the most important gaps in road safety management practises is the lack of mature methods for estimating reliability. Road safety performance assessment systems have been developed; however, these provide only historical or retrospective analyses. Effective safety management requires a prospective viewpoint. The main goal of this research is to assist in reducing accident rates in Cyprus by providing ample time to the authorities to react to high risk situations through a safety prediction early warning system. This ultimately will prevent accidents from occurring which subsequently could save lives. Traditional approaches focuses solidly on empirical data concerning road network dynamic properties, despite the fact that the most vulnerable component of the system is the human element. This paper described the integration of agent-based simulation with Bayesian Belief Networks (BBN) for improved quantification of accident probability. The BBN is developed using multidisciplinary influences.

Automated assistance for human factors analysis in complex systems.

A tool and a method for scenario-based workload assessment and performance validation in complex socio-technical systems design, such as command and control rooms of military vessels, are described. We assess workload in terms of communication and the task load that each agent is able to handle. The method employs subjective task and communication estimates used to calculate the workload of human operators, using static and scenario-based analyses. This enables the identification of bottlenecks to be addressed by the designer with the appropriate allocation of function between humans and smart technology. This task is supported by the functional allocation adviser tool. A case study demonstrating the use of the tool for the design of the command and control room of a military vessel is presented.