William F. Ormsby

Expanding our horizons in verification, validation, and accreditation research and practice

Many different types of modeling and simulation (M&S) applications, consisting of a combination of software, hardware, and humanware, are used in dozens of disciplines under diverse objectives including acquisition, analysis, education, entertainment, research, and training. Certification of sufficient accuracy of an M&S application by conducting verification, validation, and accreditation (VV&A) requires multifaceted knowledge and experience, and poses substantial technical and managerial challenges for researchers, practitioners, and managers. The challenges can only be met by using a very broad spectrum of approaches and expanding our horizons in VV&A. This paper presents 13 strategic directions to meet those challenges. The strategic directions provide guidelines for successful VV&A research and practice.

Planning for verification, validation, and accreditation of modeling and simulation applications

A comprehensive and detailed verification, validation and accreditation (VV&A) plan and its proper execution are crucially important for the successful accreditation of a modeling and simulation (M&S) application. We provide guidance in developing and executing such a plan throughout the entire M&S application development life cycle. The U.S. Department of Defense (DoD) is the largest sponsor and user of modeling and simulation applications in the world. DoD uses many different types of M&S applications, consisting of a combination of software, hardware, and humanware, under diverse objectives including acquisition, analysis and training.

Conceptual modelling for designing large-scale simulations

Large-scale complex simulations take many years to develop and cost millions of dollars. Development of such simulations requires many areas of expertise and poses significant technical and managerial challenges. Some of the challenges can be met with the development and use of a conceptual model (CM). This paper describes a life cycle for the development of a simulation CM. The role of the CM in large-scale complex simulation model design is explained. The objectives under which a simulation CM should be created and used are enunciated. The life cycle presented provides a structured blueprint of simulation CM development and specifies the work products to be created under the designated processes together with the integrated verification and validation activities.

Achieving reusability and composability with a simulation conceptual model

Reusability and composability (R&C) are two important quality characteristics that have been very difficult to achieve in the Modelling and Simulation (M&S) discipline. Reuse provides many technical and economical benefits. Composability has been increasingly crucial for M&S of a system of systems, in which disparate systems are composed with each other. The purpose of this paper is to describe how R&C can be achieved by using a simulation conceptual model (CM) in a community of interest (COI). We address R&C in a multifaceted manner covering many M&S areas (types). M&S is commonly employed where R&C are very much needed by many COIs. We present how a CM developed for a COI can assist in R&C for the design of any type of large-scale complex M&S application in that COI. A CM becomes an asset for a COI and offers significant economic benefits through its broader applicability and more effective utilization.

Well-defined intended uses: an explicit requirement for accreditation of modeling and simulation applications

A modeling and simulation (M&S) application is built for a specific purpose and its acceptability assessment is carried out with respect to that purpose. The accreditation decision for an M&S application is also made with respect to that purpose. The purpose is commonly expressed in terms of intended uses. The quality of expressing the intended uses significantly affects the quality of the acceptability assessment as well as the quality of making the accreditation decision. The purpose of this paper is to provide guidance in proper definition of the intended uses. It uses an M&S application for simulating the U.S. National Missile Defense (NMD) system design as an example to illustrate the definition of the intended uses.

Network-centric military system architecture assessment methodology

Recent paradigm shift from considering software as a product to treating software as a service has produced capabilities for engineering complex Network-Centric Military Systems (NCMSs). As the industry works on building a cyber-infrastructure and the Department of Defence moves towards network-centric operations and warfare, military systems are becoming increasingly more network-centric. Assessment of the architecture of such a NCMS is a very complex process, involves the measurement and evaluation of hundreds of qualitative and quantitative elements, mandates subject matter expert evaluation, and requires the integration of disparate evaluations obtained by experimentation, demonstration, trial, testing, direct measurement, analysis and examination. Planning and managing such measurements and evaluations require a unifying methodology and should not be performed in an ad hoc manner. This paper presents such a methodology named Military System Architecture Assessment Methodology (MSAAM). MSAAM advocates mission-oriented risk-driven architecture assessment based on four assessment perspectives: product, process, people and project. It enables the decomposition of architecture assessment into small pieces corresponding to leaf indicators and provides a structured framework for overcoming the complexity of architecture assessment.

Quality Assessment of Modeling and Simulation of Network-Centric Military Systems

Modeling and simulation (M&S) of network-centric military systems poses significant technical challenges. A network-centric military system (also known as network-centric operations, network-centric warfare or FORCEnet) is a system of systems aligning and integrating other systems such as battlefields, computers, databases, mobile devices, people (users), processes, satellites, sensors, warriors, shooters, and weapons into a globally networked distributed complex system. Characteristics of a network-centric military system are described using a layered architecture. Challenges for M&S of network-centric military systems are presented. The paper focuses on the quality assessment challenge and advocates the use of a quality model with four perspectives: product, process, project, and people. A hierarchy of quality indicators is presented for network-centric military system M&S. An approach is described for conducting collaborative assessment of M&S quality using the quality indicators.