Yaman Barlas

Formal aspects of model validity and validation in system dynamics

Model validation constitutes a very important step in system dynamics methodology. Yet, both published and informal evidence indicates that there has been little effort in system dynamics community explicitly devoted to model validity and validation. Validation is a prolonged and complicated process, involving both formal/quantitative tools and informal/qualitative ones. This paper focuses on the formal aspects of validation and presents a taxonomy of various aspects and steps of formal model validation. First, there is a very brief discussion of the philosophical issues involved in model validation, followed by a flowchart that describes the logical sequence in which various validation activities must be carried out. The crucial nature of structure validity in system dynamics (causal-descriptive) models is emphasized. Then examples are given of specific validity tests used in each of the three major stages of model validation: Structural tests, structure-oriented behavior tests and behavior pattern tests. Also discussed is if and to what extent statistical significance tests can be used in model validation. Among the three validation stages, the special importance of structure-oriented behavior tests is emphasized. These are strong behavior tests that can provide information on potential structure flaws. Since structure-oriented behavior tests combine the strength of structural orientation with the advantage of being quantifiable, they seem to be the most promising direction for research on model validation.

A dynamic model of salinization on irrigated lands

A dynamic simulation model of salt accumulation on irrigated lands is presented. The original version of the model is part of a large-scale socio-economic model of irrigation-based regional development. The model introduced in this paper is a systemic one in the sense that it integrates four major sub-processes of rootzone salinization: irrigation, drainage, groundwater discharge and groundwater intrusion. It provides a comprehensive and general description of the long-term process of salt accumulation in lowlands under continuous irrigation practice, where irrigated lands are annually increased. Analysis of the model and simulation results reveal, under what conditions the salinity reaches alarming levels and with what strategies it can be controlled. For instance, in situations where the mixing of drainage water into irrigation water supplies is high, rootzone salinity quickly reaches alarming levels. More importantly, in this setting, the typical strategy of increasing the drainage in order to control the salinity level yields unprecedented exponentially growing salinity levels, a catastrophic result for the agriculture. The model structure can represent the basin wide salinization process on different geographical settings in agricultural development. In general, the model provides an experimental simulation platform, which can be used by the policy makers in the long term strategic management of large scale irrigation development projects. The model can also be of interest to the students and learners in teaching and research, in the related fields of environmental sciences.

Effects of patch connectivity and arrangement on animal metapopulation dynamics: a simulation study

Abstract We constructed a simulation model of metapopulation dynamics consisting of two or three habitat patches using STELLA. our simulations show that, given the assumptions of the deterministic model, the metapopulation is doomed to global extinction with or without interpatch immigration when all local populations are below minimum viable population (MVP) size. This suggests that for a cluster of scattered small populations, it is preferable to focus on augmenting individual population sizes rather than enhancing interpatch immigration. In the case when at least one of the subpopulations is above the MVP size, there is a critical size for that subpopulation above which the metapopulation persists and otherwise collapses. Also, when a metapopulation system is composed of more than two patches, the spatial configuration in terms of patch connection and the relative position of the above-MVP subpopulation will have significant effects on metapopulation dynamics and persitence. All simulation results from the three-patch animal metapopulation model suggest that both the number of interpatch connections and the magnitude represented by them are crucial for overall patch connectivity. The magnitude of interpatch immigration is positively related to the minimum size of the above-MVP subpopulation in both the two- and three-patch metapopulation systems due to population sink effect. The phenomenon is especially significant when subpopulations in sink patches are well below MVP. Appropriate introduction of stochastic components into the model may increase its realism especially for the cases where all subpopulations are well below MVP. Although the current version of the model involves no more than three patches, it may serve as a general conceptual framework and a specific simulator for modeling metapopulation dynamics incorporating a variety of spatial arrangements of habitat patches.

Demand forecasting and sharing strategies to reduce fluctuations and the bullwhip effect in supply chains

Supply chain inventories are prone to fluctuations and instability. Known as the bullwhip effect, small variations in the end item demand create oscillations that amplify throughout the chain. By using system dynamics simulation, we investigate some of the structural sources of the bullwhip effect, and explore the effectiveness of information sharing to eliminate the undesirable fluctuations. Extensive simulation analysis is carried out on parameters of some standard ordering policies, as well as external demand and lead-time parameters. Simulation results show that (i) a major structural cause of the bullwhip effect is isolated demand forecasting performed at each echelon of the supply chain, and (ii) demand and forecast sharing strategies can significantly reduce the bullwhip effect, even though they cannot completely eliminate it. We specifically show how each policy is improved by demand and forecast sharing. Future research involves more advanced ordering and forecasting methods, modelling of other well-known sources of bullwhip, and more complex supply network structures.

Dynamic modelling of a shallow freshwater lake for ecological and economic sustainability

Abstract This research deals with the dynamic simulation modelling of a shallow freshwater lake ecosystem and analysis of potential sustainable management policies. The study region consists of a shallow freshwater lake and its surroundings, where fishing is a major commercial activity. The lake is under high nutrient loads, hence eutrophic with macrophyte dominance. The goal of this research is to find a balance between the ecosystem and economic activities in the region. To this end, a system dynamics model of the wetland is constructed. The results obtained from model simulations show that there is no threat of a shift to algal dominance in the near future. The major problem seems to be a potential decline in the welfare of the inhabitants, mainly due to unsustainable population increase. Different scenario runs reveal that the lake would have become eutrophic with algal dominance, if the crayfish population did not collapse due to a fungus disease in 1986. One particular scenario analysis (the recovery of crayfish sometime in the future within the model time frame) results in increase in crayfish harvest; hence in income from fishing, leading to betterment in social conditions. As for the alternative policies tested, ‘improved agricultural techniques’ is the only policy that leads to better social conditions, through increased yield per hectare. It is hoped that the dynamic simulation model will serve as a laboratory to study the different features of the eutrophication problem in shallow freshwater lakes and to analyse different policy alternatives with an integrated, systemic approach.

An autocorrelation function test for output validation

Output validation is a major aspect of simulation validation in general. In continu ous simulation (e.g., System Dynamics), output validation involves demonstrating that the model is able to reproduce the dynamic time patterns that have been observed in the behavior of the real system. But standard sta tistical tests cannot be readily used to make such dynamic time pattern com parisons (i.e. periodicities, amplitudes, trends...) In this paper, we propose an output validity test that addresses the above need. The test consists of comparing the autocorrelation functions of the observed and model-generated outputs. Using simulation experiments, we demonstrate the application of the test to different situations, such as systems with large noise, observation errors, and models with parameter errors. The test is shown to detect significant errors in the fundamental periods of the observed and model-generated outputs. The test can also determine if the observed output has high frequency noise components (such as observa tion errors) not present in the model output. Experiments also demonstrate that interpreting the results of the autocorrelation function test is rather intuitive and simple, which is a major advantage of the test.

Tests of Model Behavior that can Detect Structural Flaws: Demonstration with Simulation Experiments

Validation of System Dynamics (SD) models involves two general types of tests: Tests of model structure and tests of model behavior. Yet, since SD models are causal models, the essence of model validity lies always in structural validity: “Right behavior for the right reasons”. (The nature of SD model validity has been discussed in various SD literature. For example, Forrester 1961 (Chapter 13); Forrester 1968; Forrester and Senge 1980; Bell and Senge 1980; Richardson and Pugh 1981 (Chapter 5 and 6) and Barlas 1985). Standard behavior tests, which compare the model-generated behavior to the observed reference behavior are generally “weak” in SD context, since they can not separate spurious behavior accuracy (“right behavior for the wrong reasons”) from true behavior validity. Such tests provide no structural information. Structure tests, on the other hand, are “strong” tests, since they evaluate the model structure directly. But their their major drawback is that they are informal, qualitative, hence difficult to communicate. (See Forrester and Senge 1980, and Richardson and Pugh 1981 (chapter 5) for some specific behavior and structure tests). Thus, it seems like a “third type of test” would be most appropriate for SD validation purposes: Quantitative/formal behavior tests that can provide some structural information (“structurally-oriented” behavior tests). Interestingly, such tests already exist in the SD validation “repertoire”. (For example, “Extreme Condition” simulations, “Behavior Sensitivity” testing, in Forrester and Senge 1980 and Richardson and Pugh 1981). But these tests are usually listed along with all other behavior tests, which undermines the major difference that exists between the former and the latter.

Dynamic simulation for strategic insurance management

In this paper, a dynamic model-based management consultancy project carried out for a major insurance company in Turkey is presented. The objective of the project was to address certain strategic managerial problems of the company by using systemic dynamic simulation. The main strategic problem of concern was that the company exhibited a fast growth between 1988 and 1993, followed by persistent stagnation and even a slight decline. This paper describes the main structures of the model, presents the validity tests and lists the major results of the study. The model is developed, calibrated and validated using real data for seven years, between 1989 and 1996. The main benefit of the model is that it generates a systemic and dynamic understanding of the company’s internal and external interactions so as to enable creative solutions for existing and potential problems. One of the recommendations of the project has actually been initiated as a pilot project. A new interactive gaming version of the model is in the final stages of completion. The model and the game version can be used as a ‘‘learning laboratory’’ in the company, which would be a first step toward ‘‘organizational learning’’. Copyright

A Dynamic Simulator for the Management of Disorders of the Body Water Homeostasis

A dynamic model is built to study the water regulation of human body and related disorders, focusing on the fundamental feedback mechanisms involved in their normal and abnormal physiology. The simulation model is extended to include therapeutic interventions related to the most common body fluid disorder, namely, water intoxication/hyponatremia. The modeling approach is based on system dynamics methodology. Comparisons with experimental and field data show that the model adequately reproduces typical dynamics of the body fluid variables in their normal and diseased states. Finally, an interactive game version is developed to test the possible effects of alternative treatment options on a simulated patient. Simulation and game results reveal the subtleties involved during and after administration of various pharmacological interventions. For example, hypertonic saline should be administered concurrently and in delicate balance with drugs that increase urine flow. The simulator offers a virtual laboratory for experimental research and education on diagnosis and alternative therapies of body water disorders in general and hyponatremia in particular.

A Comprehensive Model of Goal Dynamics in Organizations: Setting, Evaluation and Revision

Goal setting plays crucial role in decision making in organizations as well as in individuals. Most improvement activities consist of the following cycle: set a goal, measure and evaluate the current performance (against the set goal), take actions (e.g. training) to improve performance, evaluate and revise the goal itself if necessary, again measure and evaluate the performance against the current goal, and so on ... [Forrester 1975; Senge 1990; Lant 1992; Sterman 2000]. So goals constitute a base for the decisions and the managerial actions. In an organization, the performance level is evaluated against a goal and, further, the effectiveness of the goal itself can and must be periodically evaluated. Among various research methods to analyze the dynamics of goals in organizations (and individuals), an important one is simulation modeling – more specifically system dynamics modeling that is particularly suitable to model qualitative, intangible and ‘soft’ variables involved in human and social systems [Forrester 1961; Forrester 1994; Morecroft and Sterman 1994; Sterman 2000; Spector et al 2001]. System dynamics is designed specifically to model, analyze and improve dynamic socio-economic and managerial systems, using a feedback perspective. Dynamic strategic management