New Frontiers in the Validation of Simulation Models - Structural Dominance Analysis

Models should be relevant for coping with the complexity of the real world. At the same time, the methods by which they are constructed and validated must be rigorous; otherwise the quality of the model suffers.
We take a test of model structure, demonstrating a new technique of analysis of the model's structure called Structural Dominance Analysis (SDA). SDA has been developed in the science of complex systems [Kampmann and Oliva 2006; Kampmann and Oliva 2009]. SDA makes use of one axiom of System Dynamics: model structure influences the behavioral outcome of a simulation [Forrester 1993]. In principle, SDA evaluates the relative partial influence of individual feedback loops on the behavior of the variables of interest. Based on this evaluation, loop dominance then signifies which particular piece of model structure is dominant, i.e., most influential for the behavior in a certain period of time. Research about loop dominance has bred several formal and informal techniques to enable SDA [Kampmann and Oliva 2008]. In this paper, we revert to an behavioral technique by Ford [1999]. The rationale for using SDA as a validation method is to compare the discovered dominant model structures with the model structures the modeler or expert perceives as dominant in the real system under study. Especially, the comparison of the shifts of dominance between structures over time is a fruitful source for model validation. In this respect, the presented approach is based on a formal quantitative analysis, but for the interpretation it relies on a qualitative comparison of the model with the real system under study. The Structural Dominance Test (SDT) is a means of validating the macro-structure of a simulation model. To apply SDA for the issue of validation is a novelty of this paper. Its application requires a feedback based, continuous-time simulation model, as used in System Dynamics. A brief case study demonstrates the new technique for validation purposes.