1. Introduction

The importance of a conceptual model (CM) during the simulation development life cycle has been recognized more and more as the complexity and size of simulations have increased during the past decade. Emphasizing this importance, Robinson devoted two chapters on conceptual modelling in his book (Robinson, 2004) and provided a comprehensive literature survey (Robinson, 2006).

A simulation model goes through three major layers of abstraction during its development life cycle as depicted in Figure 1.

Figure 1.

Layers of simulation model abstraction.

Full figure and legend (15K)

The lowest abstraction layer 'simulation model implementation/programming' is accomplished by using programming languages (eg C++, Java, and C#) or simulation software products. In discrete-event simulation, the 'simulation model design' abstraction layer is transformed into the 'simulation model implementation/programming' abstraction layer by using strategies such as event scheduling, activity scanning, three-phase approach, and process interaction (Balci, 1988). In distributed simulation, this transformation is typically accomplished by using the High-Level Architecture (HLA), which is a US Department of Defense (DoD), Object Management Group (OMG), and IEEE Standard (IEEE, 2000). In continuous simulation, a software product for solving partial differential equations is used for this transformation.

The middle abstraction layer 'simulation model design' is created by using either object-oriented or procedural paradigm. Under object-oriented model design, modularization is advocated for the abstraction in such a way that each module (submodel) possesses the highest possible cohesion and the lowest possible coupling to enable maintainability and reusability. Different types of simulation follow different abstraction approaches for this layer. For example, in continuous simulation, model design consists of a set of partial differential equations. In system dynamics type of simulation, a set of equations for levels and rates forms the model design. In discrete-event simulation, model design may represent a federation of models distributed on different computers over the Internet.

We view and define a simulation CM as the highest layer of abstraction that is closer to the level of thinking of a simulation modeller.

The main purpose of this paper is to describe a life cycle for the development of a simulation CM. Section 2 defines simulation CM, explains its role in large-scale complex simulation model design, and enunciates the objectives under which a simulation CM should be created and used. Section 3 presents a simulation conceptual modelling life cycle, which we created and used for the development of a multimedia simulation CM for the Battlespace Environment and Signatures Toolkit (BEST) project (NRL, 2004) funded by the US Missile Defense Agency (MDA). The concluding remarks are given in Section 4.

2. Simulation conceptual model

A simulation Conceptual Model (CM) is the model formulated in the mind of the simulation modeller and specified in a variety of communicative forms intended for different users such as managers, analysts, and simulation developers as depicted in Figure 2.

Figure 2.

Simulation CM definition.

Full figure and legend (175K)

In the development life cycle of a large-scale simulation (Balci, 2003), conceptual modelling should be conducted either immediately before or after requirements engineering (DMSO, 2006). In some cases, a CM can be used to identify, elicit, and specify modelling and simulation (M&S) requirements and precedes requirements engineering (Balci, 2003); in some other cases, a CM can be developed based on a given M&S requirements specification document and follows requirements engineering. In any case, conceptual modelling must always precede the design of a simulation model.

Simulation is used by a community of interest (COI) such as air traffic control, automobile manufacturing, Ballistic Missile Defense (BMD), business process reengineering, computer networks, military training, network-centric operations and warfare, supply chain management, telecommunications, and transportation. Each COI faces serious problems in

1. reusing earlier developed simulation models or model components (submodels),
2. communication among the stakeholders, managers, analysts, and simulation developers,
3. coping with multidisciplinary knowledge required for simulation model development,
4. overcoming the complexity of large-scale simulation model design, and
5. verification, validation, and certification (VV&C) of large-scale complex simulation models.

We advocate the development and use of a CM to assist in the design of not just one simulation model but many within the problem domain of a COI and to alleviate the problems listed above. A CM becomes an asset for a COI and provides significant economical benefits through its reuse in designing many simulation models within the problem domain of that COI.

Reuse of simulation models/submodels at the lowest abstraction layer 'implementation/programming' (Figure 1) is extremely difficult if not impossible due to the use of different programming languages (eg C++, Java, and C#), operating systems (eg Unix and Windows), hardware platforms (eg Intel and Sparc), and commercial simulation software products (eg Arena, AutoMod, and OpNet). The reuse at the middle abstraction layer 'design' (Figure 1) is possible if the same paradigm (eg object-oriented and procedural) is used. For example, an object-oriented model design represented in unified modelling language (UML) can be reused for another object-oriented model design; however, for large-scale simulations, high complexity of design hinders reuse. Design reuse also becomes difficult or infeasible from one type of simulation to another (eg continuous, discrete-event, distributed/parallel, and system dynamics). We believe that reuse can be more effectively accomplished for large-scale simulations of any type at the 'simulation conceptual model' abstraction layer (Figure 1). Yilmaz and Ören (2004) present a CM for reusable simulations under the conceptual framework of a model–simulator–experimental frame.

A multi-million-dollar large-scale M&S project involves dozens of people with different backgrounds and areas of expertise such as sponsoring agency representatives, stakeholders, potential users of the simulation model, program managers, project managers, analysts, engineers, and M&S developers. One large M&S project we have been involved with had a team of about 120 people and a budget of more than $30 million annually. Communication among the members of such a large team poses significant challenges not just because of the large team size but also because of the diversity of the areas of expertise required in developing the simulation model. A multimedia CM can be created to facilitate the communication among the people involved in a large-scale M&S project. We developed a multimedia CM for the BEST project, which was provided on a server computer and accessed over the Internet using a web browser through a secure connection.

A person may be an expert on M&S but may not have sufficient knowledge of the problem domain for developing a simulation model. Subject matter experts (SMEs) join the M&S project team to provide the needed expertise. For example, the BEST project team included SMEs with expertise on aerospace engineering, BMD, chemistry, engineering science and mechanics, missile systems engineering, M&S, physics, and software engineering. SMEs need to collaborate with each other for developing a simulation model. Due to the diversity of areas of expertise, it is difficult for one SME to clearly understand another SME's work. In this situation, a CM can be built to provide background knowledge and higher-level multimedia descriptions of an area of expertise in a way that the CM enables a SME to understand another SME's work and facilitates the collaboration among the SMEs for designing a large-scale complex simulation model.

Building a CM to provide such a multimedia knowledge base is certainly very challenging and time consuming; however, the CM is built and maintained for use many times for many M&S projects in that problem domain of a COI; therefore, the return on investment is realized with the reuse of the CM.

A CM should be developed to facilitate modularization or decomposition of a simulation model design in such a way that each model component or submodel is realized with the highest possible cohesion and the lowest possible coupling to promote reusability and maintainability. A CM should provide the problem domain-specific information and knowledge, architectural styles, and design patterns needed for the modularization. Proper modularization is critically important to overcome the complexity of simulation model design.

VV&C require creativity, insight, and knowledge of the problem domain (Balci, 1998, 2003). A simulation model is developed for a set of intended uses (IUs) and its acceptability is judged and is certified with respect to those IUs. Proper specification of the simulation model IUs is critically important (Balci and Ormsby, 2000). A CM should be developed to assist in proper formulation of the IUs and enable effective and efficient VV&C with respect to those IUs.

A CM is created, used, maintained, and reused for designing a simulation model in a particular problem domain of a COI. For example, we created a CM for the problem domain of M&S of ballistic missile trajectory, signatures, and phenomenology for the BMDCOI. BMD engineers and scientists have been working for more than a decade and continue to work on building models and simulations of ballistic missile trajectory, signatures (including the ones listed in Figure 4), and phenomenology. Our CM can be used for designing many simulations of any type (eg continuous, discrete-event, and physics-based) in this problem domain.

Figure 4.

BMD universe of discourse.

Full figure and legend (374K)

Similarly, a CM can be developed to assist, for example, in the design of traffic simulations in the traffic engineering problem domain of the transportation COI. Such a CM can assist in designing many simulations of any type in that problem domain.

A CM should be created and used under a number of objectives to overcome the problems stated above as problems (a) to (e) faced by a COI. We list 15 objectives below; however, this is not meant to be an exhaustive list and a CM can be created and used for other objectives.

A simulation CM should be created and used to:

1. assist in designing not just one simulation model but many in a particular problem domain of a COI;
2. assist in designing a simulation model of any kind (eg continuous, discrete-event, distributed/parallel, and system dynamics);
3. enable reuse at the conceptual abstraction layer in such a way that designing a large-scale complex simulation model within a COI's problem domain is significantly facilitated;
4. enable effective communication among the people involved in a large-scale M&S project such as stakeholders, potential users, managers, analysts, and simulation developers with a stratified specification in a variety of communicative forms such as animation, audio, chart, diagram, drawing, equation, graph, image, text, and video;
5. assist in overcoming the complexity of designing large-scale complex simulation models;
6. provide a multimedia knowledge base covering the areas of expertise needed for designing large-scale complex simulation models;
7. enable an SME involved in an M&S project to understand another SME's work from a technical perspective;
8. facilitate the collaboration among the SMEs for designing a large-scale complex simulation model;
9. assist in VV&C of large-scale complex simulation models;
10. enable effective and efficient VV&C of large-scale complex simulation models;
11. assist in the specifications of test designs, test cases, and test procedures;
12. guide the managers, analysts, and developers in designing large-scale complex simulation models;
13. assist in proper formulation of simulation model IUs;
14. assist in the generation of new M&S requirements; and
15. provide significant economical benefits through its reuse.

3. Simulation conceptual modelling life cycle

A simulation conceptual modelling life cycle describes the blueprint of CM development and specifies the work products to be created under the designated processes together with the integrated verification and validation (V&V) activities. A life cycle is critically needed to modularize and structure the CM development and provide valuable guidance for project management. A life cycle is also required to show the V&V activities as integrated within the CM development based on the principle that dictates that V&V must go hand in hand with the development life cycle (Balci, 1998).

Figure 3 presents the life cycle we developed and used for the BEST project. The work products (or artefacts) created during the life cycle are shown by shaded oval symbols. The dashed arrows represent the processes used in creating the work products. The solid arrows represent the 'V&V activities' integrated within the life cycle.

Figure 3.

Simulation conceptual modelling life cycle.

Full figure and legend (56K)

The dashed arrows are intended to show the direction of CM development and must not be interpreted to imply sequential execution. The life cycle is iterative in nature and we bounce back and forth between the work products until we have sufficiently high confidence in their quality. A deficiency or error identified in a 'V&V' activity may require the repetition of earlier processes and revision of earlier work products.

3.1. Problem formulation

Problem formulation (problem structuring or problem definition) is the process by which the universe of discourse is analysed to create a formulated problem, which is sufficiently well defined to enable specific action (Balci and Nance, 1985). This process takes the universe of discourse description as input and produces the work product formulated problem as output.

The universe of discourse refers to a specific problem domain for a COI. For example, for the BMDCOI, the universe of discourse can be defined as (a) a class containing all the entities referred to in a BMD discourse, or (b) as an inclusive class of entities that is tacitly implied or explicitly delineated as the subject of a statement, discourse, or theory related to BMD. Figure 4 shows the elements of the BMD universe of discourse including the following: space tracking and surveillance system (STSS), space-based infrared system (SBIRS), space-based laser (SBL), air bourne laser (ABL), in-flight interceptor communications system (IFICS), upgraded early warning radar (UEWR), ground-based interceptor (GBI), X-band radar (XBR), IFICS data terminal (IDT), ground-based radar-prototype (GBR-P), battle management command, control, and communications (BMC3), ground-based midcourse defense (GMD), sea-based midcourse defense (SMD), and theater high altitude air defense (THAAD).

Many M&S problem domains exist within the BMD universe of discourse depicted in Figure 4. For example, a COI tries to model and simulate the trajectory of an exoatmospheric ballistic missile for the IU of estimating the position of the missile after t time units given its current position. This estimation is critically important for deciding when to fire the ground-based or sea-based kill vehicle to intercept the incoming missile. Such simulation-based estimation is very complex since the trajectory is influenced by many factors including missile speed, gravity, temperature, composition of the missile hard body and its chemical interactions with atmosphere, and position of the sun and moon.

The process of problem formulation should aim to specify the real problem in its entirety in a structured manner (Balci and Nance, 1985). Key stakeholders and decision makers should be identified and their needs and objectives should be clearly and explicitly specified. A problem domain boundary should be established. A problem domain boundary is a logical and physical grouping of problem domain elements that will be taken into consideration for the simulation conceptual modelling. The boundary must be large enough to encapsulate all essential elements of the problem domain.

Formulated problem credibility is the degree to which the formulated problem is believable. The formulated problem credibility assessment deals with the integrative evaluation of (a) the quality of the formulated problem as a work product, (b) the problem formulation process used in creating the formulated problem, and (c) the project management characteristics (eg people, planning, and resources) related to this life-cycle stage.

Formulated problem quality deals with the evaluation of how well the problem definition is structured and how completely and accurately the real problem is included in the problem definition. The most challenging task of problem formulation is to identify and specify the entire real problem. Figure 5 illustrates three scenarios:

1. Scenario 1 represents the case where the formulated problem is completely disjoint from the real problem implying that the formulated problem is completely different than the real problem.
2. Scenario 2 represents the case where the formulated problem contains only some parts of the real problem.
3. Scenario 3 represents the ideal case where the formulated problem contains the entire real problem.

Figure 5.

Formulated problem scenarios.

Full figure and legend (53K)

Balci and Nance (1985) define Type III Error as the error of solving the wrong problem. It occurs in those cases represented by scenarios 1 and 2 in Figure 5. When Type III Error occurs, the problem solution becomes irrelevant. Therefore, Type III Error is extremely important and its probability of occurrence must be kept as small as possible.

Table 1 presents a number of indicators for estimating the probability of Type III Error and for evaluating how well the problem is formulated.

Table 1 - Formulated problem credibility assessment indicators.

Full table

We use the certification methodology presented by Balci (2001) and the evaluation environment (EE) web-based secure software system (Orca, 2003) for conducting collaborative evaluations using geographically dispersed SMEs for the life cycle V&V activities employing hundreds of indicators such as the ones given in Table 1.

We assume that the formulated problem can be solved by using simulation.

3.2. System investigation

In this life-cycle process, we use the term 'system' to refer to that portion of the universe of discourse related to the formulated problem (eg missile trajectory estimation). In other words, the system is the real-life entity or reality that contains the formulated problem. We define the system so as to establish the context based on which a simulation model will be created to solve a stated problem. In this process, we clearly and explicitly define the objectives that will guide the development of a simulation CM based on the context defined.

This process takes the formulated problem as input and produces the work product system and objectives definition as output.

In this process, a system boundary must be established. A system boundary is a logical and physical grouping of the formulated problem elements to be represented in the system definition and the simulation CM. Whatever is included in the system definition will be included in the CM representation. Whatever is excluded from the system definition will be left out from the CM.

Deciding what to include and what to exclude is considered to be an art. It is well to remember the dictum that (Elmaghraby, 1968)'Modelling is an artful balancing of opposites; on the one hand, a model should not contain unnecessary details and become needlessly complex and difficult to analyze; on the other hand, it should not exclude the essential details of what it represents'.

Identification of the system boundary plays a crucial role in the artful balancing of opposites and provides the foundation for conceptualizing a simulation model.

Characteristics of the system embodying the formulated problem must be identified for the purpose of taking them into consideration in developing the CM. The following system characteristics (Shannon, 1975) are commonly considered for model development and each characteristic should be examined with respect to the identified objectives.

1. Change: How often and how much the system of interest changes over a period of time will determine how often and how much the CM representation must be updated. Note that the CM is expected to be maintained and reused over many years for designing many simulations.
2. Environment: A system's environment consists of those problem domain elements left out of the system definition based on the identified system boundary. For example, a system definition for the formulated problem of missile trajectory estimation may exclude the stars. In this case, the stars become part of the environment. All problem domain elements left within the system's environment should be examined to assess the significance of their influence on the system's state with regard to the stated objectives. Underestimating the influence of an environment element may result in inaccurate system definition or improper CM formulation.
3. Counterintuitive behaviour: Refers to such behaviour that is contrary to what one would intuitively expect. Cause and effect are often not closely related in time or space. Symptoms may appear long after the primary causes. To be able to identify counterintuitive behaviour, it is essential that SMEs are used to the extent possible.
4. Drift to low performance: A system may show a drift to low performance over a period of time due to the deterioration of its components (eg some satellites have a life expectancy of 5 years, some machines deteriorate in performance over time). If this characteristic exists, it should be incorporated within the system definition and the CM should be built accordingly.
5. Interdependency: System interdependency characteristics should be examined prior to the abstraction of the real system for the purpose of modelling. In a complex system, many activities or events take place simultaneously and influence each other. Identification of the relationships among the interdependent system elements or components is crucially needed for system definition and CM development.
6. Organization: System complexity can be overcome by way of decomposing the system into subsystems and subsystems into other subsystems. This decomposition can be carried out by examining how system elements or components are organized. The CM can then be decomposed into submodels, and submodels into other submodels to mimic the system decomposition. Such modularization is crucial for overcoming the complexity of the system, CM, and the simulations to be designed.

In this life-cycle process, we also define the IUs for which simulation models may be developed. The IUs define the point of reference with respect to which a simulation model is developed and used. Model acceptability assessment is carried out with respect to the prescribed IUs of the model. Therefore, proper definition of the IUs is considered to be extremely important. Balci and Ormsby (2000) provide guidelines for properly defining the IUs.

Many categories of IUs for a simulation model exist including:

• Analysis (eg given input and system definition, identify output).
• Synthesis or design identification (eg given input and output, identify system design).
• Instrumentation or control (eg given system definition and output, identify input).
• Comparison (eg comparing competitive system designs to carry out a specified function; comparing several proposed operating policies or procedures).
• Evaluation (eg determining how well a proposed system design performs when evaluated against specific criteria; to gain insight about the way the system operates).
• Sensitivity analysis (eg determining which of many factors are the most significant in affecting overall system behaviour; establishing the nature of the relationship among one or more significant factors and the system's behaviour).
• Prediction (eg predicting the earliest and latest times to intercept a threat missile; given threat missile initial conditions, predicting the threat missile trajectory; forecasting the behaviour of a system under some projected set of conditions).
• Ranking and selection (eg ranking a number of alternatives and selecting the best one).
• Optimization (eg determining exactly which combination of factor levels will produce the optimal overall behaviour of the system).

The system and objectives definition V&V deals with the integrative evaluation of (a) the quality of the system and objectives definition as a work product, (b) the system investigation process used in creating the system and objectives definition, and (c) the project management characteristics related to this life-cycle stage.

3.3. High-level design

This life-cycle process is executed to take the system and objectives definition as input and produce the work product High-Level CM Design as output. High-Level CM Design should contain high-level conceptualizations that are very close to the level of thinking of a simulation model designer. High-Level CM Design can be created based on:

1. concept of operations (ConOps),
2. conceptual models of the mission space (CMMS)/functional description of the mission space (FDMS),
3. object-oriented analysis (OOA),
4. knowledge engineering (KE), and
5. operational and system views.

IEEE (1998) defines ConOps as a user-oriented document that describes a system's operational characteristics from the viewpoint of an end user. It states that 'The ConOps document is used to communicate overall quantitative and qualitative system characteristics to the user, buyer, developer, and other organizational elements (eg training, facilities, staffing, and maintenance). It describes the user organization(s), mission(s), and organizational objectives from an integrated systems point of view'. A ConOps document can be very useful for creating a High-Level CM Design.

The US DoD Defense Modelling and Simulation Office (DMSO) efforts on developing CMMS and FDMS focus on creating simulation-independent high-level descriptions of military operations mission space, real-world processes, entities, and military, physical, and civil environments (DMSO, 2007). Both CMMS and FDMS aim to provide a repository of information and representations for assisting simulation developers. Waite (1997) presents object-oriented representation schemas for CMMS. CMMS and FDMS can be used for developing a High-Level CM Design.

Object-oriented analysis aims to develop representations of the problem domain of interest by employing use cases, actors, classes, objects instantiated from classes, collaborations among objects, object characterization using attributes, and object behaviour description using methods. Object-oriented analysis is commonly conducted under the Unified Process (Jacobson et al, 1999) and using the UML, which is an international standard maintained by OMG (OMG, 2007). Object-oriented analysis may include the following activities (Schach, 2007): analysis workflow of the Unified Process; identification of actors and use cases; extracting boundary, control, and entity classes; functional modelling; entity class modelling; dynamic modelling; and class–responsibility–collaboration analysis. Object-oriented analysis can provide the underpinnings of a High-Level CM Design.

KE can significantly assist in creating a High-Level CM Design. KE is related to mathematical logic and involved in cognitive science where the knowledge is produced by mainly humans and is structured according to our understanding of how human reasoning and logic works (Wikipedia, 2007). KE may be conducted by executing the following iterative processes: (a) knowledge elicitation and collection using problem domain SMEs, (b) knowledge organization and data mining, (c) structuring knowledge and representing it in the form of knowledge bases, and (d) V&V of the inserted knowledge and data. The artefacts (work products) produced by the KE processes can be very useful in constructing a High-Level CM Design.

The DoD Architecture Framework (DoDAF) (DoDAF, 2004a, 2004b, 2004c) defines a common approach for describing enterprise-wide and network-centric system of systems architectures using 26 different representation formats (products) including Operational Views and System Views. The Operational View contains graphical and textual products to describe the tasks and activities, operational nodes and elements, and information exchanges required to accomplish prescribed missions including both warfighting missions and business processes. The System View consists of a set of graphical and textual products to describe systems, system of systems, and interconnections providing for, or supporting, DoD functions including both warfighting and business functions. The DoDAF products, especially the Operational Views and System Views, can be used for creating a High-Level CM Design.

The High-Level CM Design V&V deals with the integrative evaluation of (a) the quality of the High-Level CM Design as a work product, (b) the high-level design process used in creating the High-Level CM Design, and (c) the project management characteristics related to this life-cycle stage.

3.4. Low-level design

We consider large-scale complex CMs for problem domains of a COI such as the one depicted in Figure 4. Designs of such complex CMs demand modularization to overcome the complexity. Therefore, it is expected that the CM be modularized into a hierarchy of submodels. The CM is decomposed into submodels at level 1. A submodel at level 1 is further decomposed into other submodels at level 2. Then, a submodel at level 2 is further decomposed into submodels at level 3. The decomposition continues until the leaf submodels, the ones not further decomposed, are manageable in complexity.

This decomposition must be carried out in such a manner that each leaf submodel possesses the highest possible cohesion and the lowest possible coupling. Submodel cohesion refers to the degree of interaction among the elements of a submodel. Submodel coupling refers to the degree of dependency between two submodels.

This life-cycle process is executed to take the High-Level CM Design as input and produce the work product Detailed Conceptual Submodel Design as output. Detailed Conceptual Submodel Design refers to the entire hierarchy of submodels and detailed design of each submodel.

Detailed Conceptual Submodel Design can be represented using the UML diagrams, namely, activity diagram, class diagram, collaboration diagram, component diagram, deployment diagram, object diagram, sequence diagram, state diagram, and use case diagram.

Architectural styles, design patterns, and conceptual frameworks can be employed for creating a Detailed Conceptual Submodel Design. Architectural styles include client-server, distributed objects, and peer-to-peer. A design pattern is a generally accepted repeatable solution to a commonly encountered design problem. Design patterns can provide the underpinnings of a Detailed Conceptual Submodel Design. Example design patterns include model-view-controller, publish-and-subscribe, and abstract factory. Derrick and Balci (1997) define conceptual framework as 'an underlying structure and organization of ideas which constitute the outline and basic frame that guide a modeler in representing a system in the form of a model' and present a multifaceted conceptual framework for visual simulation modelling.

A Detailed Conceptual Submodel Design may contain algorithms and data commonly needed for simulation modelling of the problem domain of interest. Algorithms should be represented in a programming-language independent manner so that they can be reused for different simulation model designs. The data should be given with the underlying assumptions.

Submodel Design V&V deals with the integrative evaluation of (a) the quality of the Detailed Conceptual Submodel Design as a work product, (b) the low-level design process used in creating the Detailed Conceptual Submodel Design, and (c) the project management characteristics related to this life-cycle stage.

3.5. Integration

All submodels created separately are integrated to form the entire CM in this life-cycle process producing the work product Integrated CM as output. The integration is typically accomplished by using hyperlinks.

Integrated CM V&V deal with the integrative evaluation of (a) the quality of the Integrated CM as a work product, (b) the integration process used in creating the Integrated CM, and (c) the project management characteristics related to this life-cycle stage.

3.6. Specification

This life-cycle process is executed to take the Integrated CM as input and produce the work product CM Specification as output. The specification process transforms the Integrated CM into a multimedia representation in a variety of communicative forms such as: animation, audio, chart, diagram (eg UML diagrams), drawing, equation, graph, image, text, and video. The multimedia communicative forms should be used for the purpose of accomplishing some of the objectives stated in section 2 for creating a CM.

CM Specification V&V deals with the integrative evaluation of (a) the quality of the CM Specification as a work product, (b) the specification process used in creating the CM Specification, and (c) the project management characteristics related to this life-cycle stage.

After completion of the CM Specification, the CM and the data used must go through rigorous V&V. CM V&V is considered as part of the overall product evaluation. In addition to V&V, some quality characteristics such as maintainability and reusability should also be assessed.

CM verification deals with substantiating that the CM is transformed from one form into another with sufficient accuracy throughout the life cycle. It deals with the CM transformational accuracy assessment. CM validation deals with substantiating that the CM possesses sufficient accuracy in representing the problem domain and the system that contains the problem as defined under the prescribed objectives. It deals with the CM representational and behavioural accuracy assessment.

CM maintainability is the degree to which the CM facilitates changes. CM reusability is the degree to which the CM facilitates its reuse for designing many simulation models.

Data verification deals with substantiating that the data are transformed from one representation into another with sufficient accuracy. Data verification addresses the question of 'Are we acquiring the data right?' Data validation deals with substantiating that the data representation has sufficient accuracy. Data validation addresses the question of 'Are we acquiring the right data?'

3.7. Use

The Use process refers to the life-cycle stage during which the CM is employed for designing many simulation models. Feedback is solicited from the CM user community during this stage and is documented. The feedback is quite valuable for the purpose of improving the CM representation as well as its reusability.

3.8. Redefinition

The redefinition process is intended to represent the maintenance stage of the life cycle and involves four types of maintenance:

• Adaptive maintenance: Adaptations required as the CM's external environment evolves.
• Corrective maintenance: Fixing deficiencies, resolving caveats, and making corrections.
• Perfective maintenance: Enhancements brought about by changing user requirements.
• Preventive maintenance or reengineering: Making changes for the purpose of preventing potential problems or for reengineering.

The redefinition process also represents the redefinition of the system and objectives for the purpose of developing a new CM by way of changing an existing one.

Like every successful product, a successful simulation CM is expected to evolve during its life time. A successful CM becomes an asset for the COI and return on investment is realized by maintaining and reusing the CM.

4. The life-cycle use

The life cycle we created and used for the BEST project provided the much-needed blueprint of development and assisted in modularizing the tasks for different development teams. It provided technical as well as managerial guidance in the development process.

Technically, people involved in the BEST project commented that the life cycle provides structure and discipline for the CM development. The life cycle lays out what work products must be produced in what sequence by executing which life-cycle processes. This enables the technical people to modularize the tasks and be able to overcome the complexity of development. The life cycle also enunciates the fact that accuracy assessment by V&V must be conducted hand in hand with the development life cycle.

Managerially, the life cycle enables the decomposition of the project management into phases throughout the life cycle. Allocation of people to different tasks, project scheduling, and configuration management are all facilitated by the structure imposed upon by the life cycle.

The simulation conceptual modelling life cycle is applicable for large-scale as well as for small-scale developments. It can be used for any type of development and it is not specific to military applications. Our experience shows that the life cycle imposes a structure and a disciplined approach for the construction of any type of simulation conceptual model, whether it is small or large.

5. Conclusions

A simulation CM represents the highest layer of abstraction that is closer to the level of thinking of managers, analysts, and simulation model designers. It is specified in a variety of multimedia communicative forms and reused for designing many simulation models and for many types of simulation within a problem domain of a COI. The CM plays a critical role in designing large-scale complex simulation models and alleviates many commonly encountered problems if created properly. Fifteen objectives are enunciated for creating a CM. A life cycle is presented to modularize and structure the CM development. The life cycle dictates which processes to execute, what work products to produce, and what V&V activities to perform hand in hand with the development.

References

1. Balci O (1988). The implementation of four conceptual frameworks for simulation modelling in high-level languages. In: Abrams MA, Haigh PL and Comfort JC (eds). Proceedings of the 1988 Winter Simulation Conference. IEEE: Piscataway, NJ, pp 287–295.
2. Balci O (1998). Verification, validation, and testing. In: Banks J (ed). The Handbook of Simulation. John Wiley & Sons: New York, NY, August, pp 335–393.
3. Balci O (2001). A methodology for certification of modelling and simulation applications. ACM Trans Model Comput Simulat (TOMACS) 11(4): 352–377.
4. Balci O (2003). Verification, validation, and certification of modelling and simulation applications. In: Chick S, Sánchez PJ, Ferrin D and Morrice DJ (eds). Proceedings of the 2003 Winter Simulation Conference. IEEE: Piscataway, NJ, pp 150–158.
5. Balci O and Nance RE (1985). Formulated problem verification as an explicit requirement of model credibility. Simulation 45(2): 76–86. | Article | ISI |
6. Balci O and Ormsby WF (2000). Well-defined intended uses: An explicit requirement for accreditation of modelling and simulation applications. In: Joines JA, Barton RR, Kang K and Fishwick PA (eds). Proceedings of the 2000 Winter Simulation Conference. IEEE: Piscataway, NJ, pp 849–854.
7. Derrick EJ and Balci O (1997). DOMINO: A multifaceted conceptual framework for visual simulation modelling. INFOR—Canadian J Opl Res Inform Process 35(2): 93–120.
8. DMSO (2006). DoD Verification, Validation, and Accreditation Recommended Practices Guide, Build 3.0 (September) Defense Modelling and Simulation Office (DMSO): Alexandria, VA, http://vva.dmso.mil/.
9. DMSO (2007). Defense Modelling and Simulation Office. Alexandria, VA, https://www.dmso.mil/.
10. DoDAF (2004a). DoD Architecture Framework Version 1.0 Volume I: Definitions and Guidelines. Architecture Framework Working Group, Department of Defense: Washington, DC, February.
11. DoDAF (2004b). DoD Architecture Framework Version 1.0 Volume II: Product Descriptions. Architecture Framework Working Group, Department of Defense: Washington, DC, 9 February.
12. DoDAF (2004c). DoD Architecture Framework Version 1.0 Deskbook. Architecture Framework Working Group, Department of Defense: Washington, DC, 9 February.
13. Elmaghraby SE (1968). The role of modelling in IE design. J Indl Eng 19(6): 292–305.
14. IEEE (1998). IEEE Guide for Information Technology—System Definition—Concept of Operations (ConOps) Document IEEE Standard 1362, IEEE: New York, NY.
15. IEEE (2000). IEEE Standard for Modelling and Simulation (M&S) High Level Architecture (HLA) IEEE Standard 1516, 1516-1, 1516-2, and 1516-3, IEEE: New York, NY.
16. Jacobson I, Booch G and Rumbaugh J (1999). The Unified Software Development Process. Addison-Wesley: Reading, MA.
17. NRL (2004). Battlespace Environment and Signatures Toolkit (BEST). Naval Research Laboratory: Washington, DC.
18. OMG (2007). Unified modelling language (UML). The Object Management Group (OMG), http://www.omg.org/.
19. Orca (2003). A web-based collaborative evaluation environment. Orca Computer, Inc.: Blacksburg, VA, http://www.orcacomputer.com/ee/.
20. Robinson S (2004). Simulation: The Practice of Model Development and Use. John Wiley & Sons Ltd: West Sussex, UK.
21. Robinson S (2006). Issues in conceptual modelling for simulation: setting a research agenda. In: Garnett J, Brailsford S, Robinson S and Taylor S (eds). Proceedings of the 2006 Operational Research Society Simulation Workshop. Operational Research Society: Birmingham, UK, pp 165–174.
22. Schach SR (2007). Object-Oriented and Classical Software Engineering 7th edn., McGraw Hill, New York, NY.
23. Shannon RE (1975). Systems Simulation: The Art and Science. Prentice-Hall: Englewood Cliff, NJ.
24. Waite WF (1997). Object-oriented representation schemas for conceptual models of the mission space (CMMS). Proceedings of the 1997 Spring Simulation Interoperability Workshop, Orlando, FL, 3–7 March.
25. Wikipedia (2007). Knowledge engineering. Wikipedia Encyclopedia, Wikimedia Foundation, Inc.: St Petersburg, FL, http://en.wikipedia.org/wiki/Knowledge_engineering.
26. Yilmaz L and Ören TI (2004). A conceptual model for reusable simulations within a model–simulator–context framework. In: Proceedings of the Conference on Conceptual Modelling and Simulation, Part of the I3M—International Mediterranean Modelling Multiconference, Genoa, Italy, 28–31 October, pp 235–241.