Carrying Out Nested Monte Carlo Simulation Using a SubModel

Many models have uncertain (epistemic) parameters as well as randomly variable (aleatory) parameters. An uncertain parameter represents ignorance that can theoretically (but perhaps not practically) be reduced through investigation (e.g., the mean failure time for a batch of light bulbs). Variability is inherent in many systems (e.g., the distribution of failure times for a batch of light bulbs) and cannot be reduced. It is often valuable to explicitly separate variability from uncertainty in a model.

With SubModels, this is accomplished by inserting a SubModel (e.g., a dynamic Monte Carlo simulation) within an outer model (e.g., a static Monte Carlo simulation). This is referred to as "nested" Monte Carlo simulation. The inner model simulates the random variability of the system, while the outer model represents the ignorance in key parameters.

In the example above, the outer model would sample a probability distribution that represents the uncertainty in the mean lifetime of a light bulb, and the inner model would simulate the performance of a number of random light bulbs (whose lifetime is sampled from a distribution with the mean specified by the outer model).

Note that for this kind of model, if you were to link an output from the SubModel to the Output Interface as a distribution, conceptually on the outside of the SubModel the output would actually represent a distribution of distributions. That is, because the SubModel is a Monte Carlo model, every time it is called, it produces a distribution. However, because the outer model itself is a Monte Carlo simulation, every element inside it results in a distribution of results. Hence, running the outer model multiple times results in a distribution of distributions for any SubModel output.

If you were to go inside the SubModel and view a result, it would display only the results from the last time the SubModel was run (and only if Save Results for most recent Simulation was checked).

However, on the outside of the SubModel, for a nested Monte Carlo run, it is necessary to display the distribution of distributions. Of course, GoldSim’s standard Distribution Result only displays a single distribution. (It can display distributions from multiple outputs, but this is not the same as displaying nested distributions).

To support nested Monte Carlo simulations, Distribution Result elements modify their behavior under certain circumstances to enable display of nested Monte Carlo results. In particular, Distribution Result elements modify their behavior if and only if the following conditions are all met:

• The Distribution Result element is located in the outer model (i.e., not in a SubModel);

• The selected result references a distribution type output from a Monte Carlo SubModel (for this to occur, the SubModel must be configured to run multiple realizations);

• The SubModel whose result is being referenced is located within the main model (i.e., it cannot be nested inside another SubModel); and

Under these circumstances, there are two differences from a standard Distribution display:

All Realizations. “Realizations” refers to realizations of the outer model. Since each realization of the outer model produces a distribution, showing “All Realizations” simply displays multiple distributions (one for each realization of the outer model). The chart display looks like this:

Statistics can only be shown for one realization at a time (which you can select from a drop list at the top of the dialog). In the example above, statistics are being shown for realization #24 of the outer model.

Realization. “Realization” refers to the realization of the outer model (which you must select). The chart (and Distribution Summary) then show that single outer model realization (i.e., a single distribution):

Statistic. When you select this option, a drop-list to the immediate right allows you to select or define a statistic. The result displayed is then the distribution of the specified statistic (each outer model realization produces one value for this statistic):

Probability. This option displays a probability distribution of probabilities. When you select this option, a drop-list to the immediate right allows you to define the probability that you are interested in, based on a condition:

This instructs GoldSim to compute the (inner model) probability of a particular outcome for the specified result. Each outer model realization produces one value for this probability. It is the probability of the inner model’s result being <, <=, =, >= or > the specified value. In the example above, for each outer model realization, GoldSim computes a single value: the probability of the output being less than 9000 hours. The overall result then displays a distribution that quantifies the uncertainty in this particular probability:

How do we interpret such a distribution? This is meant to represent a complex concept (a probability distribution of a probability!). Perhaps this is best understood by examining this particular example. In this example, we could state the following:

• There is approximately a 70% chance that the probability of the result being less than 9000 hours will be less than one. That is, in 70% of the outer model realizations, the probability of the result being less than 9000 hours was less than one, and in 30% of the realizations it was exactly one.

• There is no chance that the probability of the result being less than 9000 hours is below (approximately) 48%. That is, in none of the outer model realizations was the probability of the result being less than 9000 hours less than (approximately) 48%.

All Results. This combines all realizations together to form a single distribution that represents both variability (from the inner SubModel) and uncertainty (from the outer model). Hence, if you run 50 realizations of the SubModel and 100 realizations of the outer model, the distribution would be constructed from 5000 realizations.The chart (and Distribution Summary) then show the single distribution that represents all model uncertainties:

The text above describes how distribution results can be viewed for a nested Monte Carlo simulation. But what if you wanted to view time history results? Viewing time history results for nested Monte Carlo simulations is complicated by the fact that Time History Definition SubModel outputs consist of multiple time histories for each realization of the parent model. As a result, if the parent model was run for 100 realizations, and the SubModel was run for 50 realizations, the corresponding time history results would be based on 5000 individual time histories (50 * 100).

To support nested Monte Carlo simulations, Time History Result elements modify their behavior under certain circumstances to enable display of nested Monte Carlo results. In particular, Time History Result elements modify their behavior to display such results if and only if the following conditions are all met:

• The Time History Result element is located in the outer (parent) model (i.e., not in a SubModel);

• Within the Result Properties dialog, “SubModel Time” is selected for the Time Display Setting.

• A single result is selected for display in the Result element. The selected result must reference a Time History Definition output from a SubModel. The Output Definition for the output must be “Realization Histories” (for this to occur, the SubModel must be configured to run multiple realizations);

• The SubModel whose result is being referenced is located within the main model (i.e., it cannot be nested inside another SubModel); and

Under these circumstances, special Display options are provided at the top of the Time History display dialogs:

Histories for Realization. “Realization” refers to the realization of the parent model (which you must select). Since each realization of the parent model produces multiple histories for the SubModel, showing “Histories for Realization” simply displays multiple SubModel histories for the selected parent model realization. The chart display looks like this:

Note that the display also provides a separate field (Show). In the figure above, the SubModel histories for the specified parent realization (in this case, #8) are being displayed “Individually”. The tool-tip indicates that the particular result indicated by the cursor corresponds to SubModel history #5.

A second option allows these 5 histories to be displayed “As Probabilities” (i.e., percentile ranges):

Statistic for each Realization. “Realization” refers to the realization of the parent model. Since each realization of the parent model produces a single statistic (which you must specify) for the SubModel histories, showing “Statistic for each Realization” displays a single time history for each realization of the parent model. Each individual realization represents a statistic computed using the SubModel histories.The chart display looks like this:

In the figure above, there are 10 realizations shown (one for each parent realization). Each realization represents the 50^th percentile computed using the 5 SubModel histories that were produced for that parent realization. In this case, each parent realization is being displayed “Individually”. Note that the display also provides a separate field (Show). A second option allows these 10 parent realizations to be displayed “As Probabilities”:

Histories for all Realizations. Since each realization of the parent model produces multiple histories for the SubModel, showing “Histories for all Realizations” simply displays all SubModel histories for all parent realizations. The chart display looks like this:

In the figure above, there are 50 histories shown (5 SubModel histories for each of 10 realizations of the parent model). In this case, each history is being displayed “Individually”. Note that the display also provides a separate field (Show). A second option allows these 50 histories to be displayed “As Probabilities”:

Note: If you have applied screening to the inner model (the SubModel), the screening is ignored for nested Monte Carlo runs. However, if you have applied screening and/or categories to the parent model, the screening is used to create the displays.

Note: You cannot export time history results (i.e., to a spreadsheet or text file) from a SubModel.

An example illustrating how nested Monte Carlo simulation is carried out (SubModel2.gsm) can be found in the General Examples/SubModel folder in your GoldSim directory (accessed by selecting File | Open Example... from the main menu).