Random <0.1% variation in GCAM outputs

[harounchahed]

Hi,

I have run the same GCAM scenario (with the same config files and input data) 3 times in a row and the results were slightly different at each run. The results are different by ~0.0007% at each run, so it's an insignificant change and is likely due to a random seed setting in GCAM. However, I am curious to know what is this variation due to and am wondering if it can be avoided (e.g., by setting a value for the random seed if that exists).

Thanks in advance for your help!

[pralitp]

Hello Haroun,

Right. Since GCAM 5.4, by default, running the same scenario repeatedly is only guaranteed to reproduce results within solution tolerance. However, this is not because GCAM is explicitly using any randomization. Instead it is because since version 5.4, by default, GCAM utilizes multiple CPU/cores to perform computations. And due to the inexact way numeric values are represented in a computer the introduction of these parallel computations causes non-deterministic behavior.

The error from these non-deterministic operations is small, on the order of 1e-9 difference for market supplies and demands which need to be solved. However, some markets in GCAM are also modeled in a range of values as low as 1e-6. And certainly a lot of decisions with in the solution algorithm operate on small value thresholds. Thus the non-deterministic operations could, and often do, lead to very different paths through the solver. Thus the results are only guaranteed to be reproducible within solution tolerance.

If exact reproducibility is important for your work then you should re-compile GCAM with parallelism turned off as detailed in the build instructions documentation.

[harounchahed]

This is very helpful - thanks a lot Pratel!