Experimenter replications repeat exactly after "Reset Experiment"

[nmzenke]

[ FlexSim 16.0.1 ]

I ran the experimenter for 10 replications of a scenario and then looked at the reported performance data. One of the ten replications reported a value for a performance measure that was well outside of an expected value. I decided to rerun the experimenter to see if the behavior showed up again when running 10 more replications. Well it did show up again but it was in the exact same replication. Repeating this one more time gave me the same result again. When we "Reset Experiment" and then re-run, is the experimenter supposed to produce the exact same replication results as last time if we haven't changed anything? Is there a way to change this so it doesn't repeat and instead generates new data?

[Matthew_Gillespie · Accepted solution]

This is by design. Your model should run the same way every time if you don't change anything. This makes it possible to debug your model when it's not doing what you want it to. This extends to the experimenter too. Each replication should run exactly the same each time you run the experimenter. This way you can choose a specific replication and run it in your model (using the same scenario variables and random stream numbers) when you need to debug or you want to see how a replication produced a certain result.

If you need more data you can increase the number of replications.

There is a way to stop the model repeating random streams (Statistics > Repeat Random Streams), but in general we don't recommend this as it makes it nearly impossible to debug models.

[nmzenke]

Thanks for clarifying!

[tee_hiett]

Matthew, Is there a way to eliminate the Repeat Random Streams option for Experiments? I have unchecked the Repeat Random Streams in the model under the Debug tab but that does not seem to work with the Experiment.

Thanks, Tee

[Matthew_Gillespie]

@Cliff King

[AD11]

@Matthew Gillespie , @Cliff King , Was this question about preventing repeat streams for an experiment answered? Running into a similar issue.

Thank you

[jason_lightfootVL7B4]

What are you trying to do? If you want to test a different set of generated numbers run an experiment (each replication runs different streams) or set the interactive run of the model to use the streams from a given replication (set that in the experimenter). Also, if you want to compare two scenarios then having the replications repeat their own set of streams is best practice as a variance reduction technique.

If you turn off repeat streams and encounter an error or interesting result, you will never be able to repeat that situation again - from an audit point of view this should be avoided.