Appending real time stats

[ijpulidos]

Description

Changing the default behavior of the multistate reporter to append in the YAML real time statistics file instead of overwriting.

We already had the machinery for this, we probably want to discuss if we want to expose this option to the user.

I tested this resuming a run, but we probably want to write a proper test for this behavior. WIP.

Resolves #691

Todos

• Implement feature / fix bug
• Add tests
• Update documentation as needed
• Update changelog to summarize changes in behavior, enhancements, and bugfixes implemented in this PR

Status

• Ready to go

Changelog message

Real time statistics in YAML file are now appended when resuming simulations, instead of being overwritten.

[codecov]

Codecov Report

Merging #692 (e862a22) into main (abb2f61) will decrease coverage by 0.26%.
The diff coverage is 66.66%.

[ijpulidos]

A limitation that we have with this approach is that we would need the interval for writing the real time yaml files to be a multiple of the checkpoint interval, or else we would have duplicated iterations information when we resume simulations (think when we stop a simulation in the somewhere in middle of a checkpoint interval and we resume it, the real time yaml file will have duplicated iterations).

I guess we could add a check for this in the constructor of the MultiStateSampler object to avoid this issue.

[ijpulidos]

Looking at the code probably the better API point where we should be creating this is in MultiStateSampler.create, rather than in the constructor. Since this is the place where the storage reporter is created as well, which is the one that handles the checkpointing.

[ijpulidos]

This breaks quite a few tests that rely on not making any checkpoints but doing the online analysis. I wonder what's the best approach here.

[mikemhenry]

I think we only need the check if we are writing checkpoints, right?

[ijpulidos]

I think we only need the check if we are writing checkpoints, right?

Not sure exactly what you mean here, but we don't have a way to have no checkpointing at all, as far as I can tell. Tests just make the checkpoint_interval very large to avoid creating checkpoints.

[ijpulidos]

Not sure exactly what you mean here, but we don't have a way to have no checkpointing at all, as far as I can tell. Tests just make the checkpoint_interval very large to avoid creating checkpoints.

We maybe want to have this capability in the future, maybe passing checkpoint_interval=None for the reporter to not generate any checkpoints at all.