Enable Restarting RMG Jobs from a Seed Mechanism

[amarkpayne]

Motivation or Problem

There are many instances where it is desirable to restart an RMG job from the last iteration instead of having to start the mechanism generation from the beginning. RMG currently has a restart file, but it has not been maintained and will be deprecated starting in the next release. Furthermore, these restart files are extremely slow.

The agreed upon solution to this problem is to restart RMG jobs from the seed mechanism files that RMG already outputs. These files allow for the complete reconstruction of the model core and edge at the time of the restart. The only missing piece of information missing is the filter tensors, which help RMG decide which species to react, which species have already reacted, and which species not to react because their expected rate is well below what would be included based on the tolerance. This PR saves these filter tensors and allows for them to be loaded so that RMG jobs can be restarted using this technique

Description of Changes

• Reorganized when seed mechanisms are saved
• Save the filter tensors (and necessary mappings) to a Filters sub folder of the seed mechanism
• Add in a block to the RMG input file format for telling RMG to perform a restart job from a seed mechanism
• Automatically generate input files entitled restart_from_seed.py that can be run to restart the job
• Updated the documentation to describe how the user can use this restart feature
• Saving the seed mechanism each iteration is now the default behavior

Testing

I have restarted the superminimal and minimal examples using this technique, and the same mechanism was obtained if the job was run to completion or if the job was restarted from a seed mechanism from an intermediate iteration seed mechanism.

I have not added anything in the way of unit testing yet, but I am happy to add tests for the crucial features of this PR.

[codecov]

Codecov Report

Merging #1641 into master will decrease coverage by 0.13%.
The diff coverage is 17.82%.

@@            Coverage Diff             @@
##           master    #1641      +/-   ##
==========================================
- Coverage   41.79%   41.66%   -0.14%     
==========================================
  Files         177      176       -1     
  Lines       30207    30215       +8     
  Branches     6252     6256       +4     
==========================================
- Hits        12625    12588      -37     
- Misses      16659    16703      +44     
- Partials      923      924       +1

Impacted Files	Coverage Δ
rmgpy/solver/files/listener/input.py	100% <ø> (ø)	⬆️
rmgpy/solver/files/liquid_phase_constSPC/input.py	100% <ø> (ø)	⬆️
rmgpy/tools/data/sim/simple/input.py	100% <ø> (ø)	⬆️
rmgpy/tools/data/sim/mbSampled/input.py	100% <ø> (ø)	⬆️
rmgpy/tools/data/sim/liquid/input.py	100% <ø> (ø)	⬆️
rmgpy/rmg/model.py	41.07% <0%> (-0.63%)	⬇️
rmgpy/stats.py	59.81% <0%> (+0.33%)	⬆️
rmgpy/rmg/main.py	21.85% <15.78%> (-1.23%)	⬇️
rmgpy/rmg/input.py	36.58% <7.14%> (-1.36%)	⬇️
rmgpy/data/kinetics/library.py	43.13% <78.57%> (+0.03%)	⬆️
... and 2 more

---

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 04d4d42...3a00ee3. Read the comment docs.

[amarkpayne]

Thanks for the helpful comments @mliu49 ! There are a couple of things I need to change depending on what we want to do with the old style restart

[mjohnson541]

In general I think this looks like it avoids a lot of the bugs that could easily occur with this feature. I've brought up a few issues I noticed below.

[mliu49]

Are the changes to seed paths done as well?

[mjohnson541]

Couple small additional things.

[amarkpayne]

I have force pushed the requested changes and confirmed that the (updated, the species map changed from a dictionary to a list) minimal restart example works.

[mliu49]

I think we should use the strict=False argument since we don't generate resonance structures for the restart species.

[amarkpayne]

Done

[mliu49]

I think this doesn't do anything. It should set the coreSeedPath, edgeSeedPath, filtersPath, and speciesMapPath like you do when loading the input file.

Alternatively, if you don't think we should support both methods, you can remove this and set the -r flag to simply print a warning that the old restart method has been removed and the flag does not do anything.

[amarkpayne]

Fixed, thanks for the catch!

[mliu49]

@amarkpayne was correct, and the problem was in fact the extra indentation, which was messing up the family name parsing and causing the reaction to be loaded as a library reaction with .library (and .family) as seedMechanism.name (here). We missed the warning, which would have led us to the cause much faster.

I added a few commits to resolve this and some related issues:

• Switch to using the longDesc property of the entries instead of the private _longDesc attribute in getLibraryReactions. Not a bug, but the point of a private attribute is that it shouldn't be used.
• Improve parsing of reaction template and family name in getLibraryReactions, including stripping the family name. I discovered while debugging that the reaction template was completely wrong in some cases because the comment strings weren't exactly as expected.
• Remove leading spaces from kinetics comments in getLibraryReactions. I wasn't sure the best place to put this, but it seemed to make sense there. This prevents the gradual increase in indentation with each read/write cycle.

[amarkpayne]

I have also confirmed that with @mliu49 fixes enable unlimited successive restarts. I have also addressed the last remaining comment that I need to fixup, and I have squashed all of the fixups and force pushed. This should be ready for final approval/merging

[mliu49]

I think this is good to go now. @mjohnson541, do you have any comments on the fixes that I added?

[mjohnson541]

Took a look now, looks fine to me!

Outdated