multiprocessing RunitimeError for Windows #118
Merged
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
PR concerning errors like ("NameError: name 'np' is not defined") when using multiprocess exec mode on windows:
Linked to #104 & #93
Is it convenient to add the following "workaround code" to \cadCAD_init_.py file to fixe the issue?
or should we only add the code to \cadCAD\engine_init_.py] file to fixe the issue?
to limit the setting to multiprocess exec mode and reduce potential side effects. Indeed, although this seems to be a stable workaround for windows, there are some comments (see [https://github.com/uqfoundation/dill/issues/115]) that are mitigated about the solution.
There are 2 other less convincing alternatives for our issue:
1 - To declare the missing global variables (which are not correctly serialized without dill.settings['recurse'] = True) at the function level (see [https://github.com/uqfoundation/multiprocess/issues/6]) => could be less practical, since this requires to re-declare global variables locally...
2 - to use cloudpickle instead of dill (see [https://github.com/apache/airflow/issues/7870]), => no idea on the impact of the migration
Regards,