Setting the default value for --ckpt_path in Lightning CLI

[JRF63]

ckpt_path seems to be parsed differently that the rest of the arguments. This doesn't work:

class CustomCLI(LightningCLI):
    def add_arguments_to_parser(self, parser: LightningArgumentParser) -> None:
        parser.set_defaults(
            {
                "ckpt_path", "last"
            }
        )

and would return this error:

  File "C:\Users\User\.micromamba\envs\pytorch\Lib\site-packages\jsonargparse\_core.py", line 887, in set_defaults
    raise NSKeyError(f'No action for key "{dest}" to set its default.')
jsonargparse._namespace.NSKeyError: No action for key "ckpt_path" to set its default.

[zhangzw16]

Hi, any idea what the problem and how to solve it?

[mauvilsa]

This doesn't work because add_arguments_to_parser is called here cli.py#L454. However, ckpt_path is added to the parser here cli.py#L505, which happens after and it is not allowed to add defaults for arguments that don't exist.

If we change the location of where add_arguments_to_parser gets called, a bunch of unit tests start failing. Which likely means other features depend on this and can't be changed. Possible improvements that could allow this could be:

• Feature request in https://github.com/omni-us/jsonargparse/issues asking to allow setting defaults without pre-existing argument. Apart from ckpt_path here, there can be other reasons why this could be desired.
• Add to LightningCLI a new method add_defaults_to_parser which would be called after cli.py#L505.

If LightningCLI is modified for this, might be good beforehand to analyze other not-yet-supported cases. This is to avoid a new feature making it even harder to add these other cases.

[zhangzw16]

Thanks for your quick and comprehensive reply! I recently encountered a similar error:

File "..../jsonargparse/_namespcae.py", line 153, in _parse_required_key
    raise NSKeyError(f'Key "{key}" not found in namespace.')
jsonargparse._namespace.NSKeyError: Key "data.data_manager" not found in namespace.

I eventually found out that this was because the variable self.scaler was missing in data.data_manager, but I had written this variable in link_arguments (see code below), which caused the problem. I feel that this could be avoided with a pre-check rather than throwing an error in jsonargparse, making it easier for us to spot the code errors.

class ProbTSCli(LightningCLI):
    
    def add_arguments_to_parser(self, parser):
        data_to_model_link_args = [
            "scaler"
        ]
        data_to_forecaster_link_args = [
            "target_dim",
            "history_length",
            "context_length",
            "prediction_length",
            "lags_list",
            "freq",
            "time_feat_dim",
            "global_mean",
            "dataset"
        ]
        for arg in data_to_model_link_args:
            parser.link_arguments(f"data.data_manager.{arg}", f"model.{arg}", apply_on="instantiate")
        for arg in data_to_forecaster_link_args:
            parser.link_arguments(f"data.data_manager.{arg}", f"model.forecaster.init_args.{arg}", apply_on="instantiate")