couldn't open import "*.libsonnet": no match locally or in the Jsonnet library paths.

[tshu-w]

Issue

Due to the extensive functionality of jsonnet (as mentioned in #117 (comment)), I tried switching the default configuration to jsonnet format, but I ran into problems with import.

Step to reproduce

Here is the minimal config to reproduce:
my directory structure:

.
├── conf
│   ├── name.libsonnet
│   └── test.jsonnet
└── test.py

test.py:

from jsonargparse import CLI

def command(
    name: str = "Lucky",
    prize: int = 100
):
    """
    Args:
        name: Name of winner.
        prize: Amount won.
    """
    print(f'{name} won {prize}€!')

if __name__ == '__main__':
    CLI(parser_mode="jsonnet")

conf/name.libsonnet:

"Mike"

conf/test.jsonnet:

local name = import 'name.libsonnet';

{
  "name": name,
  "prize": 100,
}

If I run python test.py --config conf/test.jsonnet, I got the following error:

Click to expand

Traceback (most recent call last):
  File "/home1/wts/lightning-template/debug/test.py", line 15, in <module>
    CLI(parser_mode="jsonnet")
  File "/home1/wts/.local/share/conda/envs/template/lib/python3.9/site-packages/jsonargparse/cli.py", line 69, in CLI
    cfg = parser.parse_args(args)
  File "/home1/wts/.local/share/conda/envs/template/lib/python3.9/site-packages/jsonargparse/deprecated.py", line 112, in patched_parse
    cfg = parse_method(*args, _skip_check=_skip_check, **kwargs)
  File "/home1/wts/.local/share/conda/envs/template/lib/python3.9/site-packages/jsonargparse/core.py", line 345, in parse_args
    cfg, unk = self.parse_known_args(args=args)
  File "/home1/wts/.local/share/conda/envs/template/lib/python3.9/site-packages/jsonargparse/core.py", line 230, in parse_known_args
    namespace, args = self._parse_known_args(args, namespace)
  File "/home1/wts/.local/share/conda/envs/template/lib/python3.9/argparse.py", line 2062, in _parse_known_args
    start_index = consume_optional(start_index)
  File "/home1/wts/.local/share/conda/envs/template/lib/python3.9/argparse.py", line 2002, in consume_optional
    take_action(action, args, option_string)
  File "/home1/wts/.local/share/conda/envs/template/lib/python3.9/argparse.py", line 1930, in take_action
    action(self, namespace, argument_values, option_string)
  File "/home1/wts/.local/share/conda/envs/template/lib/python3.9/site-packages/jsonargparse/actions.py", line 209, in __call__
    self.apply_config(parser, cfg, self.dest, values)
  File "/home1/wts/.local/share/conda/envs/template/lib/python3.9/site-packages/jsonargparse/actions.py", line 228, in apply_config
    cfg_file = parser.parse_path(value, **kwargs)
  File "/home1/wts/.local/share/conda/envs/template/lib/python3.9/site-packages/jsonargparse/core.py", line 514, in parse_path
    parsed_cfg = self.parse_string(cfg_str,
  File "/home1/wts/.local/share/conda/envs/template/lib/python3.9/site-packages/jsonargparse/deprecated.py", line 112, in patched_parse
    cfg = parse_method(*args, _skip_check=_skip_check, **kwargs)
  File "/home1/wts/.local/share/conda/envs/template/lib/python3.9/site-packages/jsonargparse/core.py", line 558, in parse_string
    cfg = self._load_config_parser_mode(cfg_str, cfg_path, ext_vars)
  File "/home1/wts/.local/share/conda/envs/template/lib/python3.9/site-packages/jsonargparse/core.py", line 598, in _load_config_parser_mode
    cfg_str = _jsonnet.evaluate_snippet(cfg_path, cfg_str, ext_vars=ext_vars, ext_codes=ext_codes)
RuntimeError: RUNTIME ERROR: couldn't open import "name.libsonnet": no match locally or in the Jsonnet library paths.
	conf/test.jsonnet:1:14-37	thunk <name>
	conf/test.jsonnet:4:11-15	object <anonymous>
	During manifestation

Discovery

And I also find evaluate_snippet(filename, expr) use filename to detect locally directory:

import _jsonnet
snippet = """
local name = import 'name.libsonnet';
{
  "name": name,
  "prize": 100,
}
"""
_jsonnet.evaluate_snippet("", snippet) # this got error
_jsonnet.evaluate_snippet("conf/test.jsonnet", snippet) # works fine
_jsonnet.evaluate_snippet("conf", snippet) # works fine too

It seems like jsonargparse somehow change current working directory which can be verify through print(os.getcwd())

which make cfg_str = _jsonnet.evaluate_snippet(cfg_path, cfg_str, ext_vars=ext_vars, ext_codes=ext_codes) failed

but cfg_str = _jsonnet.evaluate_snippet("", cfg_str, ext_vars=ext_vars, ext_codes=ext_codes) works.

BTW, why not use evaluate_file(filename) directly but evaluate_snippet(filename, expr). They both accept ext_vars and ext_codes

[mauvilsa]

It seems like jsonargparse somehow change current working directory which can be verify through print(os.getcwd())

Yes. This is so that relative paths found inside configs work as expected.

which make cfg_str = _jsonnet.evaluate_snippet(cfg_path, cfg_str, ext_vars=ext_vars, ext_codes=ext_codes) failed

but cfg_str = _jsonnet.evaluate_snippet("", cfg_str, ext_vars=ext_vars, ext_codes=ext_codes) works.

The idea of giving cfg_path was so that error messages include the file name. Haven't looked at the problem yet, but might be that the fix is to do os.path.basename(cfg_path).

BTW, why not use evaluate_file(filename) directly but evaluate_snippet(filename, expr). They both accept ext_vars and ext_codes

It is evaluate_snippet because the function _load_config_parser_mode is used for both file paths and strings. This allows for example to provide an entire config via an environment variable.

[mauvilsa]

This is fixed in commit f08329c.

[tshu-w]

The idea of giving cfg_path was so that error messages include the file name. Haven't looked at the problem yet, but might be that the fix is to do os.path.basename(cfg_path).

Yes, I can confirm basename fix this issue.

Thx you for your quick fix, feel free to close it when released. ;)

[mauvilsa]

This is now part of v4.3.0 which has just been released.