poetry 1.1.14 RunTimeError - Poetry configuration is invalid

[dazza-codes]

• I am on the latest Poetry version.

• I have searched the issues of this repo and believe that this is not a duplicate.

• If an exception occurs when executing a command, I executed it again in debug mode (-vvv option).

  • cannot yet reproduce this outside of a gitlab-CI job

• OS version and name: docker image python:3.7

• Poetry version: 1.1.14

• Link of a Gist with the contents of your pyproject.toml file: sorry, this is private

Issue

From the stack trace, I cannot tell whether there is a problem with the poetry install and config file or a problem with a pyproject.toml file, but the latter is unlikely since there are no changes to it and it has been working with all previous releases of poetry (and poetry check and poetry build are working OK).

This is a log from a gitlab-CI job, which is a repeatable failure (not a one-off). Nothing has changed in the .gitlab-ci.yml settings for some time, only the poetry version or a pip version might be new. The gitlab CI pipelines have worked last week with poetry 1.1.14 and pip 22.2.1, so this failure is difficult to understand and replicate.

$ python -V
Python 3.7.13
$ python -m pip install --upgrade pip virtualenv
Requirement already satisfied: pip in /usr/local/lib/python3.7/site-packages (22.0.4)
Collecting pip
  Downloading pip-22.2.2-py3-none-any.whl (2.0 MB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 2.0/2.0 MB 15.5 MB/s eta 0:00:00
Collecting virtualenv
  Downloading virtualenv-20.16.3-py2.py3-none-any.whl (8.8 MB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 8.8/8.8 MB 46.7 MB/s eta 0:00:00
Collecting filelock<4,>=3.4.1
  Downloading filelock-3.7.1-py3-none-any.whl (10 kB)
Collecting distlib<1,>=0.3.5
  Downloading distlib-0.3.5-py2.py3-none-any.whl (466 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 467.0/467.0 KB 38.6 MB/s eta 0:00:00
Collecting platformdirs<3,>=2.4
  Downloading platformdirs-2.5.2-py3-none-any.whl (14 kB)
Collecting importlib-metadata>=4.8.3
  Downloading importlib_metadata-4.12.0-py3-none-any.whl (21 kB)
Collecting zipp>=0.5
  Downloading zipp-3.8.1-py3-none-any.whl (5.6 kB)
Collecting typing-extensions>=3.6.4
  Downloading typing_extensions-4.3.0-py3-none-any.whl (25 kB)
Installing collected packages: distlib, zipp, typing-extensions, platformdirs, pip, filelock, importlib-metadata, virtualenv
  Attempting uninstall: pip
    Found existing installation: pip 22.0.4
    Uninstalling pip-22.0.4:
      Successfully uninstalled pip-22.0.4
Successfully installed distlib-0.3.5 filelock-3.7.1 importlib-metadata-4.12.0 pip-22.2.2 platformdirs-2.5.2 typing-extensions-4.3.0 virtualenv-20.16.3 zipp-3.8.1
WARNING: Running pip as the 'root' user can result in broken permissions and conflicting behaviour with the system package manager. It is recommended to use a virtual environment instead: https://pip.pypa.io/warnings/venv
$ curl -sSL https://install.python-poetry.org | python -
Retrieving Poetry metadata
# Welcome to Poetry!
This will download and install the latest version of Poetry,
a dependency and package manager for Python.
It will add the `poetry` command to Poetry's bin directory, located at:
/root/.local/bin
You can uninstall at any time by executing this script with the --uninstall option,
and these changes will be reverted.
Installing Poetry (1.1.14)
Installing Poetry (1.1.14): Creating environment
Installing Poetry (1.1.14): Installing Poetry
Installing Poetry (1.1.14): Creating script
Installing Poetry (1.1.14): Done
Poetry (1.1.14) is installed now. Great!
To get started you need Poetry's bin directory (/root/.local/bin) in your `PATH`
environment variable.
Add `export PATH="/root/.local/bin:$PATH"` to your shell configuration file.
Alternatively, you can call Poetry explicitly with `/root/.local/bin/poetry`.
You can test that everything is set up by executing:
`poetry --version`
$ export PATH="${HOME}/.local/bin:${PATH}"
$ poetry --version
Poetry version 1.1.14

$ virtualenv venv
created virtual environment CPython3.7.13.final.0-64 in 934ms
  creator CPython3Posix(dest=/builds/XXXX/venv, clear=False, no_vcs_ignore=False, global=False)
  seeder FromAppData(download=False, pip=bundle, setuptools=bundle, wheel=bundle, via=copy, app_data_dir=/root/.local/share/virtualenv)
    added seed packages: pip==22.2.2, setuptools==63.4.1, wheel==0.37.1
  activators BashActivator,CShellActivator,FishActivator,NushellActivator,PowerShellActivator,PythonActivator
$ source venv/bin/activate
$ python -m pip install --upgrade pip
Looking in indexes: https://gitlab-token:****@gitlab.com/api/v4/projects/XXXX/packages/pypi/simple
Requirement already satisfied: pip in ./venv/lib/python3.7/site-packages (22.2.2)

$ poetry install -v --no-interaction --extras all
  Stack trace:
  9  ~/.local/share/pypoetry/venv/lib/python3.7/site-packages/clikit/console_application.py:131 in run
     status_code = command.handle(parsed_args, io)
  8  ~/.local/share/pypoetry/venv/lib/python3.7/site-packages/clikit/api/command/command.py:120 in handle
     status_code = self._do_handle(args, io)
  7  ~/.local/share/pypoetry/venv/lib/python3.7/site-packages/clikit/api/command/command.py:163 in _do_handle
     self._dispatcher.dispatch(PRE_HANDLE, event)
  6  ~/.local/share/pypoetry/venv/lib/python3.7/site-packages/clikit/api/event/event_dispatcher.py:22 in dispatch
     self._do_dispatch(listeners, event_name, event)
  5  ~/.local/share/pypoetry/venv/lib/python3.7/site-packages/clikit/api/event/event_dispatcher.py:[89](https://gitlab.com/jupiterintel/jupiter_gis/-/jobs/2830195091#L89) in _do_dispatch
     listener(event, event_name, self)
  4  ~/.local/share/pypoetry/venv/lib/python3.7/site-packages/poetry/console/config/application_config.py:[116](https://gitlab.com/jupiterintel/jupiter_gis/-/jobs/2830195091#L116) in set_env
     poetry = command.poetry
  3  ~/.local/share/pypoetry/venv/lib/python3.7/site-packages/poetry/console/commands/command.py:10 in poetry
     return self.application.poetry
  2  ~/.local/share/pypoetry/venv/lib/python3.7/site-packages/poetry/console/application.py:69 in poetry
     self._poetry = Factory().create_poetry(Path.cwd())
  1  ~/.local/share/pypoetry/venv/lib/python3.7/site-packages/poetry/factory.py:33 in create_poetry
     base_poetry = super(Factory, self).create_poetry(cwd)
  RuntimeError
  The Poetry configuration is invalid:
    - [extras.all.25] None is not of type 'string'
  
  at ~/.local/share/pypoetry/venv/lib/python3.7/site-packages/poetry/core/factory.py:43 in create_poetry
       39│             message = ""
       40│             for error in check_result["errors"]:
       41│                 message += "  - {}\n".format(error)
       42│ 
    →  43│             raise RuntimeError("The Poetry configuration is invalid:\n" + message)
       44│ 
       45│         # Load package
       46│         name = local_config["name"]
       47│         version = local_config["version"]

[dazza-codes]

$ poetry config --list
cache-dir = "/root/.cache/pypoetry"
experimental.new-installer = true
installer.parallel = true
virtualenvs.create = true
virtualenvs.in-project = null
virtualenvs.path = "{cache-dir}/virtualenvs"  # /root/.cache/pypoetry/virtualenvs

Could the problem be virtualenvs.in-project = null? AFAIK, all these config settings are defaults from the installation. Those config values are from the gitlab-CI job. On my dev-laptop, I use virtualenvs.in-project = false.

[dazza-codes]

This might be slightly off topic, but I bumped into this while checking some config settings. Something is misbehaving in config management. For example, in my dev-laptop, there seems to be an obsolete config value that I cannot remove, i.e.

$ poetry config --list
cache-dir = "/home/joe/.cache/pypoetry"
experimental.new-installer = true
installer.parallel = true
settings.virtualenvs.in-project = false
virtualenvs.create = true
virtualenvs.in-project = false
virtualenvs.path = "{cache-dir}/virtualenvs"  # /home/joe/.cache/pypoetry/virtualenvs

$ poetry config --unset settings.virtualenvs.in-project

  ValueError

  Setting settings.virtualenvs.in-project does not exist

  at ~/.local/pipx/venvs/poetry/lib/python3.8/site-packages/poetry/console/commands/config.py:248 in handle
      244│                 raise ValueError("You must pass exactly 1 value")
      245│ 
      246│             return 0
      247│ 
    → 248│         raise ValueError("Setting {} does not exist".format(self.argument("key")))
      249│ 
      250│     def _handle_single_value(self, source, key, callbacks, values):
      251│         validator, normalizer, _ = callbacks
      252│ 

The config file looks like this, from $HOME/.config/pypoetry/config.toml

[settings]
[settings.virtualenvs]
in-project = false

[virtualenvs]
in-project = false

After a manual edit to remove the block for [settings.virtualenvs], it's gone and poetry is working OK.

[dazza-codes]

Explicitly setting some additional config values did not solve the problem 😞

$ poetry config virtualenvs.create false
$ poetry config virtualenvs.in-project false
$ poetry config -vvv --list
Loading configuration file /root/.config/pypoetry/config.toml
Loading configuration file /root/.config/pypoetry/auth.toml
cache-dir = "/root/.cache/pypoetry"
experimental.new-installer = true
installer.parallel = true
virtualenvs.create = false
virtualenvs.in-project = false
virtualenvs.path = "{cache-dir}/virtualenvs"  # /root/.cache/pypoetry/virtualenvs

$ poetry install -vvv --no-interaction --extras all
  Stack trace:
  9  ~/.local/share/pypoetry/venv/lib/python3.7/site-packages/clikit/console_application.py:131 in run
      129│             parsed_args = resolved_command.args
      130│ 
    → 131│             status_code = command.handle(parsed_args, io)
      132│         except KeyboardInterrupt:
      133│             status_code = 1
  8  ~/.local/share/pypoetry/venv/lib/python3.7/site-packages/clikit/api/command/command.py:120 in handle
      118│     def handle(self, args, io):  # type: (Args, IO) -> int
      119│         try:
    → 120│             status_code = self._do_handle(args, io)
      121│         except KeyboardInterrupt:
      122│             if io.is_debug():
  7  ~/.local/share/pypoetry/venv/lib/python3.7/site-packages/clikit/api/command/command.py:163 in _do_handle
      161│         if self._dispatcher and self._dispatcher.has_listeners(PRE_HANDLE):
      [162](https://gitlab.com/jupiterintel/jupiter_gis/-/jobs/2834295647#L162)│             event = PreHandleEvent(args, io, self)
    → [163](https://gitlab.com/jupiterintel/jupiter_gis/-/jobs/2834295647#L163)│             self._dispatcher.dispatch(PRE_HANDLE, event)
      [164](https://gitlab.com/jupiterintel/jupiter_gis/-/jobs/2834295647#L164)│ 
      [165](https://gitlab.com/jupiterintel/jupiter_gis/-/jobs/2834295647#L165)│             if event.is_handled():
  6  ~/.local/share/pypoetry/venv/lib/python3.7/site-packages/clikit/api/event/event_dispatcher.py:22 in dispatch
       20│ 
       21│         if listeners:
    →  22│             self._do_dispatch(listeners, event_name, event)
       23│ 
       24│         return event
  5  ~/.local/share/pypoetry/venv/lib/python3.7/site-packages/clikit/api/event/event_dispatcher.py:89 in _do_dispatch
       87│                 break
       88│ 
    →  89│             listener(event, event_name, self)
       90│ 
       91│     def _sort_listeners(self, event_name):  # type: (str) -> None
  4  ~/.local/share/pypoetry/venv/lib/python3.7/site-packages/poetry/console/config/application_config.py:116 in set_env
      114│ 
      115│         io = event.io
    → 116│         poetry = command.poetry
      117│ 
      118│         env_manager = EnvManager(poetry)
  3  ~/.local/share/pypoetry/venv/lib/python3.7/site-packages/poetry/console/commands/command.py:10 in poetry
       8│     @property
       9│     def poetry(self):
    → 10│         return self.application.poetry
      11│ 
      12│     def reset_poetry(self):  # type: () -> None
  2  ~/.local/share/pypoetry/venv/lib/python3.7/site-packages/poetry/console/application.py:69 in poetry
       67│             return self._poetry
       68│ 
    →  69│         self._poetry = Factory().create_poetry(Path.cwd())
       70│ 
       71│         return self._poetry
  1  ~/.local/share/pypoetry/venv/lib/python3.7/site-packages/poetry/factory.py:33 in create_poetry
       31│             io = NullIO()
       32│ 
    →  33│         base_poetry = super(Factory, self).create_poetry(cwd)
       34│ 
       35│         locker = Locker(
  RuntimeError
  The Poetry configuration is invalid:
    - [extras.all.25] None is not of type 'string'
  
  at ~/.local/share/pypoetry/venv/lib/python3.7/site-packages/poetry/core/factory.py:43 in create_poetry
       39│             message = ""
       40│             for error in check_result["errors"]:
       41│                 message += "  - {}\n".format(error)
       42│ 
    →  43│             raise RuntimeError("The Poetry configuration is invalid:\n" + message)
       44│ 
       45│         # Load package
       46│         name = local_config["name"]
       47│         version = local_config["version"]

[dimbleby]

The error message tells you exactly which key it is not happy with. If you are unable / unwilling to share your pyproject.toml, I doubt that there's much more that anyone can do for you.

The Poetry configuration is invalid:
    - [extras.all.25] None is not of type 'string'

[dazza-codes]

I can share the extras section:

[tool.poetry.extras]
all = [
    "contextily",
    "descartes",

    "dask-snowflake",
    "distributed",
    "streamz",

    "geopy",
    "pyepsg",
    "pygeos",

    "bokeh",
    "folium",
    "graphviz",
    "ipyparallel",
    "jupyterlab",
    "matplotlib",
    "seaborn",

    "GeoAlchemy2",
    "SQLAlchemy",
    "pg8000",
    "psycopg2-binary",
    "snowflake-connector-python",

    "pyarrow",

    "netcdf4",
    "scipy",
    "xarray",
    "zarr",

    # redis-queue support
    "arq",
    "rq",
    "rqmonitor",

    "prefect",

    "fastapi",
    "strawberry-graphql",
    "mangum",
    "hypercorn",
    "watchgod",
]

The comment for # redis-queue support does not seem to cause any problems and, for some reason, this bug has mysteriously disappeared in recent gitlab-CI jobs without any changes to the pyproject.toml file. I can only guess that poetry.lock was corrupted or a poetry update fixed it or that the error was generated from a library somewhere in the dependency tree that is now fixed.

[radoering]

Probably, similar to #6133 and #6135 and fixed by python-poetry/tomlkit#221 with the latest tomlkit version.

[github-actions]

This issue has been automatically locked since there has not been any recent activity after it was closed. Please open a new issue for related bugs.