Colcon is incopatible with pyparsing 3

[KazNX]

Hi,

I've just done a pip3 update and found this has broken colcon due to a version mismatch with pyparsing. Exception message below.

ERROR:colcon.colcon_core.entry_point:Exception loading extension 'colcon_core.environment_variable.defaults': (pyparsing 3.0.4 (/usr/local/lib/python3.6/dist-packages), Requirement.parse('pyparsing<3,>=2.0.2'), {'packaging'})
Traceback (most recent call last):
  File "/usr/local/lib/python3.6/dist-packages/colcon_core/entry_point.py", line 120, in load_entry_points
    extension_type = load_entry_point(entry_point)
  File "/usr/local/lib/python3.6/dist-packages/colcon_core/entry_point.py", line 168, in load_entry_point
    return entry_point.load()
  File "/usr/local/lib/python3.6/dist-packages/pkg_resources/__init__.py", line 2454, in load
    self.require(*args, **kwargs)
  File "/usr/local/lib/python3.6/dist-packages/pkg_resources/__init__.py", line 2477, in require
    items = working_set.resolve(reqs, env, installer, extras=self.extras)
  File "/usr/local/lib/python3.6/dist-packages/pkg_resources/__init__.py", line 782, in resolve
    raise VersionConflict(dist, req).with_context(dependent_req)
pkg_resources.ContextualVersionConflict: (pyparsing 3.0.4 (/usr/local/lib/python3.6/dist-packages), Requirement.parse('pyparsing<3,>=2.0.2'), {'packaging'})

[pablogs9]

+1 here. Any solution for that?

[mikepurvis]

Can either of you confirm that colcon actually breaks with pyparsing 3.x, or was this just a semver best practices thing, to not assume that a new major version will always be safe to take? To check it, install colcon-core from source with that requirement dropped from the setup file.

Alternatively, try running colcon from a venv where you have a newer pip, since pip > 20.3 now has a backtracking-capable resolver that can detect and correct these situations.

[KazNX]

Some additional notes and steps to reproduce.

• As of yesterday it was fine on Ubuntu 20.04 because that was still fetching pyparsing version 2.4.6
• Locking pyparsing to 2.4.6 or 2.4.7 while upgrading setuptools works fine - e.g., use the following pip command to install colcon:
  • pip3 install -U setuptools && pip3 install pyparsing==2.4.6 colcon-common-extensions

Reproduction guide with docker

# Start a Unbutu 18.04 image
docker run --rm -it ubuntu:bionic /bin/bash
# Remaining commands run in the image:
apt update && apt install -y python3-pip
pip3 install colcon-common-extensions
# The following command will fail `pyparsing` version checks.
colcon --help

[KazNX]

@mikepurvis

Can either of you confirm that colcon actually breaks with pyparsing 3.x

Unfotunately I cannot. While I can see this is an early versioning check validation failure, rather than a runtime error, I am not familiar enough with the python ecosystem to understand where the version check is configured. I dug around yesteray to no avail. As such, I'm not familiar enough with how to circumvent the version check and try running with version 3.

[mikepurvis]

@KazNX Per my previous message: "To check it, install colcon-core from source with that requirement dropped from the setup file."

[Acuadros95]

I am getting the same error, but this is a dependency from the packaging 21.1 packet: detail
Anyway, they are aware of this problem and should be fixed soon on a new release: pypa/packaging#481

[pablogs9]

@Acuadros95 pypa/packaging#481 is closed but this issue continues. Any idea?

[Acuadros95]

@pablogs9 The fix is not released yet, see pypa/packaging#483