Using rules_python requires undocumented dependency on bazel_skylib

[oliverlee]

I'm getting the following error:

> bazel fetch //...
ERROR: Failed to load Starlark extension '@bazel_skylib//lib:dicts.bzl'.
Cycle in the workspace file detected. This indicates that a repository is used prior to being defined.
The following chain of repository dependencies lead to the missing definition.
 - @bazel_skylib

with this:

# //:WORKSPACE.bazel
load("@bazel_tools//tools/build_defs/repo:http.bzl", "http_archive")

http_archive(
    name = "rules_python",
    sha256 = "497ca47374f48c8b067d786b512ac10a276211810f4a580178ee9b9ad139323a",
    strip_prefix = "rules_python-0.16.1",
    url = "https://github.com/bazelbuild/rules_python/archive/refs/tags/0.16.1.tar.gz",
)

load("@rules_python//python:repositories.bzl", "python_register_toolchains")

python_register_toolchains(
    name = "python3_10",
    python_version = "3.10",
)

load("@python3_10//:defs.bzl", "interpreter")

and

# //:BUILD.bazel

I assume this is the cause:
https://github.com/bazelbuild/rules_python/blob/main/python/config_settings/transition.bzl#L5

bazel_skylib isn't needed with rules_python 0.14.0.

[yin]

I managed to fix the problem locally by adding the repository to my WORKSPACE file:

http_archive(
    name = "bazel_skylib",
    sha256 = "74d544d96f4a5bb630d465ca8bbcfe231e3594e5aae57e1edbf17a6eb3ca2506",
    urls = [
        "https://mirror.bazel.build/github.com/bazelbuild/bazel-skylib/releases/download/1.3.0/bazel-skylib-1.3.0.tar.gz",
        "https://github.com/bazelbuild/bazel-skylib/releases/download/1.3.0/bazel-skylib-1.3.0.tar.gz",
    ],
)

load("@bazel_skylib//:workspace.bzl", "bazel_skylib_workspace")

bazel_skylib_workspace()

[aiuto]

Almost every rule set depends on skylib. And they might depend on different versions.
The safest way to approach it is to do what you have done and bring skylib in via your own WORKSPACE or MODULE.bzlmod file.

[oliverlee]

I assumed it wasn't necessary given that there is a vendored copy of bazel_skylib and the README states:

This exists so that users of rules_python don't have to install bazel-skylib copied from https://github.com/bazelbuild/bazel-skylib/blob/1.0.3

Maybe it's worth cleaning up the documentation if that statement is no longer valid?

[yin]

I'd rather clean up the Getting Started Guide... That is what people follow and I hear that they give up with Bazel.io altogether after the first error message during their first tutorial... Do you need a hand with that?

…

On Wed, 11 Jan 2023 at 18:04, Oliver Lee ***@***.***> wrote: I assumed it wasn't necessary given that there is a vendored copy of bazel_skylib and the README states: This exists so that users of rules_python don't have to install bazel-skylib copied from https://github.com/bazelbuild/bazel-skylib/blob/1.0.3 Maybe it's worth cleaning up the documentation if that statement is no longer valid? — Reply to this email directly, view it on GitHub <#932 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AACRYNKG7ZXLYLUNMMDXBCLWR5RBVANCNFSM6AAAAAATBSNW4Y> . You are receiving this because you commented.Message ID: ***@***.***>

-- Thank you Kind regards, Matej Sadovsky Software Engineer

[oliverlee]

No it is not problem for me, thanks for asking :)

Cleaning up the Getting Started guide sounds reasonable but feel free to close if don't think it's worth doing.

[alexeagle]

Nope this was a legit bug and an accidental regression in the last release. Sorry about that!

Fixed in #1001

[alexeagle]

@oliverlee where is that README? I'm trying to clean up but I don't see that text in rules_python anywhere.

[oliverlee]

https://github.com/bazelbuild/rules_python/blob/main/third_party/github.com/bazelbuild/bazel-skylib/README.md

[alexeagle]

@yin where is "the Getting Started Guide"? Want to make sure I found all the outdated spots.

[yin]

@alexeagle With "Getting Started Guide" I meant https://github.com/bazelbuild/rules_python/blob/main/README.md

I may have been confusing this with another issue I had with the python_rules version in the doc being outdated.

I've got this error, when trying to set up python toolchain on GitLab CI. My test were failing, because the docker image I am using has preinstalled Python 2.7.

When I added the bazel_skylib dependency explicitly, commit: https://gitlab.com/yin2/language/-/commit/dcb25a5b5969de2e578ae8358e4905ec02eb791c

.. I get another error:

ERROR: /root/.cache/bazel/_bazel_root/4755c6ea53a31c010470a9916f866540/external/rules_python/python/pip_install/pip_repository.bzl:175:13: name 'json' is not defined

And this is the CI run: https://gitlab.com/yin2/language/-/jobs/3583021972

[oliverlee]

It looks like json was introduced in Bazel 4:
https://blog.bazel.build/2021/01/19/bazel-4-0.html#starlark

But your CI runner is using Bazel 3.5.0:
https://gitlab.com/yin2/language/-/jobs/3583021972#L32

I guess rules_python could check the Bazel version with bazel-skylib now that it is a dependency.

[alexeagle]

We do have a check already (this is exactly why we vendored a bit of skylib to begin with)
https://github.com/bazelbuild/rules_python/blob/main/python/pip_install/repositories.bzl#L118-L123

I guess you don't happen to call pip_install_dependencies somehow, so you didn't have that version check applied.

I don't think there's more to do here - thanks!

[yin]

Can we explain this in the docs?

Following the README.md did not get me to a usable workspace. I don't know, which GitLab CI guide I followed anymore, gcr.io/bazel-public/bazel:latest is the official Bazel 3.5.0 docker image and it was not updated with Bazel 4.0

[alexeagle]

docker run --rm -it gcr.io/bazel-public/bazel:6.0.0 version gives me 6.0. If there's a missing validation on the minimum Bazel version, we should fix it in code so we don't rely on someone finding it in the docs.

[yin]

@alexeagle Thank you! The correct docker image is listed in bazelbuild's Release Notes and Official Bazel docs.

I have been using the wrong docker image all along: l.gcr.io/google/bazel:latest. This image stopped being released in 2020 and is tagged with version 3.5.0.