[bazel] Bump Bazel version to 6.2.1 #17021

dmcardle · 2023-01-10T18:19:39Z

This PR experimentally bumps the Bazel version to ~~6.0.0~~ 6.2.1, the latest release. Let's see what CI makes of it.

drewmacrae

I did some testing earlier and it looks good. Passed all non-broken bazel tests we have IIRC. We need this for remote execution to let us tag fpga tests as "exclusive-if-local"

This might need users to update and should probably be scheduled and announced.

LGTM

dmcardle · 2023-01-10T18:34:35Z

Thanks, @drewmacrae! I saw your PR (#16949) and hoped I could separate this part of it.

dmcardle · 2023-01-10T21:14:46Z

@timothytrippel Do you have concerns about the failing airgapped build? It looks like it hits UnrecoverableHttpException, which is a different failure mode than I reported in #16960.

timothytrippel · 2023-01-11T17:02:41Z

@timothytrippel Do you have concerns about the failing airgapped build? It looks like it hits UnrecoverableHttpException, which is a different failure mode than I reported in #16960.

Yes, we should probably test this in our airgapped environment before pushing this through.

a-will · 2023-01-18T19:07:42Z

Looks like some more dependencies may need to be added to the fetch list in util/prep-bazel-airgapped-build.sh

INFO: Repository remote_java_tools instantiated at:
  /DEFAULT.WORKSPACE.SUFFIX:363:6: in <toplevel>
  /home/vsts/.cache/bazel/_bazel_vsts/5c27df7f718ab35828b0e0cfe0993663/external/bazel_tools/tools/build_defs/repo/utils.bzl:233:18: in maybe
Repository rule http_archive defined at:
  /home/vsts/.cache/bazel/_bazel_vsts/5c27df7f718ab35828b0e0cfe0993663/external/bazel_tools/tools/build_defs/repo/http.bzl:372:31: in <toplevel>
WARNING: Download from https://mirror.bazel.build/bazel_java_tools/releases/java/v11.9/java_tools-v11.9.zip failed: class com.google.devtools.build.lib.bazel.repository.downloader.UnrecoverableHttpException Unknown host: mirror.bazel.build
WARNING: Download from https://github.com/bazelbuild/java_tools/releases/download/java_v11.9/java_tools-v11.9.zip failed: class com.google.devtools.build.lib.bazel.repository.downloader.UnrecoverableHttpException Unknown host: github.com
ERROR: An error occurred during the fetch of repository 'remote_java_tools':
   Traceback (most recent call last):
	File "/home/vsts/.cache/bazel/_bazel_vsts/5c27df7f718ab35828b0e0cfe0993663/external/bazel_tools/tools/build_defs/repo/http.bzl", line 132, column 45, in _http_archive_impl
		download_info = ctx.download_and_extract(
Error in download_and_extract: java.io.IOException: Error downloading [https://mirror.bazel.build/bazel_java_tools/releases/java/v11.9/java_tools-v11.9.zip, https://github.com/bazelbuild/java_tools/releases/download/java_v11.9/java_tools-v11.9.zip] to /home/vsts/.cache/bazel/_bazel_vsts/5c27df7f718ab35828b0e0cfe0993663/external/remote_java_tools/temp9415537091301209138/java_tools-v11.9.zip: Unknown host: github.com
ERROR: /DEFAULT.WORKSPACE.SUFFIX:363:6: fetching http_archive rule //external:remote_java_tools: Traceback (most recent call last):
	File "/home/vsts/.cache/bazel/_bazel_vsts/5c27df7f718ab35828b0e0cfe0993663/external/bazel_tools/tools/build_defs/repo/http.bzl", line 132, column 45, in _http_archive_impl
		download_info = ctx.download_and_extract(
Error in download_and_extract: java.io.IOException: Error downloading [https://mirror.bazel.build/bazel_java_tools/releases/java/v11.9/java_tools-v11.9.zip, https://github.com/bazelbuild/java_tools/releases/download/java_v11.9/java_tools-v11.9.zip] to /home/vsts/.cache/bazel/_bazel_vsts/5c27df7f718ab35828b0e0cfe0993663/external/remote_java_tools/temp9415537091301209138/java_tools-v11.9.zip: Unknown host: github.com
ERROR: /home/vsts/.cache/bazel/_bazel_vsts/5c27df7f718ab35828b0e0cfe0993663/external/bazel_tools/tools/jdk/BUILD:327:27: @bazel_tools//tools/jdk:toolchain_java8 depends on @remote_java_tools//:JavaBuilder in repository @remote_java_tools which failed to fetch. no such package '@remote_java_tools//': java.io.IOException: Error downloading [https://mirror.bazel.build/bazel_java_tools/releases/java/v11.9/java_tools-v11.9.zip, https://github.com/bazelbuild/java_tools/releases/download/java_v11.9/java_tools-v11.9.zip] to /home/vsts/.cache/bazel/_bazel_vsts/5c27df7f718ab35828b0e0cfe0993663/external/remote_java_tools/temp9415537091301209138/java_tools-v11.9.zip: Unknown host: github.com
ERROR: /home/vsts/.cache/bazel/_bazel_vsts/5c27df7f718ab35828b0e0cfe0993663/external/bazel_tools/tools/jdk/BUILD:327:27: @bazel_tools//tools/jdk:toolchain_java8 depends on @remote_java_tools//:GenClass in repository @remote_java_tools which failed to fetch. no such package '@remote_java_tools//': java.io.IOException: Error downloading [https://mirror.bazel.build/bazel_java_tools/releases/java/v11.9/java_tools-v11.9.zip, https://github.com/bazelbuild/java_tools/releases/download/java_v11.9/java_tools-v11.9.zip] to /home/vsts/.cache/bazel/_bazel_vsts/5c27df7f718ab35828b0e0cfe0993663/external/remote_java_tools/temp9415537091301209138/java_tools-v11.9.zip: Unknown host: github.com
ERROR: /home/vsts/.cache/bazel/_bazel_vsts/5c27df7f718ab35828b0e0cfe0993663/external/bazel_tools/tools/jdk/BUILD:327:27: @bazel_tools//tools/jdk:toolchain_java8 depends on @remote_java_tools//:TurbineDirect in repository @remote_java_tools which failed to fetch. no such package '@remote_java_tools//': java.io.IOException: Error downloading [https://mirror.bazel.build/bazel_java_tools/releases/java/v11.9/java_tools-v11.9.zip, https://github.com/bazelbuild/java_tools/releases/download/java_v11.9/java_tools-v11.9.zip] to /home/vsts/.cache/bazel/_bazel_vsts/5c27df7f718ab35828b0e0cfe0993663/external/remote_java_tools/temp9415537091301209138/java_tools-v11.9.zip: Unknown host: github.com
ERROR: /home/vsts/.cache/bazel/_bazel_vsts/5c27df7f718ab35828b0e0cfe0993663/external/bazel_tools/tools/jdk/BUILD:327:27: @bazel_tools//tools/jdk:toolchain_java8 depends on @remote_java_tools//:jacoco_coverage_runner_filegroup in repository @remote_java_tools which failed to fetch. no such package '@remote_java_tools//': java.io.IOException: Error downloading [https://mirror.bazel.build/bazel_java_tools/releases/java/v11.9/java_tools-v11.9.zip, https://github.com/bazelbuild/java_tools/releases/download/java_v11.9/java_tools-v11.9.zip] to /home/vsts/.cache/bazel/_bazel_vsts/5c27df7f718ab35828b0e0cfe0993663/external/remote_java_tools/temp9415537091301209138/java_tools-v11.9.zip: Unknown host: github.com
ERROR: Analysis of target '//sw/device/silicon_creator/rom/e2e:rom_e2e_debug_test_otp_rma' failed; build aborted:

dmcardle · 2023-01-18T20:35:04Z

I had to add a couple arguments to the bazel fetch command, but Bazel 6 now passes the airgapped build test!

dmcardle · 2023-01-18T22:04:44Z

I see that //sw/device/lib/ujson/rust:roundtrip_test failed on CI:

---- test::test_foo stdout ----
sent: {
  "foo": 5,
  "bar": 10,
  "message": "Hello"
}
thread 'test::test_foo' panicked at 'assertion failed: `(left == right)`
  left: `1`,
 right: `0`:

Oddly, it repeatedly passes on my machine. I even tried running it a bunch of times, since it seems to be a randomized test.

./bazelisk.sh test --runs_per_test=5000 --cache_test_results=no //sw/device/lib/ujson/rust:roundtrip_test

I think I'll ask CI to try it again.

drewmacrae · 2023-01-19T14:23:59Z

I think we're on-track to have the extra-resources feature get cherry-picked to 6.1.0 so we'll probably want to upgrade when it's released.

dmcardle · 2023-01-19T19:33:29Z

@drewmacrae Great! Let's proceed to upgrade to Bazel 6 and then upgrade to 6.1.0 when it's time :)

@cfrantz Do you have any thoughts on the failure in //sw/device/lib/ujson/rust:roundtrip_test? It feels unrelated to me, but I don't want to lose track of it.

drewmacrae · 2023-03-23T19:20:22Z

Now that we have bazel with "extra resources" I'd prefer we use something later than 6.1.0 so I've added #17664

alphan · 2023-07-05T17:53:09Z

Do we want to revive this now that ES freeze is over?

@dmcardle @arunthomas @a-will @timothytrippel @cfrantz

dmcardle · 2023-07-06T13:40:56Z

I'm biased, but I think this is a good idea!

Bazel 6 adds support for Bzlmod, which may simplify our third-party dependencies.

The new cquery flag --output=files addresses a minor (but recurring) pain point. We could probably replace bazelisk.sh's implementation of outquery, which I recall being a little buggy with multiple files.

The latest release, 6.2.1, also includes Drew's PR that I believe would help a single Bazel instance manage multiple FPGAs — basically a finer-grained control than the "exclusive" tag. There'd still be some integration work like teaching opentitantool how to select the correct JTAG adapter for a given FPGA. bazelbuild/bazel#17229

Release notes: https://github.com/bazelbuild/bazel/releases/tag/6.0.0

timothytrippel · 2023-07-12T19:37:12Z

Yes I think we should be safe to upgrade. Once this PR gets rebased and is passing CI I can test it on our nightly regression infra to make sure the newer version works there too.

Signed-off-by: Dan McArdle <[email protected]>

dmcardle · 2023-07-13T13:29:59Z

Thanks, folks!

Just rebased and changed from Bazel 6.0.0 to 6.2.1.

I also created an experimental PR to see if we can omit the changes to the airgapped script: #19187.

dmcardle · 2023-07-13T16:38:23Z

Yes I think we should be safe to upgrade. Once this PR gets rebased and is passing CI I can test it on our nightly regression infra to make sure the newer version works there too.

@timothytrippel CI is green! Could you give this a shot on the nightly infra? I'll wait for you to give the go-ahead before merging this PR.

timothytrippel · 2023-07-14T21:52:25Z

I tested this and it seems to work on our infrastructure, so I am merging.

dmcardle added the SW:Build System label Jan 10, 2023

dmcardle self-assigned this Jan 10, 2023

drewmacrae reviewed Jan 10, 2023

View reviewed changes

dmcardle force-pushed the dmcardle/bump-bazel-version branch from 63d2cbd to e0724db Compare January 18, 2023 18:37

dmcardle force-pushed the dmcardle/bump-bazel-version branch from e0724db to e3d1f57 Compare January 18, 2023 19:40

dmcardle requested review from timothytrippel and drewmacrae January 18, 2023 20:34

dmcardle marked this pull request as ready for review January 18, 2023 20:34

timothytrippel approved these changes Jul 12, 2023

View reviewed changes

timothytrippel requested review from cfrantz and removed request for drewmacrae July 12, 2023 19:37

cfrantz approved these changes Jul 12, 2023

View reviewed changes

[bazel] Bump Bazel version to 6.2.1

4578cb8

Signed-off-by: Dan McArdle <[email protected]>

dmcardle force-pushed the dmcardle/bump-bazel-version branch from e3d1f57 to 4578cb8 Compare July 13, 2023 13:13

dmcardle changed the title ~~[bazel] Bump Bazel version to 6.0.0~~ [bazel] Bump Bazel version to 6.2.1 Jul 13, 2023

timothytrippel merged commit 7bbcd1f into lowRISC:master Jul 14, 2023

dmcardle deleted the dmcardle/bump-bazel-version branch July 14, 2023 22:34

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[bazel] Bump Bazel version to 6.2.1 #17021

[bazel] Bump Bazel version to 6.2.1 #17021

dmcardle commented Jan 10, 2023 •

edited

Loading

drewmacrae left a comment

dmcardle commented Jan 10, 2023

dmcardle commented Jan 10, 2023

timothytrippel commented Jan 11, 2023

a-will commented Jan 18, 2023

dmcardle commented Jan 18, 2023

dmcardle commented Jan 18, 2023

drewmacrae commented Jan 19, 2023

dmcardle commented Jan 19, 2023

drewmacrae commented Mar 23, 2023

alphan commented Jul 5, 2023

dmcardle commented Jul 6, 2023

timothytrippel commented Jul 12, 2023

dmcardle commented Jul 13, 2023

dmcardle commented Jul 13, 2023

timothytrippel commented Jul 14, 2023

[bazel] Bump Bazel version to 6.2.1 #17021

[bazel] Bump Bazel version to 6.2.1 #17021

Conversation

dmcardle commented Jan 10, 2023 • edited Loading

drewmacrae left a comment

Choose a reason for hiding this comment

dmcardle commented Jan 10, 2023

dmcardle commented Jan 10, 2023

timothytrippel commented Jan 11, 2023

a-will commented Jan 18, 2023

dmcardle commented Jan 18, 2023

dmcardle commented Jan 18, 2023

drewmacrae commented Jan 19, 2023

dmcardle commented Jan 19, 2023

drewmacrae commented Mar 23, 2023

alphan commented Jul 5, 2023

dmcardle commented Jul 6, 2023

timothytrippel commented Jul 12, 2023

dmcardle commented Jul 13, 2023

dmcardle commented Jul 13, 2023

timothytrippel commented Jul 14, 2023

dmcardle commented Jan 10, 2023 •

edited

Loading