Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Make macOS bindist configure more deterministic #1462

Merged
merged 1 commit into from
Jan 5, 2021
Merged

Conversation

mboes
Copy link
Member

@mboes mboes commented Dec 29, 2020

GHC from a bindist requires running ./configure and then make install. One would assume that the result is always the same, in
particular if we run this twice on the same host with the same
environment variables. However, this is not the case. See #1461.

The fix is to set an environment variable telling ar and friends to
not futz around with timestamps in static library archives. This
environment variable is only necessary and only has an effect on
macOS, but there's no harm in setting it always. If the bindists stop
having a configure stage, or start using libtool instead of ar and
ranlib, then the fix will better be pushed within the bindists
themselves, using the -D flag to libtool, introduced in XCode
10.2.

Fixes #1461.

@mboes mboes requested a review from aherrmann December 29, 2020 22:22
@mboes
Copy link
Member Author

mboes commented Dec 29, 2020

With this change, on GitHub Actions the build goes down from 36 minutes to less than 13 minutes. With a 99.9% cache hit rate on BuildBuddy.

Copy link
Member

@aherrmann aherrmann left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Great, thanks for tracking this down!

@aherrmann aherrmann added the merge-queue merge on green CI label Jan 4, 2021
@aherrmann
Copy link
Member

This makes me think that we should also be passing D to ar in the static link actions.

@mboes
Copy link
Member Author

mboes commented Jan 5, 2021

If we used libtool instead of ar on macOS, then that would be possible. But I believe ar does not support the -D flag. In fact Bazel sometimes picks libtool over ar, which is deprecated on macOS. But we override that here. Perhaps migrating to libtool completely is what we should be aiming for. Or better yet, maybe it's possible now to leverage the CC toolchain API to give us a linking action that works, without us having to hardcode either ar or libtool.

GHC from a bindist requires running `./configure` and then `make
install`. One would assume that the result is always the same, in
particular if we run this twice on the same host with the same
environment variables. However, this is not the case. See #1461.

The fix is to set an environment variable telling `ar` and friends to
not futz around with timestamps in static library archives. This
environment variable is only necessary and only has an effect on
macOS, but there's no harm in setting it always. If the bindists stop
having a configure stage, or start using `libtool` instead of `ar` and
`ranlib`, then the fix will better be pushed within the bindists
themselves, using the `-D` flag to `libtool`, introduced in XCode
10.2.

Fixes #1461.
@TerrorJack TerrorJack force-pushed the macos-deterministic branch from 726bb31 to db86f9a Compare January 5, 2021 08:11
@mergify mergify bot merged commit ad98ba4 into master Jan 5, 2021
@mergify mergify bot deleted the macos-deterministic branch January 5, 2021 08:46
@mergify mergify bot removed the merge-queue merge on green CI label Jan 5, 2021
@aherrmann
Copy link
Member

@mboes Good point about the CC Starlark API. I've opened #1469 to track this.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Bindist builds on macOS are non-deterministic
2 participants