add github actions caching #1005
Closed
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Summary: X-link: facebookincubator/zstrong#1100 Update generated github actions to only run the fetch and and build steps when there are sources expected for a manifest For local github actions testing using `act` this speeds up the test runs, and in real github CI it makes it clearer which steps are actually doing something on the given runner (we don't know exactly what it has installed beforehand) Also set the windows git config the same as on internal CI Reviewed By: bigfootjon Differential Revision: D67839708
Summary: Speed up github builds with github actions CI caching based on the cache-key mechanism already present in getdeps. Cached fizz windows build completed in 22% of original time, linux build in 54% of the original time, and mac in 72% of original time. This saves a lot of waiting around for PR builds on OSS changes as windows signal is usually the lagging one. Speed ups will vary across the different Meta OSS projects. Windows benefits most from caching dependencies as we don't use system packages there at all, linux next as its default runner is slower than mac, and then finaly mac (the strongest hardware of the default github runners). Github allows [up to 10GB cache per repo](https://github.blog/changelog/2021-11-23-github-actions-cache-size-is-now-increased-to-10gb-per-repository/), expiring data over capacity. You can see the size of the caches generated in the [caches view of githhub actions UX](https://github.com/ahornby/fizz/actions/caches?query=sort%3Asize-desc), looks like our usage is pretty small so far. More background: Github actions caching decides its own compression format (currently it prefers zstd), but they are free to change that in future, hence no .zstd suffix or similar on the cache keys. Github actions caches from main are used from feature branches but not vice versa, hence a PR can't pollute cache for the trunk build. This allows us to benefit from caching on main and PRs where dependency versions match. The final item being built, and dependencies inside the same repo as the final are not cached. A different mechanism would be necessary for those (e.g. using a tool like ccache/sccache or caching cargo/buck2 output). This means that when building EdenFS, sapling could not be cached (its in the same repo), but folly could be as we have a key for it based on the folly-rev.txt file. When there is a cache hit the build step is skipped using extension of the conditionals introduced in previous diff D67839708 Reviewed By: bigfootjon Differential Revision: D67839730
|
This pull request was exported from Phabricator. Differential Revision: D67839730 |
ahornby
added a commit
to ahornby/proxygen
that referenced
this pull request
Jan 6, 2025
Summary: X-link: facebook/sapling#1005 Speed up github builds with github actions CI caching based on the cache-key mechanism already present in getdeps. Cached fizz windows build completed in 22% of original time, linux build in 54% of the original time, and mac in 72% of original time. This saves a lot of waiting around for PR builds on OSS changes as windows signal is usually the lagging one. Speed ups will vary across the different Meta OSS projects. Windows benefits most from caching dependencies as we don't use system packages there at all, linux next as its default runner is slower than mac, and then finaly mac (the strongest hardware of the default github runners). Github allows [up to 10GB cache per repo](https://github.blog/changelog/2021-11-23-github-actions-cache-size-is-now-increased-to-10gb-per-repository/), expiring data over capacity. You can see the size of the caches generated in the [caches view of githhub actions UX](https://github.com/ahornby/fizz/actions/caches?query=sort%3Asize-desc), looks like our usage is pretty small so far. More background: Github actions caching decides its own compression format (currently it prefers zstd), but they are free to change that in future, hence no .zstd suffix or similar on the cache keys. Github actions caches from main are used from feature branches but not vice versa, hence a PR can't pollute cache for the trunk build. This allows us to benefit from caching on main and PRs where dependency versions match. The final item being built, and dependencies inside the same repo as the final are not cached. A different mechanism would be necessary for those (e.g. using a tool like ccache/sccache or caching cargo/buck2 output). This means that when building EdenFS, sapling could not be cached (its in the same repo), but folly could be as we have a key for it based on the folly-rev.txt file. When there is a cache hit the build step is skipped using extension of the conditionals introduced in previous diff D67839708 Reviewed By: bigfootjon Differential Revision: D67839730
facebook-github-bot
pushed a commit
to facebook/wangle
that referenced
this pull request
Jan 6, 2025
Summary: X-link: facebook/sapling#1005 Speed up github builds with github actions CI caching based on the cache-key mechanism already present in getdeps. Cached fizz windows build completed in 22% of original time, linux build in 54% of the original time, and mac in 72% of original time. This saves a lot of waiting around for PR builds on OSS changes as windows signal is usually the lagging one. Speed ups will vary across the different Meta OSS projects. Windows benefits most from caching dependencies as we don't use system packages there at all, linux next as its default runner is slower than mac, and then finaly mac (the strongest hardware of the default github runners). Github allows [up to 10GB cache per repo](https://github.blog/changelog/2021-11-23-github-actions-cache-size-is-now-increased-to-10gb-per-repository/), expiring data over capacity. You can see the size of the caches generated in the [caches view of githhub actions UX](https://github.com/ahornby/fizz/actions/caches?query=sort%3Asize-desc), looks like our usage is pretty small so far. More background: Github actions caching decides its own compression format (currently it prefers zstd), but they are free to change that in future, hence no .zstd suffix or similar on the cache keys. Github actions caches from main are used from feature branches but not vice versa, hence a PR can't pollute cache for the trunk build. This allows us to benefit from caching on main and PRs where dependency versions match. The final item being built, and dependencies inside the same repo as the final are not cached. A different mechanism would be necessary for those (e.g. using a tool like ccache/sccache or caching cargo/buck2 output). This means that when building EdenFS, sapling could not be cached (its in the same repo), but folly could be as we have a key for it based on the folly-rev.txt file. When there is a cache hit the build step is skipped using extension of the conditionals introduced in previous diff D67839708 Reviewed By: bigfootjon Differential Revision: D67839730 fbshipit-source-id: c384a216eb27ccd3f816e3c31b167232bda571a6
facebook-github-bot
pushed a commit
to facebook/fboss
that referenced
this pull request
Jan 6, 2025
Summary: X-link: facebook/sapling#1005 Speed up github builds with github actions CI caching based on the cache-key mechanism already present in getdeps. Cached fizz windows build completed in 22% of original time, linux build in 54% of the original time, and mac in 72% of original time. This saves a lot of waiting around for PR builds on OSS changes as windows signal is usually the lagging one. Speed ups will vary across the different Meta OSS projects. Windows benefits most from caching dependencies as we don't use system packages there at all, linux next as its default runner is slower than mac, and then finaly mac (the strongest hardware of the default github runners). Github allows [up to 10GB cache per repo](https://github.blog/changelog/2021-11-23-github-actions-cache-size-is-now-increased-to-10gb-per-repository/), expiring data over capacity. You can see the size of the caches generated in the [caches view of githhub actions UX](https://github.com/ahornby/fizz/actions/caches?query=sort%3Asize-desc), looks like our usage is pretty small so far. More background: Github actions caching decides its own compression format (currently it prefers zstd), but they are free to change that in future, hence no .zstd suffix or similar on the cache keys. Github actions caches from main are used from feature branches but not vice versa, hence a PR can't pollute cache for the trunk build. This allows us to benefit from caching on main and PRs where dependency versions match. The final item being built, and dependencies inside the same repo as the final are not cached. A different mechanism would be necessary for those (e.g. using a tool like ccache/sccache or caching cargo/buck2 output). This means that when building EdenFS, sapling could not be cached (its in the same repo), but folly could be as we have a key for it based on the folly-rev.txt file. When there is a cache hit the build step is skipped using extension of the conditionals introduced in previous diff D67839708 Reviewed By: bigfootjon Differential Revision: D67839730 fbshipit-source-id: c384a216eb27ccd3f816e3c31b167232bda571a6
facebook-github-bot
pushed a commit
to facebook/mvfst
that referenced
this pull request
Jan 6, 2025
Summary: X-link: facebook/sapling#1005 Speed up github builds with github actions CI caching based on the cache-key mechanism already present in getdeps. Cached fizz windows build completed in 22% of original time, linux build in 54% of the original time, and mac in 72% of original time. This saves a lot of waiting around for PR builds on OSS changes as windows signal is usually the lagging one. Speed ups will vary across the different Meta OSS projects. Windows benefits most from caching dependencies as we don't use system packages there at all, linux next as its default runner is slower than mac, and then finaly mac (the strongest hardware of the default github runners). Github allows [up to 10GB cache per repo](https://github.blog/changelog/2021-11-23-github-actions-cache-size-is-now-increased-to-10gb-per-repository/), expiring data over capacity. You can see the size of the caches generated in the [caches view of githhub actions UX](https://github.com/ahornby/fizz/actions/caches?query=sort%3Asize-desc), looks like our usage is pretty small so far. More background: Github actions caching decides its own compression format (currently it prefers zstd), but they are free to change that in future, hence no .zstd suffix or similar on the cache keys. Github actions caches from main are used from feature branches but not vice versa, hence a PR can't pollute cache for the trunk build. This allows us to benefit from caching on main and PRs where dependency versions match. The final item being built, and dependencies inside the same repo as the final are not cached. A different mechanism would be necessary for those (e.g. using a tool like ccache/sccache or caching cargo/buck2 output). This means that when building EdenFS, sapling could not be cached (its in the same repo), but folly could be as we have a key for it based on the folly-rev.txt file. When there is a cache hit the build step is skipped using extension of the conditionals introduced in previous diff D67839708 Reviewed By: bigfootjon Differential Revision: D67839730 fbshipit-source-id: c384a216eb27ccd3f816e3c31b167232bda571a6
facebook-github-bot
pushed a commit
to facebook/openr
that referenced
this pull request
Jan 6, 2025
Summary: X-link: facebook/sapling#1005 Speed up github builds with github actions CI caching based on the cache-key mechanism already present in getdeps. Cached fizz windows build completed in 22% of original time, linux build in 54% of the original time, and mac in 72% of original time. This saves a lot of waiting around for PR builds on OSS changes as windows signal is usually the lagging one. Speed ups will vary across the different Meta OSS projects. Windows benefits most from caching dependencies as we don't use system packages there at all, linux next as its default runner is slower than mac, and then finaly mac (the strongest hardware of the default github runners). Github allows [up to 10GB cache per repo](https://github.blog/changelog/2021-11-23-github-actions-cache-size-is-now-increased-to-10gb-per-repository/), expiring data over capacity. You can see the size of the caches generated in the [caches view of githhub actions UX](https://github.com/ahornby/fizz/actions/caches?query=sort%3Asize-desc), looks like our usage is pretty small so far. More background: Github actions caching decides its own compression format (currently it prefers zstd), but they are free to change that in future, hence no .zstd suffix or similar on the cache keys. Github actions caches from main are used from feature branches but not vice versa, hence a PR can't pollute cache for the trunk build. This allows us to benefit from caching on main and PRs where dependency versions match. The final item being built, and dependencies inside the same repo as the final are not cached. A different mechanism would be necessary for those (e.g. using a tool like ccache/sccache or caching cargo/buck2 output). This means that when building EdenFS, sapling could not be cached (its in the same repo), but folly could be as we have a key for it based on the folly-rev.txt file. When there is a cache hit the build step is skipped using extension of the conditionals introduced in previous diff D67839708 Reviewed By: bigfootjon Differential Revision: D67839730 fbshipit-source-id: c384a216eb27ccd3f816e3c31b167232bda571a6
facebook-github-bot
pushed a commit
to facebookincubator/katran
that referenced
this pull request
Jan 6, 2025
Summary: X-link: facebook/sapling#1005 Speed up github builds with github actions CI caching based on the cache-key mechanism already present in getdeps. Cached fizz windows build completed in 22% of original time, linux build in 54% of the original time, and mac in 72% of original time. This saves a lot of waiting around for PR builds on OSS changes as windows signal is usually the lagging one. Speed ups will vary across the different Meta OSS projects. Windows benefits most from caching dependencies as we don't use system packages there at all, linux next as its default runner is slower than mac, and then finaly mac (the strongest hardware of the default github runners). Github allows [up to 10GB cache per repo](https://github.blog/changelog/2021-11-23-github-actions-cache-size-is-now-increased-to-10gb-per-repository/), expiring data over capacity. You can see the size of the caches generated in the [caches view of githhub actions UX](https://github.com/ahornby/fizz/actions/caches?query=sort%3Asize-desc), looks like our usage is pretty small so far. More background: Github actions caching decides its own compression format (currently it prefers zstd), but they are free to change that in future, hence no .zstd suffix or similar on the cache keys. Github actions caches from main are used from feature branches but not vice versa, hence a PR can't pollute cache for the trunk build. This allows us to benefit from caching on main and PRs where dependency versions match. The final item being built, and dependencies inside the same repo as the final are not cached. A different mechanism would be necessary for those (e.g. using a tool like ccache/sccache or caching cargo/buck2 output). This means that when building EdenFS, sapling could not be cached (its in the same repo), but folly could be as we have a key for it based on the folly-rev.txt file. When there is a cache hit the build step is skipped using extension of the conditionals introduced in previous diff D67839708 Reviewed By: bigfootjon Differential Revision: D67839730 fbshipit-source-id: c384a216eb27ccd3f816e3c31b167232bda571a6
facebook-github-bot
pushed a commit
to facebookexperimental/rust-shed
that referenced
this pull request
Jan 6, 2025
Summary: X-link: facebook/sapling#1005 Speed up github builds with github actions CI caching based on the cache-key mechanism already present in getdeps. Cached fizz windows build completed in 22% of original time, linux build in 54% of the original time, and mac in 72% of original time. This saves a lot of waiting around for PR builds on OSS changes as windows signal is usually the lagging one. Speed ups will vary across the different Meta OSS projects. Windows benefits most from caching dependencies as we don't use system packages there at all, linux next as its default runner is slower than mac, and then finaly mac (the strongest hardware of the default github runners). Github allows [up to 10GB cache per repo](https://github.blog/changelog/2021-11-23-github-actions-cache-size-is-now-increased-to-10gb-per-repository/), expiring data over capacity. You can see the size of the caches generated in the [caches view of githhub actions UX](https://github.com/ahornby/fizz/actions/caches?query=sort%3Asize-desc), looks like our usage is pretty small so far. More background: Github actions caching decides its own compression format (currently it prefers zstd), but they are free to change that in future, hence no .zstd suffix or similar on the cache keys. Github actions caches from main are used from feature branches but not vice versa, hence a PR can't pollute cache for the trunk build. This allows us to benefit from caching on main and PRs where dependency versions match. The final item being built, and dependencies inside the same repo as the final are not cached. A different mechanism would be necessary for those (e.g. using a tool like ccache/sccache or caching cargo/buck2 output). This means that when building EdenFS, sapling could not be cached (its in the same repo), but folly could be as we have a key for it based on the folly-rev.txt file. When there is a cache hit the build step is skipped using extension of the conditionals introduced in previous diff D67839708 Reviewed By: bigfootjon Differential Revision: D67839730 fbshipit-source-id: c384a216eb27ccd3f816e3c31b167232bda571a6
facebook-github-bot
pushed a commit
to facebook/CacheLib
that referenced
this pull request
Jan 6, 2025
Summary: X-link: facebook/sapling#1005 Speed up github builds with github actions CI caching based on the cache-key mechanism already present in getdeps. Cached fizz windows build completed in 22% of original time, linux build in 54% of the original time, and mac in 72% of original time. This saves a lot of waiting around for PR builds on OSS changes as windows signal is usually the lagging one. Speed ups will vary across the different Meta OSS projects. Windows benefits most from caching dependencies as we don't use system packages there at all, linux next as its default runner is slower than mac, and then finaly mac (the strongest hardware of the default github runners). Github allows [up to 10GB cache per repo](https://github.blog/changelog/2021-11-23-github-actions-cache-size-is-now-increased-to-10gb-per-repository/), expiring data over capacity. You can see the size of the caches generated in the [caches view of githhub actions UX](https://github.com/ahornby/fizz/actions/caches?query=sort%3Asize-desc), looks like our usage is pretty small so far. More background: Github actions caching decides its own compression format (currently it prefers zstd), but they are free to change that in future, hence no .zstd suffix or similar on the cache keys. Github actions caches from main are used from feature branches but not vice versa, hence a PR can't pollute cache for the trunk build. This allows us to benefit from caching on main and PRs where dependency versions match. The final item being built, and dependencies inside the same repo as the final are not cached. A different mechanism would be necessary for those (e.g. using a tool like ccache/sccache or caching cargo/buck2 output). This means that when building EdenFS, sapling could not be cached (its in the same repo), but folly could be as we have a key for it based on the folly-rev.txt file. When there is a cache hit the build step is skipped using extension of the conditionals introduced in previous diff D67839708 Reviewed By: bigfootjon Differential Revision: D67839730 fbshipit-source-id: c384a216eb27ccd3f816e3c31b167232bda571a6
facebook-github-bot
pushed a commit
to facebook/folly
that referenced
this pull request
Jan 6, 2025
Summary: X-link: facebook/sapling#1005 Speed up github builds with github actions CI caching based on the cache-key mechanism already present in getdeps. Cached fizz windows build completed in 22% of original time, linux build in 54% of the original time, and mac in 72% of original time. This saves a lot of waiting around for PR builds on OSS changes as windows signal is usually the lagging one. Speed ups will vary across the different Meta OSS projects. Windows benefits most from caching dependencies as we don't use system packages there at all, linux next as its default runner is slower than mac, and then finaly mac (the strongest hardware of the default github runners). Github allows [up to 10GB cache per repo](https://github.blog/changelog/2021-11-23-github-actions-cache-size-is-now-increased-to-10gb-per-repository/), expiring data over capacity. You can see the size of the caches generated in the [caches view of githhub actions UX](https://github.com/ahornby/fizz/actions/caches?query=sort%3Asize-desc), looks like our usage is pretty small so far. More background: Github actions caching decides its own compression format (currently it prefers zstd), but they are free to change that in future, hence no .zstd suffix or similar on the cache keys. Github actions caches from main are used from feature branches but not vice versa, hence a PR can't pollute cache for the trunk build. This allows us to benefit from caching on main and PRs where dependency versions match. The final item being built, and dependencies inside the same repo as the final are not cached. A different mechanism would be necessary for those (e.g. using a tool like ccache/sccache or caching cargo/buck2 output). This means that when building EdenFS, sapling could not be cached (its in the same repo), but folly could be as we have a key for it based on the folly-rev.txt file. When there is a cache hit the build step is skipped using extension of the conditionals introduced in previous diff D67839708 Reviewed By: bigfootjon Differential Revision: D67839730 fbshipit-source-id: c384a216eb27ccd3f816e3c31b167232bda571a6
facebook-github-bot
pushed a commit
to facebook/fb303
that referenced
this pull request
Jan 6, 2025
Summary: X-link: facebook/sapling#1005 Speed up github builds with github actions CI caching based on the cache-key mechanism already present in getdeps. Cached fizz windows build completed in 22% of original time, linux build in 54% of the original time, and mac in 72% of original time. This saves a lot of waiting around for PR builds on OSS changes as windows signal is usually the lagging one. Speed ups will vary across the different Meta OSS projects. Windows benefits most from caching dependencies as we don't use system packages there at all, linux next as its default runner is slower than mac, and then finaly mac (the strongest hardware of the default github runners). Github allows [up to 10GB cache per repo](https://github.blog/changelog/2021-11-23-github-actions-cache-size-is-now-increased-to-10gb-per-repository/), expiring data over capacity. You can see the size of the caches generated in the [caches view of githhub actions UX](https://github.com/ahornby/fizz/actions/caches?query=sort%3Asize-desc), looks like our usage is pretty small so far. More background: Github actions caching decides its own compression format (currently it prefers zstd), but they are free to change that in future, hence no .zstd suffix or similar on the cache keys. Github actions caches from main are used from feature branches but not vice versa, hence a PR can't pollute cache for the trunk build. This allows us to benefit from caching on main and PRs where dependency versions match. The final item being built, and dependencies inside the same repo as the final are not cached. A different mechanism would be necessary for those (e.g. using a tool like ccache/sccache or caching cargo/buck2 output). This means that when building EdenFS, sapling could not be cached (its in the same repo), but folly could be as we have a key for it based on the folly-rev.txt file. When there is a cache hit the build step is skipped using extension of the conditionals introduced in previous diff D67839708 Reviewed By: bigfootjon Differential Revision: D67839730 fbshipit-source-id: c384a216eb27ccd3f816e3c31b167232bda571a6
facebook-github-bot
pushed a commit
to facebook/proxygen
that referenced
this pull request
Jan 6, 2025
Summary: X-link: facebook/sapling#1005 Speed up github builds with github actions CI caching based on the cache-key mechanism already present in getdeps. Cached fizz windows build completed in 22% of original time, linux build in 54% of the original time, and mac in 72% of original time. This saves a lot of waiting around for PR builds on OSS changes as windows signal is usually the lagging one. Speed ups will vary across the different Meta OSS projects. Windows benefits most from caching dependencies as we don't use system packages there at all, linux next as its default runner is slower than mac, and then finaly mac (the strongest hardware of the default github runners). Github allows [up to 10GB cache per repo](https://github.blog/changelog/2021-11-23-github-actions-cache-size-is-now-increased-to-10gb-per-repository/), expiring data over capacity. You can see the size of the caches generated in the [caches view of githhub actions UX](https://github.com/ahornby/fizz/actions/caches?query=sort%3Asize-desc), looks like our usage is pretty small so far. More background: Github actions caching decides its own compression format (currently it prefers zstd), but they are free to change that in future, hence no .zstd suffix or similar on the cache keys. Github actions caches from main are used from feature branches but not vice versa, hence a PR can't pollute cache for the trunk build. This allows us to benefit from caching on main and PRs where dependency versions match. The final item being built, and dependencies inside the same repo as the final are not cached. A different mechanism would be necessary for those (e.g. using a tool like ccache/sccache or caching cargo/buck2 output). This means that when building EdenFS, sapling could not be cached (its in the same repo), but folly could be as we have a key for it based on the folly-rev.txt file. When there is a cache hit the build step is skipped using extension of the conditionals introduced in previous diff D67839708 Reviewed By: bigfootjon Differential Revision: D67839730 fbshipit-source-id: c384a216eb27ccd3f816e3c31b167232bda571a6
facebook-github-bot
pushed a commit
to facebookexperimental/moxygen
that referenced
this pull request
Jan 6, 2025
Summary: X-link: facebook/sapling#1005 Speed up github builds with github actions CI caching based on the cache-key mechanism already present in getdeps. Cached fizz windows build completed in 22% of original time, linux build in 54% of the original time, and mac in 72% of original time. This saves a lot of waiting around for PR builds on OSS changes as windows signal is usually the lagging one. Speed ups will vary across the different Meta OSS projects. Windows benefits most from caching dependencies as we don't use system packages there at all, linux next as its default runner is slower than mac, and then finaly mac (the strongest hardware of the default github runners). Github allows [up to 10GB cache per repo](https://github.blog/changelog/2021-11-23-github-actions-cache-size-is-now-increased-to-10gb-per-repository/), expiring data over capacity. You can see the size of the caches generated in the [caches view of githhub actions UX](https://github.com/ahornby/fizz/actions/caches?query=sort%3Asize-desc), looks like our usage is pretty small so far. More background: Github actions caching decides its own compression format (currently it prefers zstd), but they are free to change that in future, hence no .zstd suffix or similar on the cache keys. Github actions caches from main are used from feature branches but not vice versa, hence a PR can't pollute cache for the trunk build. This allows us to benefit from caching on main and PRs where dependency versions match. The final item being built, and dependencies inside the same repo as the final are not cached. A different mechanism would be necessary for those (e.g. using a tool like ccache/sccache or caching cargo/buck2 output). This means that when building EdenFS, sapling could not be cached (its in the same repo), but folly could be as we have a key for it based on the folly-rev.txt file. When there is a cache hit the build step is skipped using extension of the conditionals introduced in previous diff D67839708 Reviewed By: bigfootjon Differential Revision: D67839730 fbshipit-source-id: c384a216eb27ccd3f816e3c31b167232bda571a6
|
This pull request has been merged in e6d931c. |
facebook-github-bot
pushed a commit
to facebookincubator/hsthrift
that referenced
this pull request
Jan 6, 2025
Summary: X-link: facebook/sapling#1005 Speed up github builds with github actions CI caching based on the cache-key mechanism already present in getdeps. Cached fizz windows build completed in 22% of original time, linux build in 54% of the original time, and mac in 72% of original time. This saves a lot of waiting around for PR builds on OSS changes as windows signal is usually the lagging one. Speed ups will vary across the different Meta OSS projects. Windows benefits most from caching dependencies as we don't use system packages there at all, linux next as its default runner is slower than mac, and then finaly mac (the strongest hardware of the default github runners). Github allows [up to 10GB cache per repo](https://github.blog/changelog/2021-11-23-github-actions-cache-size-is-now-increased-to-10gb-per-repository/), expiring data over capacity. You can see the size of the caches generated in the [caches view of githhub actions UX](https://github.com/ahornby/fizz/actions/caches?query=sort%3Asize-desc), looks like our usage is pretty small so far. More background: Github actions caching decides its own compression format (currently it prefers zstd), but they are free to change that in future, hence no .zstd suffix or similar on the cache keys. Github actions caches from main are used from feature branches but not vice versa, hence a PR can't pollute cache for the trunk build. This allows us to benefit from caching on main and PRs where dependency versions match. The final item being built, and dependencies inside the same repo as the final are not cached. A different mechanism would be necessary for those (e.g. using a tool like ccache/sccache or caching cargo/buck2 output). This means that when building EdenFS, sapling could not be cached (its in the same repo), but folly could be as we have a key for it based on the folly-rev.txt file. When there is a cache hit the build step is skipped using extension of the conditionals introduced in previous diff D67839708 Reviewed By: bigfootjon Differential Revision: D67839730 fbshipit-source-id: c384a216eb27ccd3f816e3c31b167232bda571a6
facebook-github-bot
pushed a commit
to facebook/watchman
that referenced
this pull request
Jan 6, 2025
Summary: X-link: facebook/sapling#1005 Speed up github builds with github actions CI caching based on the cache-key mechanism already present in getdeps. Cached fizz windows build completed in 22% of original time, linux build in 54% of the original time, and mac in 72% of original time. This saves a lot of waiting around for PR builds on OSS changes as windows signal is usually the lagging one. Speed ups will vary across the different Meta OSS projects. Windows benefits most from caching dependencies as we don't use system packages there at all, linux next as its default runner is slower than mac, and then finaly mac (the strongest hardware of the default github runners). Github allows [up to 10GB cache per repo](https://github.blog/changelog/2021-11-23-github-actions-cache-size-is-now-increased-to-10gb-per-repository/), expiring data over capacity. You can see the size of the caches generated in the [caches view of githhub actions UX](https://github.com/ahornby/fizz/actions/caches?query=sort%3Asize-desc), looks like our usage is pretty small so far. More background: Github actions caching decides its own compression format (currently it prefers zstd), but they are free to change that in future, hence no .zstd suffix or similar on the cache keys. Github actions caches from main are used from feature branches but not vice versa, hence a PR can't pollute cache for the trunk build. This allows us to benefit from caching on main and PRs where dependency versions match. The final item being built, and dependencies inside the same repo as the final are not cached. A different mechanism would be necessary for those (e.g. using a tool like ccache/sccache or caching cargo/buck2 output). This means that when building EdenFS, sapling could not be cached (its in the same repo), but folly could be as we have a key for it based on the folly-rev.txt file. When there is a cache hit the build step is skipped using extension of the conditionals introduced in previous diff D67839708 Reviewed By: bigfootjon Differential Revision: D67839730 fbshipit-source-id: c384a216eb27ccd3f816e3c31b167232bda571a6
facebook-github-bot
pushed a commit
to facebookincubator/fizz
that referenced
this pull request
Jan 6, 2025
Summary: X-link: facebook/sapling#1005 Speed up github builds with github actions CI caching based on the cache-key mechanism already present in getdeps. Cached fizz windows build completed in 22% of original time, linux build in 54% of the original time, and mac in 72% of original time. This saves a lot of waiting around for PR builds on OSS changes as windows signal is usually the lagging one. Speed ups will vary across the different Meta OSS projects. Windows benefits most from caching dependencies as we don't use system packages there at all, linux next as its default runner is slower than mac, and then finaly mac (the strongest hardware of the default github runners). Github allows [up to 10GB cache per repo](https://github.blog/changelog/2021-11-23-github-actions-cache-size-is-now-increased-to-10gb-per-repository/), expiring data over capacity. You can see the size of the caches generated in the [caches view of githhub actions UX](https://github.com/ahornby/fizz/actions/caches?query=sort%3Asize-desc), looks like our usage is pretty small so far. More background: Github actions caching decides its own compression format (currently it prefers zstd), but they are free to change that in future, hence no .zstd suffix or similar on the cache keys. Github actions caches from main are used from feature branches but not vice versa, hence a PR can't pollute cache for the trunk build. This allows us to benefit from caching on main and PRs where dependency versions match. The final item being built, and dependencies inside the same repo as the final are not cached. A different mechanism would be necessary for those (e.g. using a tool like ccache/sccache or caching cargo/buck2 output). This means that when building EdenFS, sapling could not be cached (its in the same repo), but folly could be as we have a key for it based on the folly-rev.txt file. When there is a cache hit the build step is skipped using extension of the conditionals introduced in previous diff D67839708 Reviewed By: bigfootjon Differential Revision: D67839730 fbshipit-source-id: c384a216eb27ccd3f816e3c31b167232bda571a6
facebook-github-bot
pushed a commit
to facebookexperimental/edencommon
that referenced
this pull request
Jan 6, 2025
Summary: X-link: facebook/sapling#1005 Speed up github builds with github actions CI caching based on the cache-key mechanism already present in getdeps. Cached fizz windows build completed in 22% of original time, linux build in 54% of the original time, and mac in 72% of original time. This saves a lot of waiting around for PR builds on OSS changes as windows signal is usually the lagging one. Speed ups will vary across the different Meta OSS projects. Windows benefits most from caching dependencies as we don't use system packages there at all, linux next as its default runner is slower than mac, and then finaly mac (the strongest hardware of the default github runners). Github allows [up to 10GB cache per repo](https://github.blog/changelog/2021-11-23-github-actions-cache-size-is-now-increased-to-10gb-per-repository/), expiring data over capacity. You can see the size of the caches generated in the [caches view of githhub actions UX](https://github.com/ahornby/fizz/actions/caches?query=sort%3Asize-desc), looks like our usage is pretty small so far. More background: Github actions caching decides its own compression format (currently it prefers zstd), but they are free to change that in future, hence no .zstd suffix or similar on the cache keys. Github actions caches from main are used from feature branches but not vice versa, hence a PR can't pollute cache for the trunk build. This allows us to benefit from caching on main and PRs where dependency versions match. The final item being built, and dependencies inside the same repo as the final are not cached. A different mechanism would be necessary for those (e.g. using a tool like ccache/sccache or caching cargo/buck2 output). This means that when building EdenFS, sapling could not be cached (its in the same repo), but folly could be as we have a key for it based on the folly-rev.txt file. When there is a cache hit the build step is skipped using extension of the conditionals introduced in previous diff D67839708 Reviewed By: bigfootjon Differential Revision: D67839730 fbshipit-source-id: c384a216eb27ccd3f816e3c31b167232bda571a6
facebook-github-bot
pushed a commit
to facebook/fbthrift
that referenced
this pull request
Jan 6, 2025
Summary: X-link: facebook/sapling#1005 Speed up github builds with github actions CI caching based on the cache-key mechanism already present in getdeps. Cached fizz windows build completed in 22% of original time, linux build in 54% of the original time, and mac in 72% of original time. This saves a lot of waiting around for PR builds on OSS changes as windows signal is usually the lagging one. Speed ups will vary across the different Meta OSS projects. Windows benefits most from caching dependencies as we don't use system packages there at all, linux next as its default runner is slower than mac, and then finaly mac (the strongest hardware of the default github runners). Github allows [up to 10GB cache per repo](https://github.blog/changelog/2021-11-23-github-actions-cache-size-is-now-increased-to-10gb-per-repository/), expiring data over capacity. You can see the size of the caches generated in the [caches view of githhub actions UX](https://github.com/ahornby/fizz/actions/caches?query=sort%3Asize-desc), looks like our usage is pretty small so far. More background: Github actions caching decides its own compression format (currently it prefers zstd), but they are free to change that in future, hence no .zstd suffix or similar on the cache keys. Github actions caches from main are used from feature branches but not vice versa, hence a PR can't pollute cache for the trunk build. This allows us to benefit from caching on main and PRs where dependency versions match. The final item being built, and dependencies inside the same repo as the final are not cached. A different mechanism would be necessary for those (e.g. using a tool like ccache/sccache or caching cargo/buck2 output). This means that when building EdenFS, sapling could not be cached (its in the same repo), but folly could be as we have a key for it based on the folly-rev.txt file. When there is a cache hit the build step is skipped using extension of the conditionals introduced in previous diff D67839708 Reviewed By: bigfootjon Differential Revision: D67839730 fbshipit-source-id: c384a216eb27ccd3f816e3c31b167232bda571a6
facebook-github-bot
pushed a commit
to facebook/hhvm
that referenced
this pull request
Jan 6, 2025
Summary: X-link: facebook/sapling#1005 Speed up github builds with github actions CI caching based on the cache-key mechanism already present in getdeps. Cached fizz windows build completed in 22% of original time, linux build in 54% of the original time, and mac in 72% of original time. This saves a lot of waiting around for PR builds on OSS changes as windows signal is usually the lagging one. Speed ups will vary across the different Meta OSS projects. Windows benefits most from caching dependencies as we don't use system packages there at all, linux next as its default runner is slower than mac, and then finaly mac (the strongest hardware of the default github runners). Github allows [up to 10GB cache per repo](https://github.blog/changelog/2021-11-23-github-actions-cache-size-is-now-increased-to-10gb-per-repository/), expiring data over capacity. You can see the size of the caches generated in the [caches view of githhub actions UX](https://github.com/ahornby/fizz/actions/caches?query=sort%3Asize-desc), looks like our usage is pretty small so far. More background: Github actions caching decides its own compression format (currently it prefers zstd), but they are free to change that in future, hence no .zstd suffix or similar on the cache keys. Github actions caches from main are used from feature branches but not vice versa, hence a PR can't pollute cache for the trunk build. This allows us to benefit from caching on main and PRs where dependency versions match. The final item being built, and dependencies inside the same repo as the final are not cached. A different mechanism would be necessary for those (e.g. using a tool like ccache/sccache or caching cargo/buck2 output). This means that when building EdenFS, sapling could not be cached (its in the same repo), but folly could be as we have a key for it based on the folly-rev.txt file. When there is a cache hit the build step is skipped using extension of the conditionals introduced in previous diff D67839708 Reviewed By: bigfootjon Differential Revision: D67839730 fbshipit-source-id: c384a216eb27ccd3f816e3c31b167232bda571a6
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Summary:
Speed up github builds with github actions CI caching based on the cache-key mechanism already present in getdeps.
Cached fizz windows build completed in 22% of original time, linux build in 54% of the original time, and mac in 72% of original time. This saves a lot of waiting around for PR builds on OSS changes as windows signal is usually the lagging one. Speed ups will vary across the different Meta OSS projects.
Windows benefits most from caching dependencies as we don't use system packages there at all, linux next as its default runner is slower than mac, and then finaly mac (the strongest hardware of the default github runners).
Github allows up to 10GB cache per repo, expiring data over capacity. You can see the size of the caches generated in the caches view of githhub actions UX, looks like our usage is pretty small so far.
More background:
Github actions caching decides its own compression format (currently it prefers zstd), but they are free to change that in future, hence no .zstd suffix or similar on the cache keys.
Github actions caches from main are used from feature branches but not vice versa, hence a PR can't pollute cache for the trunk build. This allows us to benefit from caching on main and PRs where dependency versions match.
The final item being built, and dependencies inside the same repo as the final are not cached. A different mechanism would be necessary for those (e.g. using a tool like ccache/sccache or caching cargo/buck2 output). This means that when building EdenFS, sapling could not be cached (its in the same repo), but folly could be as we have a key for it based on the folly-rev.txt file.
When there is a cache hit the build step is skipped using extension of the conditionals introduced in previous diff D67839708
Reviewed By: bigfootjon
Differential Revision: D67839730