Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add a GitHub Actions Cache remote cache backend #19831

Merged
merged 7 commits into from
Sep 27, 2023
Merged

Add a GitHub Actions Cache remote cache backend #19831

Show file tree
Hide file tree
Changes from 1 commit
Commits
Show all changes
7 commits
Select commit Hold shift + click to select a range
eef4752
Squash AlreadyExists errors
huonw Sep 23, 2023
6549a39
Move features to Cargo.toml
huonw Sep 23, 2023
02c457e
Use OpenDAL as git dep for now, due to unreleased fixes
huonw Sep 23, 2023
f30486f
Implement github actions cache
huonw Sep 14, 2023
f3adeca
Basic cut at documentation
huonw Sep 26, 2023
b6dbc0c
doc typo
huonw Sep 27, 2023
984bc75
TODO -> a note
huonw Sep 27, 2023
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
35 changes: 8 additions & 27 deletions src/rust/engine/Cargo.lock
View file
Open in desktop

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

5 changes: 4 additions & 1 deletion src/rust/engine/Cargo.toml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -254,7 +254,10 @@ notify = { git = "https://github.com/pantsbuild/notify", rev = "276af0f3c5f300bf
num_cpus = "1"
num_enum = "0.5"
once_cell = "1.18"
opendal = { version = "0.39.0", default-features = false, features = [
# TODO: this is waiting for several changes to be released (likely in 0.41):
# https://github.com/apache/incubator-opendal/pull/3163
# https://github.com/apache/incubator-opendal/pull/3177
opendal = { git = "https://github.com/apache/incubator-opendal", rev = "97bcef60eb0b515bd2442ab5b671080766fa35eb", default-features = false, features = [
"services-memory",
"services-fs",
"services-ghac",
Expand Down
7 changes: 1 addition & 6 deletions src/rust/engine/fs/store/src/remote/base_opendal.rs
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -178,12 +178,7 @@ impl ByteStoreProvider for Provider {

let path = self.path(digest.hash);

let mut writer = match self
.operator
.writer_with(&path)
.content_length(digest.size_bytes as u64)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Was this a workaround that is no longer necessary post apache/opendal#3163, or did this end up in the wrong commit?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Coincidence, unrelated to the fixes I made: the content_length method was removed in 0.40.0 (apache/opendal#3044), and the upgrade to a recent HEAD here goes past that.

.await
{
let mut writer = match self.operator.writer(&path).await {
Ok(writer) => writer,
// The item already exists, i.e. these bytes have already been stored. For example,
// concurrent executions that are caching the same bytes. This makes the assumption that
Expand Down