cmd/plume: Add a prune command to clean up developer images #115
Conversation
margamanterola
force-pushed
the
marga-kinvolk/clean-up
branch
from
bb16fc3 to
8391196
Compare
cmd/plume/prune.go
Outdated
| duration := now.Sub(lastModifiedDate) | ||
| daysOld := int(duration.Hours() / 24) | ||
| if daysOld >= days { | ||
| plog.Infof("Old blob %q: %d days old\n", blob.Name, daysOld) |
There was a problem hiding this comment.
Can you add a check for the blob name prefix to be flatcar-linux- and suffix be .vhd? Then it will not delete other things like flatcar-beta-cert-test_02_28_2020_03_08_18.zip.
There was a problem hiding this comment.
Ah, and also check the channel, forgot that in the review (--channel edge currently still enumerates other channels like flatcar-linux-1772.0.0-alpha.vhd).
There was a problem hiding this comment.
Uhm... Ok. I had only tested it with the developer channel, which goes to a different bucket, so it's not relevant. I'll figure out a way of filtering the blobs per channel.
There was a problem hiding this comment.
Alright this was a bit more involved than expected, but it's working now. It checks that the stored metadata filename is the one from the spec, and also that the blob name includes the channel name being processed.
cmd/plume/prune.go
Outdated
| pruneDryRun bool | ||
| cmdPrune = &cobra.Command{ | ||
| Use: "prune [options]", | ||
| Short: "Prune old release images.", |
There was a problem hiding this comment.
I have to specify --version abc because the other commands require it. Can you please document that --version will be ignored?
There was a problem hiding this comment.
and maybe document that it takes --channel into account
There was a problem hiding this comment.
Yeah, I mentioned that in the how to test command (I was using --version none), but I figured that instead of documenting this brokenness, it's better to just override it. I've added a line to always set the version to none.
There was a problem hiding this comment.
I've documented the fact that it requires the channel now.
cmd/plume/prune.go
Outdated
| } | ||
| } | ||
| imageFileName := spec.AWS.Image | ||
| s3ObjectPath := fmt.Sprintf("%s-usr/%s/%s", *image.Architecture, version, strings.TrimSuffix(imageFileName, filepath.Ext(imageFileName))) |
There was a problem hiding this comment.
*image.Architecture returns x86_64 instead of amd64 and needs some postprocessing.
There was a problem hiding this comment.
Ok, I think I've fixed this although I don't think it matters because the S3 bucket is empty (I think whatever gets uploaded there gets deleted once it turns into a snapshot).
margamanterola
force-pushed
the
marga-kinvolk/clean-up
branch
3 times, most recently
from
aa02a1f to
150a689
Compare
When running developer builds, plume uploads prerelease images to both Azure and AWS. This change adds a new "prune" subcommand for plume, to get rid of artifacts uploaded to old builds. It also adds or improves logging messages on functions. This extra logging was needed for debugging while developing the code.
margamanterola
force-pushed
the
marga-kinvolk/clean-up
branch
from
150a689 to
5e00af9
Compare
cmd/plume: Add a prune command to clean up developer images
When running developer builds, plume uploads prerelease images to both Azure and AWS. This change adds a new "prune" subcommand for plume, to get rid of artifacts uploaded to old builds.
It also adds or improves logging messages on functions. This extra logging was needed for debugging while developing the code.
How to use / Testing done
I've already run this for Azure and AWS with days=20, so to test it, it needs to be run with a smaller number or it won't do anything.
Needed for: kinvolk/PROJECT-flatcar-linux#294