Support regex image tag filtering

[hiddeco]

At the moment Flux only offers support for wildcard filtering (e.g. prod-*, v1.8.*, etc).

This works perfectly fine if you are in control of how the images are getting tagged, however, if you are using images from an external vendor you will run in to problems pretty fast.

It is a common pattern to build additional images on top of a base image and tag them with a suffix describing the addition (e.g. 1.0 and 1.0-jmx, see for an example https://hub.docker.com/r/datadog/agent/tags/). These images will get rolled out last so Flux will detect them as the eligible new image (as it looks for the last published images that matches the filter requirements).

I briefly discussed this issue some weeks ago with @squaremo on Slack who proposed a semver:^1.8 solution (as an addition to glob:1.8.*). I do however think this solution would be insufficient as semver tagging is as common as using the latest commit hash.

Instead I would like to propose a regex filter option that looks for pattern matches, this is highly configurable and can get as complex (and nasty) as required. Eliminating issues with whatever weird tag the user wants to keep track of.

[rade]

semver tagging is #706. Would you mind explaining in more detail why what is proposed there is insufficient?

One of the advantages of semver is that it provides an order, i.e. you can always determine which is the "best" version. That is not the case for a regexp match.

[hiddeco]

Would you mind explaining in more detail why what is proposed there is insufficient?

Edge cases where images are not published using semver. The Docker ecosystem is already huge - and growing every day. One of the ideas behind Docker images it that they should be reusable. With this comes the problem that everyone has their own tagging strategy and semver is not as common as one may think.

I think this addition would make Flux 'feature complete' for 99% of the edge case scenarios as it allows complex filtering.

One of the advantages of semver is that it provides an order, i.e. you can always determine which is the "best" version. That is not the case for a regexp match.

This is true, but this is also not the case for the current glob filter.

Imagine the following scenario: you have a glob:v1.8.* filter on one of your deployment manifests, the vendor of the image you are using publishes a new v1.8.1 image and shortly after that a v1.8.1-foo image with some additions you are not interested in. Flux will now detect the v1.8.1-foo image as the latest image and deploy it to the cluster.

With a regex you could filter out this -foo addition and tell Flux you are only interested in the newest image that matches your pattern. Keeping track of the version hierarchy here does not matter, you just want to get the latest image.

As an addition @squaremo provides an additional issue that semver would have with this pattern:

mbridgen [4:20 PM]
And the reason semver won't cover that is that it treats -foo as a prerelease, so if you do want the foo variation, you can't express that
(though if that's the only reason, a tweakable semver would be better)

---

Taking all this in consideration: I think #706 should have a higher priority as it is easier for a user to configure and will also solve the scenario described above.

However, for edge cases that require complex filtering - because someone has other ideas or you really enjoy doing exotic stuff with your tags - this should be added to Flux when there is time.

[rndstr]

Imagine the following scenario: you have a glob:v1.8.* filter on one of your deployment manifests, the vendor of the image you are using publishes a new v1.8.1 image and shortly after that a v1.8.1-foo image with some additions you are not interested in. Flux will now detect the v1.8.1-foo image as the latest image and deploy it to the cluster.

With regard to exactly this scenario, semver filtering with semver:~1.8 would do what you wanted: the non-pre-release version (v1.8.1) has precedence over the pre-release version (v1.8.1-foo).