Caching fallback for new branches and new tags

[CMCDragonkai]

Specification

Atm, any new feature branch and new tag, doesn't get the benefit of caching. It has to download everything. This is due to our usage of $CI_COMMIT_REF_SLUG.

We could use the fallback cache key, but we have some problems to figure out here:

1. I use $CI_COMMIT_REF_SLUG which is the branch or tag name as the
   cache key. Meaning all jobs share the same cache within the same branch
   or tag name. The end result is that for a given branch staging, the
   final key will be staging-protected.

2. I was under the impression that the windows and macos runners would
   have separate caches, even if they are using the same
   $CI_COMMIT_REF_SLUG. Am I wrong about this? Are the windows and macos
   sharing the same cache that the linux jobs are using as well? If this is
   the case, then indeed it is possible that something might be wrong in
   the cache, as the 3 OSes should not have the same cache data.

3. If I need to separate windows and macos out, do I need to add an
   extra variable to the cache key, and if so, what would you suggest to
   get the runner platform?

4. I recently discovered the CACHE_FALLBACK_KEY which could be used
   for dealing with the fact that new feature branches and tags would start
   off with no cache because nothing has been cached for their
   $CI_COMMIT_REF_SLUG. If I bring this in and make it fallback on
   staging like CACHE_FALLBACK_KEY: "$CI_DEFAULT_BRANCH, I can imagine a
   couple problems here. Firstly there's no -protected suffix appended to
   the fallback key. Secondly nothing is pushing to this cache, I assume
   that it still continues to push to the main cache key. Finally if I do
   set it explicitly to CACHE_FALLBACK_KEY: "staging-protected" doesn't
   this end up allowing unprotected branches & tags to access the cache of
   the staging branch which is a security risk. What is the right way to
   use this, if I only want protected references (new feature branches and
   new tags) to load their cache from staging-protected?

Additional context

• Integrate ccache for caching native compilation js-db#44 - this is about native compilation caching (somewhat related)

Tasks

1. ...
2. ...
3. ...

[CMCDragonkai]

I think this is because $CI_COMMIT_REF_SLUG has the name of the feature branch and tag in it.

For feature branches the cache kicks in as soon as next commits goes to the same branch. Although it would be nice to re-use the cache from staging.

For tags, it's more complicated since each tag would be unique. So some sort of fallback should activate.

If we fallback on staging-protected, it's important that the cache is not a security risk. o nothing should be put into the cache that could be private.

Furthermore we need to ensure that such caches are also different depending on the platform OS.

[CMCDragonkai]

Not relevant anymore as we have switched to GitHub.