Cache operations will hang when using type=gha for reletively low throughputs.

[bendavies]

Behaviour

Cache operations using type=gha will start hanging after a relavatively low number of cache reads/writes.

We are a small team (5 devs) and are hitting these limits constantly on our main repository. Not exactly high throughput.
I'm unsure if using docker/build-push-action with cache type=gha is even viable, even for small teams.

Steps to reproduce this issue

1. Reproducing behaviour exibited here: https://github.com/bendavies/docker-build-cache-limits
2. First build: https://github.com/bendavies/docker-build-cache-limits/actions/runs/2155495185/attempts/1
3. Second build: https://github.com/bendavies/docker-build-cache-limits/actions/runs/2155495185/attempts/2

The second build should be lightening quick as the cache is fully primed, but takes forever because I think we have exhausted some limits of the GitHub Cache API, and are being throttled.

Cache reads and writes are affected by this throttling.

I'm not sure where this user got the figures of "150 cache requests in 5 minutes", but it seems about right:
moby/buildkit#2276 (comment)

Expected behaviour

I don't know, but the current behaviour of hanging is not great. there is no feedback as to what is hanging resulting in the user just being confused.

Can cache reads/writes be batched or can they only read/write a single item at time?

If this is not solvable with more efficient caching, a user configured timeout might be better, as waiting 10+ minutes for the cache to start responding is far worse than having a 1 minute timeout.

Actual behaviour

Hangs for an undetermined amount of time.

Configuration

https://github.com/bendavies/docker-build-cache-limits/

Logs

build-1-logs.zip
build-2-logs.zip

[bendavies]

reopening here as requested in the docs