Support Time-Based pull_policy in Docker Compose (e.g., once_per_day)

[victorwads]

Description

Description

In large companies, where hundreds of developers work with company-managed images that receive updates multiple times per week, ensuring that everyone is using an up-to-date image is a real challenge.

Currently, developers need to manually remember to run docker compose pull from time to time. Often, they only realize their image is outdated after hours of debugging unexpected issues—just to discover that the problem was already fixed in a newer image.

🚀 Why Not Just Use pull_policy: always?

• Setting pull_policy: always forces a docker pull on every docker compose up, even when there’s nothing new to update.
• This significantly slows down the development workflow, especially in environments with large images.
• Developers end up either:
  • Forgetting to pull images (causing issues).
  • Suffering slow startups every time they run docker compose up.
  • when offline or poor connection, this could take several time until docker give up to pull

🛠 Proposed Solution

Introduce a time-based pull_policy, such as:

services:
  my-service:
    image: company-registry.com/dev-image:latest
    pull_policy: once_per_day

Other possible values:

• daily
• weekly
• custom: (e.g., custom:86400 for every 24 hours)

This would allow docker compose to automatically check if a pull has already happened today. If it has not, it performs the pull. If it has, it proceeds without pulling, ensuring a balance between up-to-date images and fast startup times.

📌 Feature Summary

• Introduce pull_policy: once_per_day, once_per_week, etc.
• Allow docker compose to track the last pull time and only update when necessary.
• Improve developer workflow efficiency while ensuring images stay updated.
• Reduce unnecessary network usage and startup delays.

This feature would make a real difference in large-scale development environments! 🚀 Would love to hear feedback from the Compose team. 😃

✍️ Tags:
enhancement, docker-compose, feature-request, pull-policy, developer-experience

[ndeloof]

That would be an interesting feature, but docker engine doesn't store the time image has been pulled, so we could compute "need to be checked for update after xx" condition. So this is technically not feasible until engine offers such a feature

[marlon-sousa]

Ok.

I suppose then we should open a pr on docker to make, some how, a registry of when the images have been last cached on disk.

Docker-compose would use this information to decide if a pull trial should be performed after pull policy specified time.

Does that make sense?

[victorwads]

@ndeloof @marlon-sousa compose could easyly just save this meta data on any userData file, just to control last time compose did pull something. We don't need to validade with docker the last pull.

[ndeloof]

@victorwads it could but Compose (without r) used to be stateless and this would be a design change for a corner case. Also to be considered actual docker engine it connect to depends on active context. Better keep docker engine as the source of truth.

[marlon-sousa]

@ndeloof agreed.

Then first we need to make this info available to docker, right?

Also to be considered actual docker engine it connect to depends on active context.

Does compose connect to other engines?

I thought docker would be the only engine. If otherwise, then things start to get complicated.

Is there any ref documentation on thius matter?

[ndeloof]

Does compose connect to other engines?

Compose (and any docker xx command) can communicate to any remote docker engine, depending on the active context. You may use docker commands to control multiple machines, each of those with a distinct engine version and available images (and architectures)

By the way, Compose only relies on Moby API, so can also access an alternative implementation (typically, some use compose with podman) as long as the API is well implemented.

[ndeloof]

Actually, moby/moby#31497 introduced image metadata which can be used to know last time image was pulled.

[thaJeztah]

Setting pull_policy: always forces a docker pull on every docker compose up, even when there’s nothing new to update.

For this part, we should look where most time is taken; effectively, a docker pull should only have to do a HEAD to check if the digest it has locally is still current; from some quick tests, it still takes close to a second to do so; we should look what's taking up that much;

time docker pull --quiet ubuntu
docker.io/library/ubuntu:latest

real	0m5.163s
user	0m0.015s
sys	0m0.015s

time docker pull --quiet alpine
docker.io/library/ubuntu:latest

real	0m0.906s
user	0m0.017s
sys	0m0.013s

[thaJeztah]

Actually, moby/moby#31497 introduced image metadata which can be used to know last time image was pulled.

Reading the ticket here; I guess the request is not when a last pull actually happened, but when it was last checked (which could be "checked, but nothing to do"), so possibly slightly different. That's what's made me slightly wonder if there's things to do to make always more performant, so that it can guarantee the image to be fresh, without that coming with a real penalty (~equivalent to compose having to do a docker image inspect to learn the last time it was checked).

[ndeloof]

but when it was last checked

this is actually what metadata reports. Attempt to pull, even image is up to date, will change LastTagTime in metadata:

$ docker image inspect alpine -f '{{ .Metadata.LastTagTime }}'
2025-02-21 09:45:10.235550419 +0000 UTC
$ docker pull alpine
Using default tag: latest
latest: Pulling from library/alpine
Digest: sha256:a8560b36e8b8210634f77d9f7f9efd7ffa463e380b75e2e74aff4511df3ef88c
Status: Image is up to date for alpine:latest
docker.io/library/alpine:latest
$ docker image inspect alpine -f '{{ .Metadata.LastTagTime }}'
2025-02-21 10:07:22.382362425 +0000 UTC

[victorwads]

Just to knowledge

Ideas:
#12563
moby/moby#49515

Code Changes:
compose-spec/compose-spec#572
compose-spec/compose-go#745
#12568

Updated Docs:
https://github.com/compose-spec/compose-spec/blob/main/05-services.md#pull_policy

[victorwads]

@ndeloof Sorry to bother you, I have one more question.

Could you clarify the publication flow for Docker Compose across repositories and how it aligns with Docker Desktop updates? Specifically, how does the process work for updating documentation like this link:
🔗 Compose File Services - pull_policy

Thanks in advance!

[ndeloof]

Docker Desktop is released monthly and we use to release Compose ~one week before to get the last fixes/and features in
About docs, those are updated after a release based on recent changes taking place on the compose-spec side

[victorwads]

Thank Youu @ndeloof