[BUG] additional_contexts for builds with references to other services not working as documented

[florianhofhammer]

Description

Hi,

I am currently building a compose project making use of the service:<service_name> syntax in the additional_contexts list of my build context (link to corresponding docs).
Note that the example given in the documentation fails because the service base cannot be defined twice. I therefore renamed the second service.
I'd expect this to work fine, but docker compose build ends up not finding the corresponding image, i.e., I get failed to solve: base: failed to resolve source metadata for docker.io/library/base:latest.

I'll add the minimal compose.yaml for reproduction in the steps to reproduce, and provide a suggestion on how to either fix this issue or, if my suggestion is not easily feasible from a technical perspective, on how to update the documentation.

Thanks in advance!

Steps To Reproduce

1. Put the following minimal compose.yaml in /tmp/compose.yaml:

   services:
     base:
       build:
         context: .
         dockerfile_inline: |
           FROM docker.io/alpine:latest
   
     derived:
       build:
         context: .
         dockerfile_inline: |
           FROM base
         additional_contexts:
           base: service:base

2. Run docker compose build or sudo docker compose build, depending on whether your user is in the docker group or not
3. The build fails.

A working compose.yaml:

services:
  base:
    image: base
    build:
      context: .
      dockerfile_inline: |
        FROM docker.io/alpine:latest

  derived:
    build:
      context: .
      dockerfile_inline: |
        FROM base
      additional_contexts:
        base: service:base

Note the additional image: base key.

My observation is as follows:
The additional_contexts value, i.e., the service:base only needs to refer to the correct service.
However, the corresponding key, i.e., the base needs to refer to the corresponding service's image name.
If no image name and project name is defined, docker compose derives the project name from the working directory and therefore also the image names.
In this specific instance (project directory /tmp), this means that the following compose.yaml builds fine:

services:
  base:
    build:
      context: .
      dockerfile_inline: |
        FROM docker.io/alpine:latest

  derived:
    build:
      context: .
      dockerfile_inline: |
        FROM tmp-base
      additional_contexts:
        tmp-base: service:base

Note the tmp-base in the derived service. This behavior is not exactly intuitive, and requires to always be aware of the project name.

Suggestions

On the technical side, this problem could be resolved by the key, i.e., base/tmp-base not requiring to be the same as the base service's image name.
More specifically, the chosen key (which could also lead to asdasdasd: service:base instead of base: service:base) becomes an alias to the corresponding image name internally (with this example: asdasdasd becomes an alias for tmp-base).
This behavior would be the most intuitive but I don't know enough about the compose internals to judge its feasibility.

Otherwise, this behavior could be explicitly documented, pointing out that a user has to either

1. specify the correct, project-specific image name (here: tmp-base), or
2. set the image name for the base image explicitly (here: image: base).

In my opinion, a technical solution would be better than a solution on the documentation side because the project-specific image name is not necessarily portable and the explicit image name for the base image might collide with image names in the global namespace.

Compose Version

Docker Compose version 2.33.0

Docker Environment

Client:
 Version:    27.5.1
 Context:    default
 Debug Mode: false
 Plugins:
  buildx: Docker Buildx (Docker Inc.)
    Version:  0.20.1
    Path:     /usr/lib/docker/cli-plugins/docker-buildx
  compose: Docker Compose (Docker Inc.)
    Version:  2.33.0
    Path:     /usr/lib/docker/cli-plugins/docker-compose

Server:
 Containers: 0
  Running: 0
  Paused: 0
  Stopped: 0
 Images: 2
 Server Version: 27.5.1
 Storage Driver: btrfs
  Btrfs: 
 Logging Driver: json-file
 Cgroup Driver: systemd
 Cgroup Version: 2
 Plugins:
  Volume: local
  Network: bridge host ipvlan macvlan null overlay
  Log: awslogs fluentd gcplogs gelf journald json-file local splunk syslog
 Swarm: inactive
 Runtimes: io.containerd.runc.v2 runc
 Default Runtime: runc
 Init Binary: docker-init
 containerd version: c507a0257ea6462fbd6f5ba4f5c74facb04021f4.m
 runc version: 
 init version: de40ad0
 Security Options:
  seccomp
   Profile: builtin
  cgroupns
 Kernel Version: 6.13.2-zen1-1-zen
 Operating System: Arch Linux
 OSType: linux
 Architecture: x86_64
 CPUs: 8
 Total Memory: 15.27GiB
 Name: <redacted>
 ID: e0efb926-1352-4dfc-aaa2-42a54dcb52b4
 Docker Root Dir: /var/lib/docker
 Debug Mode: false
 Experimental: false
 Insecure Registries:
  127.0.0.0/8
 Live Restore Enabled: false

Anything else?

No response

[florianhofhammer]

Please note that the issue disappears with COMPOSE_BAKE=true set.
I missed this in my original experiments, my apologies.

In this case, I suppose the easiest solution is to document that the bake backend for compose is required in the corresponding docs.

Thanks in advance!

[ndeloof]

I can confirm this is a bug - which only affects the internal build client. COMPOSE_BAKE=true is a valid temporary workaround

[florianhofhammer]

Thanks for the quick fix!