Running docker-compose with buildkit in EKS fails with "network bridge not found"

[whitleykeith]

Environment:

Kubernetes/EKS: v1.16.8-eks-e16311
docker-compose: 1.26
Docker version 19.03.12, build 48a66213fe

Essentially I have a docker-compose file with multiple images that I want to build inside a CI Container hosted in Kubernetes. EKS has disabled the default bridge network in Kubernetes AMIs. This means docker builds in this environment require the --network=host flag, and this is true for our images as well.

We have those expressed in our docker-compose like this:

pre-flight-debian:
    image: ${ACCOUNT}.dkr.ecr.us-east-1.amazonaws.com/2269-pre-flight:10-slim-${TAG}
    network_mode: ${NETWORK:-bridge} 
    build:
      network: ${NETWORK:-bridge} 
      context: ../platform/docker/pre-flight/debian-10-slim

In our CI system we will set NETWORK=host during our docker-compose command like this:

NETWORK=host docker-compose -f docker-compose.yaml build

We log the docker-compose file and can confirm the build.network and network_mode are set to host.

We've been able to run BuildKit on single images like

DOCKER_BUILDKIT=1 docker --network=host build . -t image:latest

with no problems, but when we try to build the entire compose like this:

DOCKER_BUILDKIT=1 COMPOSE_DOCKER_CLI_BUILD=1 NETWORK=host docker-compose -f docker-compose.yaml build

we get this error:

failed to solve with frontend dockerfile.v0: failed to build LLB: executor failed running [REDACTED]: network bridge not found

The command has been relatively random, but usually the first filesystem or network call (i.e. COPY/ADD or RUN).

Does anyone know why this is?

[tonistiigi]

@ulyssessouza any idea if this could be compose related? Weird if regular docker build works. Could it be that compose always fixes network to bridge?

[whitleykeith]

@tonistiigi @ulyssessouza Should I cross-post this issue into the compose repo?

[ulyssessouza]

@whitleykeith Good idea. Even if in the end it's not a docker-compose's problem, it's better for me to keep track of it.
Please provide docker-compose's extra information (a list is available in the issue templates) and mark me on it.

[whitleykeith]

@ulyssessouza docker/compose#7682