extending services should allow overwriting port bindings

[aaronrmm]

Description

If service A has a port binding ("8000:8000"), and service B extends service A and has a port binding to the same inner port but a different outer port ("9000:8000"), then if I run both services at the same time, I should not get a Bind for 0.0.0.0:8000 failed: port is already allocated. docker-compose should identify the overlapping inner port bindings in service B, and its new binding "9000:8000" should overwrite the inherited binding "8000:8000" for service B.

My use case:
I have one service running, and I would like a neat way to run tests in an extension of that service without stopping the first service.

version: '3'
services:
  api:
    build:
      context: .
      dockerfile: ./Dockerfile
    ports:
      - "50052:8000"
    entrypoint: python main.py
    # plus a ton more other configs that would be a pain to
    # keep syncronized with the extending service

  test_api:
    extends:
      service: api
    ports:
      - "59999:8000" # same inner port
    entrypoint:
      - run_tests.sh

But running the test_api service while the api service is already running gives me the error Bind for 0.0.0.0:50052 failed: port is already allocated

[ndeloof]

The result of extends with your example is that you get both host ports 50052 and 59999 bound to container port 8080. Docker engine doesn't prevent you to bind multiple ports this way :

$ docker run -p 50052:8000 -p 59999:8000 -d nginx 
04ac8e47d1e34725c3f37dbff49db0948811a35e1796f10ab02d41064370244e
$ docker ps
CONTAINER ID   IMAGE     COMMAND                  CREATED         STATUS        PORTS                                                      NAMES
04ac8e47d1e3   nginx     "/docker-entrypoint.…"   2 seconds ago   Up 1 second   80/tcp, 0.0.0.0:50052->8000/tcp, 0.0.0.0:59999->8000/tcp   sad_nightingale

Still, something goes wrong with Compose with this same configuration as demonstrated with your reproducible example.
I'm investigating ...

[ndeloof]

With fresh new eyes on it I eventually understood the issue:
ports are merged in extended service with the source, this results into both 50052 and 59999 to be routed to container port 8000, as demonstrated by docker compose config command:

  test_api:
    build:
      context: /Users/nicolas/truc
      dockerfile: ./Dockerfile
    entrypoint:
      - run_tests.sh
    networks:
      default: null
    ports:
      - mode: ingress
        target: 8000
        published: "50052"
        protocol: tcp
      - mode: ingress
        target: 8000
        published: "59999"
        protocol: tcp

As long as the extended service api is also enabled in your compose config, both declare the same publish port 59999 which triggers a conflict when engine tries to setup networking.

There's no way (for now) to configure the merge logic to just override value.

An alternative approach is for you to use yaml anchors to share service definition and customize for each service:

x-service: &common-definition
  build:
    context: .
    dockerfile: ./Dockerfile
  ports:
    - "50052:8000"

services:
  api:
    <<: *common-definition
    ports:
      - "50052:8000"
    entrypoint: python main.py

  test_api:
    <<: *common-definition
    ports:
      - "59999:8000" # same inner port
    entrypoint:
      - run_tests.sh

As a side note, you should not declare version: x in your compose file, this is an obsolete practice

[eklunko]

An alternative approach is for you to use yaml anchors to share service definition and customize for each service:

Possible you should remove "ports" definition from shared anchor (x-service).

[qhaas]

For those finding this issue checkout this comment in issue 3729, which notes the (relatively) new !override