Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[BUG] Docker compose does not handle depends_on for extended services #12573

Closed
oliora opened this issue Feb 22, 2025 · 6 comments
Closed

[BUG] Docker compose does not handle depends_on for extended services #12573

oliora opened this issue Feb 22, 2025 · 6 comments
Assignees
Labels

Comments

@oliora
Copy link

oliora commented Feb 22, 2025

Description

In the latest version of Docker Desktop for Mac (Arm arch) Docker compose does not handle depends_on directives for extended services.

Steps To Reproduce

Note that this is a made up minimal config which will fail to actually run.

lib.yaml:

services:

  postgresql:
    image: postgres:16.4-alpine3.20
    platform: linux/amd64
    restart: unless-stopped
    shm_size: 1g
    environment:
      POSTGRES_USER: $DB_USERNAME
      POSTGRES_PASSWORD: $DB_PASSWORD
      POSTGRES_DB: $DB_DATABASE
    healthcheck:
      test: [ "CMD", "pg_isready", "-h", "127.0.0.1", "-U", "$DB_USERNAME", "-d", "$DB_DATABASE" ]
      interval: 2s
      timeout: 2s
      retries: 20
    ports:
      - "127.0.0.1:$DB_EXT_PORT:5432"

  service1:
    image: python:3.12-slim-bookworm
    restart: unless-stopped
    platform: linux/amd64
    command: "uvicorn container.api:api_app"
    depends_on:
      postgresql:
        condition: service_healthy

bug.yaml:

services:
  postgresql:
    extends:
      file: ./lib.yaml
      service: postgresql

  service1:
    extends:
      file: ./lib.yaml
      service: service1

Running command docker compose -f bug.yaml config --no-path-resolution produces the following output (interpolated compose file):

name: bug
services:
  postgresql:
    environment:
      POSTGRES_DB: ""
      POSTGRES_PASSWORD: ""
      POSTGRES_USER: ""
    healthcheck:
      test:
        - CMD
        - pg_isready
        - -h
        - 127.0.0.1
        - -U
        - ""
        - -d
        - ""
      timeout: 2s
      interval: 2s
      retries: 20
    image: postgres:16.4-alpine3.20
    networks:
      default: null
    platform: linux/amd64
    ports:
      - mode: ingress
        host_ip: 127.0.0.1
        target: 5432
        protocol: tcp
    restart: unless-stopped
    shm_size: "1073741824"
  service1:
    command:
      - uvicorn
      - container.api:api_app
    image: python:3.12-slim-bookworm
    networks:
      default: null
    platform: linux/amd64
    restart: unless-stopped
networks:
  default:
    name: bug_default

Note that there is no depends_on section for service1 where it depends on postgresql service despite there is such a dependency in lib.yaml. If I replace service extension with direct definition then there will be the depends_on section in the result file. The same issue happens if I'd run docker compose up for the same input file - the dependencies aren't handled and the startup order is fucked up.

This got broken when I've upgraded my Docker Desktop on Mac (arm arch) two days ago although I don't exactly know in which version it was working as I did not upgrade for maybe a month.

Compose Version

Docker Compose version v2.32.4-desktop.1
Docker Compose version v2.32.4-desktop.1

Docker Environment

Client:
 Version:    27.5.1
 Context:    desktop-linux
 Debug Mode: false
 Plugins:
  ai: Ask Gordon - Docker Agent (Docker Inc.)
    Version:  v0.7.3
    Path:     /Users/oliora/.docker/cli-plugins/docker-ai
  buildx: Docker Buildx (Docker Inc.)
    Version:  v0.20.1-desktop.2
    Path:     /Users/oliora/.docker/cli-plugins/docker-buildx
  compose: Docker Compose (Docker Inc.)
    Version:  v2.32.4-desktop.1
    Path:     /Users/oliora/.docker/cli-plugins/docker-compose
  debug: Get a shell into any image or container (Docker Inc.)
    Version:  0.0.38
    Path:     /Users/oliora/.docker/cli-plugins/docker-debug
  desktop: Docker Desktop commands (Beta) (Docker Inc.)
    Version:  v0.1.4
    Path:     /Users/oliora/.docker/cli-plugins/docker-desktop
  dev: Docker Dev Environments (Docker Inc.)
    Version:  v0.1.2
    Path:     /Users/oliora/.docker/cli-plugins/docker-dev
  extension: Manages Docker extensions (Docker Inc.)
    Version:  v0.2.27
    Path:     /Users/oliora/.docker/cli-plugins/docker-extension
  feedback: Provide feedback, right in your terminal! (Docker Inc.)
    Version:  v1.0.5
    Path:     /Users/oliora/.docker/cli-plugins/docker-feedback
  init: Creates Docker-related starter files for your project (Docker Inc.)
    Version:  v1.4.0
    Path:     /Users/oliora/.docker/cli-plugins/docker-init
  sbom: View the packaged-based Software Bill Of Materials (SBOM) for an image (Anchore Inc.)
    Version:  0.6.0
    Path:     /Users/oliora/.docker/cli-plugins/docker-sbom
  scout: Docker Scout (Docker Inc.)
    Version:  v1.16.1
    Path:     /Users/oliora/.docker/cli-plugins/docker-scout

Server:
 Containers: 11
  Running: 5
  Paused: 0
  Stopped: 6
 Images: 22
 Server Version: 27.5.1
 Storage Driver: overlayfs
  driver-type: io.containerd.snapshotter.v1
 Logging Driver: json-file
 Cgroup Driver: cgroupfs
 Cgroup Version: 2
 Plugins:
  Volume: local
  Network: bridge host ipvlan macvlan null overlay
  Log: awslogs fluentd gcplogs gelf journald json-file local splunk syslog
 CDI spec directories:
  /etc/cdi
  /var/run/cdi
 Swarm: inactive
 Runtimes: io.containerd.runc.v2 runc
 Default Runtime: runc
 Init Binary: docker-init
 containerd version: bcc810d6b9066471b0b6fa75f557a15a1cbf31bb
 runc version: v1.1.12-0-g51d5e946
 init version: de40ad0
 Security Options:
  seccomp
   Profile: unconfined
  cgroupns
 Kernel Version: 6.12.5-linuxkit
 Operating System: Docker Desktop
 OSType: linux
 Architecture: aarch64
 CPUs: 10
 Total Memory: 7.653GiB
 Name: docker-desktop
 ID: e389971f-bb54-4e02-9631-ec88b7c86f9a
 Docker Root Dir: /var/lib/docker
 Debug Mode: false
 HTTP Proxy: http.docker.internal:3128
 HTTPS Proxy: http.docker.internal:3128
 No Proxy: hubproxy.docker.internal
 Labels:
  com.docker.desktop.address=unix:///Users/oliora/Library/Containers/com.docker.docker/Data/docker-cli.sock
 Experimental: false
 Insecure Registries:
  hubproxy.docker.internal:5555
  127.0.0.0/8
 Live Restore Enabled: false

WARNING: daemon is not using the default seccomp profile
@kladderadeng
Copy link

I think I have the same issue. Pulling my hair out now since 2 hours trying to get a Zabbix Docker Setup running. To make it able to reproduce (I am on Ubuntu 24 LTS with compose v2.33.0), I will provide the files here.

When doing a docker compose config it only renders the services db-data-mysql, mysql-server, server-db-init and zabbix-server.
When doing a docker compose config zabbix-web-apache-mysql it renders the services mysql-server, server-db-init and zabbix-web-apache-mysql.

docker-compose.yml

services:
  server-db-init:
    extends:
      file: compose_zabbix_components.yaml
      service: server-mysql-db-init
    image: "${ZABBIX_SERVER_MYSQL_IMAGE}:${ZABBIX_ALPINE_IMAGE_TAG}${ZABBIX_IMAGE_TAG_POSTFIX}"
    volumes:
      - /etc/timezone:/etc/timezone:ro
    depends_on:
      mysql-server:
        condition: service_started
    labels:
      com.zabbix.os: "${ALPINE_OS_TAG}"

  zabbix-server:
    extends:
      file: compose_zabbix_components.yaml
      service: server-mysql
    image: "${ZABBIX_SERVER_MYSQL_IMAGE}:${ZABBIX_ALPINE_IMAGE_TAG}${ZABBIX_IMAGE_TAG_POSTFIX}"
    volumes:
      - /etc/timezone:/etc/timezone:ro
    depends_on:
      server-db-init:
        condition: service_completed_successfully
    labels:
      com.zabbix.os: "${ALPINE_OS_TAG}"

  zabbix-web-apache-mysql:
    extends:
      file: compose_zabbix_components.yaml
      service: web-apache-mysql
    image: "${ZABBIX_WEB_APACHE_MYSQL_IMAGE}:${ZABBIX_ALPINE_IMAGE_TAG}${ZABBIX_IMAGE_TAG_POSTFIX}"
    volumes:
      - /etc/timezone:/etc/timezone:ro
    depends_on:
      server-db-init:
        condition: service_completed_successfully
    labels:
      com.zabbix.os: "${ALPINE_OS_TAG}"

  zabbix-snmptraps:
    extends:
      file: compose_zabbix_components.yaml
      service: snmptraps
    image: "${ZABBIX_SNMPTRAPS_IMAGE}:${ZABBIX_ALPINE_IMAGE_TAG}${ZABBIX_IMAGE_TAG_POSTFIX}"
    labels:
      com.zabbix.os: "${ALPINE_OS_TAG}"

  mysql-server:
    extends:
      file: compose_databases.yaml
      service: mysql-server

  db-data-mysql:
    extends:
      file: compose_databases.yaml
      service: db-data-mysql

networks:
  frontend:
    driver: bridge
    enable_ipv6: "${FRONTEND_ENABLE_IPV6}"
    ipam:
      driver: "${FRONTEND_NETWORK_DRIVER}"
      config:
      - subnet: "${FRONTEND_SUBNET}"
  backend:
    driver: bridge
    enable_ipv6: "${BACKEND_ENABLE_IPV6}"
    internal: true
    ipam:
      driver: "${BACKEND_NETWORK_DRIVER}"
      config:
      - subnet: "${BACKEND_SUBNET}"
  database:
    driver: bridge
    enable_ipv6: "${DATABASE_NETWORK_ENABLE_IPV6}"
    internal: true
    ipam:
      driver: "${DATABASE_NETWORK_DRIVER}"
  tools_frontend:
    driver: bridge
    enable_ipv6: "${ADD_TOOLS_ENABLE_IPV6}"
    ipam:
      driver: "${ADD_TOOLS_NETWORK_DRIVER}"
      config:
      - subnet: "${ADD_TOOLS_SUBNET}"

volumes:
  snmptraps:
    #  mysql_socket:

secrets:
  MYSQL_USER:
    file: ${ENV_VARS_DIRECTORY}/.MYSQL_USER
  MYSQL_PASSWORD:
    file: ${ENV_VARS_DIRECTORY}/.MYSQL_PASSWORD
  MYSQL_ROOT_USER:
    file: ${ENV_VARS_DIRECTORY}/.MYSQL_ROOT_USER
  MYSQL_ROOT_PASSWORD:
    file: ${ENV_VARS_DIRECTORY}/.MYSQL_ROOT_PASSWORD

#  client-key.pem:
#    file: ${ENV_VARS_DIRECTORY}/.ZBX_DB_KEY_FILE
#  client-cert.pem:
#    file: ${ENV_VARS_DIRECTORY}/.ZBX_DB_CERT_FILE
#  root-ca.pem:
#    file: ${ENV_VARS_DIRECTORY}/.ZBX_DB_CA_FILE
#  server-cert.pem:
#    file: ${ENV_VARS_DIRECTORY}/.DB_CERT_FILE
#  server-key.pem:
#    file: ${ENV_VARS_DIRECTORY}/.DB_KEY_FILE

compose_databases.yaml

services:
  mysql-server:
    image: "${MYSQL_IMAGE}:${MYSQL_IMAGE_TAG}"
    command:
      - mysqld
      - --skip-mysqlx
      - --character-set-server=utf8mb4
      - --collation-server=utf8mb4_bin
    # Only during upgrade from versions prior 6.4 and new installations (schema deployment)
      - --log_bin_trust_function_creators=1
    # Use TLS encryption for connections to database
    #   - --require-secure-transport
    #   - --ssl-ca=/run/secrets/root-ca.pem
    #   - --ssl-cert=/run/secrets/server-cert.pem
    #   - --ssl-key=/run/secrets/server-key.pem
    restart: "${RESTART_POLICY}"
    attach: false
    volumes:
      - ${DATA_DIRECTORY}/var/lib/mysql:/var/lib/mysql:rw
      - ${ENV_VARS_DIRECTORY}/mysql_init/init_proxy_db.sql:/docker-entrypoint-initdb.d/mysql_init_proxy.sql:ro
      #   - mysql_socket:/var/run/mysqld/
    env_file:
      - path: ${ENV_VARS_DIRECTORY}/.env_db_mysql
        required: true
      - path: ${ENV_VARS_DIRECTORY}/.env_db_mysql_override
        required: false
    environment:
      - MYSQL_ROOT_PASSWORD_FILE=/run/secrets/MYSQL_ROOT_PASSWORD
    secrets:
      - MYSQL_USER
      - MYSQL_PASSWORD
      - MYSQL_ROOT_PASSWORD
    #   - server-key.pem
    #   - server-cert.pem
    #   - root-ca.pem
    stop_grace_period: 1m
    networks:
      database:
        aliases:
          - mysql-server

  db-data-mysql:
    image: "${BUSYBOX_IMAGE}:${BUSYBOX_IMAGE_TAG}"
    volumes:
      - ${DATA_DIRECTORY}/var/lib/mysql:/var/lib/mysql:rw

compose_zabbix_components.yml

services:
  server:
    init: true
    ports:
      - name: zabbix-trapper
        target: 10051
        published: "${ZABBIX_SERVER_PORT}"
        protocol: tcp
        app_protocol: zabbix-trapper
    restart: "${RESTART_POLICY}"
    attach: true
    volumes:
      - /etc/localtime:/etc/localtime:ro
      - ${DATA_DIRECTORY}/usr/lib/zabbix/alertscripts:/usr/lib/zabbix/alertscripts:ro
      - ${DATA_DIRECTORY}/usr/lib/zabbix/externalscripts:/usr/lib/zabbix/externalscripts:ro
      - ${DATA_DIRECTORY}/var/lib/zabbix/export:/var/lib/zabbix/export:rw
      - ${DATA_DIRECTORY}/var/lib/zabbix/modules:/var/lib/zabbix/modules:ro
      - ${DATA_DIRECTORY}/var/lib/zabbix/enc:/var/lib/zabbix/enc:ro
      - ${DATA_DIRECTORY}/var/lib/zabbix/ssh_keys:/var/lib/zabbix/ssh_keys:ro
      - ${DATA_DIRECTORY}/var/lib/zabbix/mibs:/var/lib/zabbix/mibs:ro
      - ${DATA_DIRECTORY}/var/lib/zabbix/ssl/certs:/var/lib/zabbix/ssl/certs:ro
      - ${DATA_DIRECTORY}/var/lib/zabbix/ssl/keys:/var/lib/zabbix/ssl/keys:ro
      - ${DATA_DIRECTORY}/var/lib/zabbix/ssl/ssl_ca:/var/lib/zabbix/ssl/ssl_ca:rw
      - snmptraps:/var/lib/zabbix/snmptraps:roz
    tmpfs: /tmp
    ulimits:
      nproc: 65535
      nofile:
        soft: 20000
        hard: 40000
    deploy:
      resources:
        limits:
          cpus: '0.70'
          memory: 1G
        reservations:
          cpus: '0.5'
          memory: 512M
    env_file:
      - path: ${ENV_VARS_DIRECTORY}/.env_srv
        required: true
      - path: ${ENV_VARS_DIRECTORY}/.env_srv_override
        required: false
    networks:
      database:
        aliases:
          - zabbix-server
      backend:
        aliases:
          - zabbix-server
      frontend:
      tools_frontend:
        #  devices:
        #   - "/dev/ttyUSB0:/dev/ttyUSB0"
    stop_grace_period: 30s
    #  cap_add:
    #    - "NET_RAW"
    sysctls:
      - net.ipv4.ip_local_port_range=1024 64999
      - net.ipv4.conf.all.accept_redirects=0
      - net.ipv4.conf.all.secure_redirects=0
      - net.ipv4.conf.all.send_redirects=0
    #   - net.ipv4.ping_group_range=0 1995
    labels:
      com.zabbix.company: "Zabbix SIA"
      com.zabbix.component: "server"

  server-mysql-db-init:
    init: true
    attach: true
    volumes:
      - /etc/localtime:/etc/localtime:ro
      - ${DATA_DIRECTORY}/var/lib/zabbix/dbscripts:/var/lib/zabbix/dbscripts:ro
      - ${DATA_DIRECTORY}/var/lib/zabbix/enc:/var/lib/zabbix/enc:ro
    #   - mysql_socket:/var/run/mysqld/
    command: init_db_only
    tmpfs: /tmp
    env_file:
      - path: ${ENV_VARS_DIRECTORY}/.env_db_mysql
        required: true
    secrets:
      - MYSQL_USER
      - MYSQL_PASSWORD
    #   - client-key.pem
    #   - client-cert.pem
    #   - root-ca.pem
    networks:
      database:
        aliases:
        - zabbix-server-mysql-init
    labels:
      com.zabbix.description: "Zabbix server with MySQL database support (database init)"
      com.zabbix.dbtype: "mysql"

  server-mysql:
    extends:
      service: server
    #  volumes:
    #   - mysql_socket:/var/run/mysqld/
    env_file:
      - path: ${ENV_VARS_DIRECTORY}/.env_db_mysql
        required: true
    secrets:
      - MYSQL_USER
      - MYSQL_PASSWORD
    #   - client-key.pem
    #   - client-cert.pem
    #   - root-ca.pem
    networks:
      backend:
        aliases:
          - zabbix-server-mysql
    labels:
      com.zabbix.description: "Zabbix server with MySQL database support"
      com.zabbix.dbtype: "mysql"

  web-apache-mysql:
    extends:
      service: web-apache
    #  volumes:
    #   - mysql_socket:/var/run/mysqld/
    env_file:
      - path: ${ENV_VARS_DIRECTORY}/.env_db_mysql
        required: true
    secrets:
      - MYSQL_USER
      - MYSQL_PASSWORD
    #   - client-key.pem
    #   - client-cert.pem
    #   - root-ca.pem
    labels:
      com.zabbix.description: "Zabbix frontend on Apache web-server with MySQL database support"
      com.zabbix.dbtype: "mysql"

  web-apache:
    profiles:
      - all
    ports:
      - name: web-http
        target: 8080
        published: "${ZABBIX_WEB_APACHE_HTTP_PORT}"
        protocol: tcp
        app_protocol: http
      - name: web-https
        target: 8443
        published: "${ZABBIX_WEB_APACHE_HTTPS_PORT}"
        protocol: tcp
        app_protocol: https
    restart: "${RESTART_POLICY}"
    attach: false
    volumes:
      - /etc/localtime:/etc/localtime:ro
      - ${DATA_DIRECTORY}/etc/ssl/apache2:/etc/ssl/apache2:ro
      - ${DATA_DIRECTORY}/usr/share/zabbix/modules/:/usr/share/zabbix/modules/:ro
    tmpfs:
      - /tmp
      - /var/lib/php/session:mode=770,uid=1997,gid=1995
    deploy:
      resources:
        limits:
          cpus: '0.70'
          memory: 512M
        reservations:
          cpus: '0.5'
          memory: 256M
    env_file:
      - path: ${ENV_VARS_DIRECTORY}/.env_web
        required: true
      - path: ${ENV_VARS_DIRECTORY}/.env_web_override
        required: false
    healthcheck:
      test: [ "CMD", "curl", "-f", "http://localhost:8080/ping" ]
      interval: 1m30s
      timeout: 3s
      retries: 3
      start_period: 40s
      start_interval: 5s
    networks:
      database:
      backend:
      frontend:
    stop_grace_period: 10s
    sysctls:
      - net.core.somaxconn=65535
    labels:
      com.zabbix.company: "Zabbix SIA"
      com.zabbix.component: "frontend"
      com.zabbix.webserver: "apache2"

  snmptraps:
    # Override snmptrapd command arguments to receive SNMP traps by DNS
    # It must be done with ZBX_SNMP_TRAP_USE_DNS=true environment variable
    #  command: /usr/sbin/snmptrapd -t -X -C -c /etc/snmp/snmptrapd.conf -Lo -A --doNotFork=yes --agentuser=zabbix --agentgroup=zabbix
    profiles:
      - full
      - all
    ports:
      - name: snmptrap
        target: 1162
        published: "${ZABBIX_SNMPTRAPS_PORT}"
        protocol: udp
        app_protocol: snmptrap
    restart: "${RESTART_POLICY}"
    attach: false
    read_only: true
    volumes:
      - snmptraps:/var/lib/zabbix/snmptraps:rwz
      - ${DATA_DIRECTORY}/var/lib/zabbix/snmptrapd_config:/var/lib/zabbix/snmptrapd_config:rw
    tmpfs: /tmp
    deploy:
      resources:
        limits:
          cpus: '0.5'
          memory: 256M
        reservations:
          cpus: '0.25'
          memory: 128M
    env_file:
      - path: ${ENV_VARS_DIRECTORY}/.env_snmptraps
        required: true
      - path: ${ENV_VARS_DIRECTORY}/.env_snmptraps_override
        required: false
    networks:
      frontend:
        aliases:
          - zabbix-snmptraps
      backend:
    stop_grace_period: 5s
    labels:
      com.zabbix.description: "Zabbix snmptraps"
      com.zabbix.company: "Zabbix SIA"
      com.zabbix.component: "snmptraps"

@oliora oliora changed the title [BUG] Docker compose config does not handle depends_on [BUG] Docker compose does not handle depends_on Feb 23, 2025
@ndeloof ndeloof self-assigned this Feb 23, 2025
@ndeloof
Copy link
Contributor

ndeloof commented Feb 24, 2025

@kladderadeng your zabbix-web-apache-mysql service extends web-apache-mysql -> web-apache which has profiles all. So the resulting service, after extends are applied, also has this profile. As long as you run docker compose config without this profile enabled service will be excluded from the final configuration rendered by command.

@oliora issue report without any detail, not even compose version (as requested by issue template) will be closed without any further investigation.

@oliora
Copy link
Author

oliora commented Feb 24, 2025

@ndeloof fair enough. I'll add required details and reopen it.

Update: provided required details.

@oliora oliora changed the title [BUG] Docker compose does not handle depends_on [BUG] Docker compose does not handle depends_on for extended services Feb 24, 2025
@glours
Copy link
Contributor

glours commented Feb 24, 2025

@oliora this is supported starting Compose v2.33.0 and above, this version will be available in the next Docker Desktop release v4.39 (coming soon)
In between you can download the binary and add it to your ~/.docker/cli-plugins directory

@oliora
Copy link
Author

oliora commented Feb 24, 2025

Good to hear it will be fixed in the next version

@kladderadeng
Copy link

@ndeloof Thank you very much. How do you say? "I dun goofed". Not only did I mix up the topic (OP was about depends_on and I was thinking about extends), it seems I did not do my full docker compose homework. I will look deeper into this and learn. Thank you very much for your polite response :-)

@ndeloof ndeloof closed this as completed Feb 24, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

No branches or pull requests

4 participants