File watching enters infinite loop with certain variable naming patterns

[damarals]

Describe the issue

When using Kestra with file watching enabled (Micronaut IO watch), flows containing variables with certain patterns can enter an infinite loop of file rewrites. This causes the flow to be continuously rewritten with identical versions, flooding the revisions history in the UI.

Example 1: Variables with Prefix Relationship

Failing Case:

id: myflow_with_error
namespace: company.team

inputs:
  - id: year
    type: SELECT
    values: ["2019", "2020"]
    defaults: "2019"

variables:
  year_month: "{{inputs.year}}_jan"
  year: "{{inputs.year}}"

tasks:
  - id: hello
    type: io.kestra.plugin.core.log.Log
    message: Hello World! 🚀

Working Case (only changed variable order):

id: myflow_with_error
namespace: company.team

inputs:
  - id: year
    type: SELECT
    values: ["2019", "2020"]
    defaults: "2019"

variables:
  year: "{{inputs.year}}"
  year_month: "{{inputs.year}}_jan"

tasks:
  - id: hello
    type: io.kestra.plugin.core.log.Log
    message: Hello World! 🚀

Example 2: More Complex Case

Failing Case (Order doesn't fix the issue):

id: 02_postgres_taxi
namespace: zoomcamp

inputs:
  - id: taxi
    type: SELECT
    displayName: Select taxi type
    values: [yellow, green]
    defaults: yellow

  - id: year
    type: SELECT
    displayName: Select year
    values: ["2019", "2020"]
    defaults: "2019"

  - id: month
    type: SELECT
    displayName: Select month
    values: ["01", "02", "03", "04", "05", "06", "07", "08", "09", "10", "11", "12"]
    defaults: "01"

variables:
  file: "{{inputs.taxi}}_tripdata_{{inputs.year}}-{{inputs.month}}.csv"
  table: "public.{{inputs.taxi}}_tripdata"
  staging_table: "public.{{inputs.taxi}}_tripdata_staging"
  data: "{{outputs.extract.outputFiles[inputs.taxi ~ '_tripdata_' ~ inputs.year ~ '-' ~ inputs.month ~ '.csv']}}"

tasks:
  - id: hello
    type: io.kestra.plugin.core.log.Log
    message: Hello World! 🚀

The failing flow above is from the Data Engineering Zoomcamp 2025's Kestra contribution section. The original flow works fine in the course material since it doesn't use the file sync configuration. The issue only became apparent when implementing the same flow in a local development environment with file watching enabled.

You can verify the original working flow in the course repository, where the Docker Compose configuration doesn't include the Micronaut file watch settings, thus avoiding this bug.

Error Logs

2025-01-21 03:05:38,164 INFO  jdbc-queue-FlowWithSource_0 i.k.c.s.FileChangedEventListener Flow myflow_with_error has been written to file /flows/company.team_myflow_with_error.yml
2025-01-21 03:05:38,168 INFO  standalone   i.k.c.s.FileChangedEventListener Flow myflow_with_error from file company.team_myflow_with_error.yml has been created or modified
[... continuous loop of similar messages ...]

Impact

• Floods revision history with identical versions
• Creates unnecessary file system operations
• Makes it difficult to track actual changes to flows
• Affects local development workflow

Current Workarounds

1. Disable file watching
2. Change variable order (works in some cases)

Additional Notes

• Issue only occurs with file watching enabled
• Other flows without these variable patterns work normally

Environment

Reproduction Steps

1. Set up Kestra using the Docker Compose configuration below
2. Create a flow with variables that either:
   • Share common prefixes (Example 1)
   • Have complex interpolation patterns (Example 2)
3. Save the flow
4. Observe continuous rewrites in the logs

Docker Compose Configuration

services:
  kestra:
    image: kestra/kestra:latest
    pull_policy: always
    user: "root"
    command: server standalone
    volumes:
      - ./flows:/flows
      - kestra-data:/app/storage
      - /var/run/docker.sock:/var/run/docker.sock
      - /tmp/kestra-wd:/tmp/kestra-wd
    environment:
      KESTRA_CONFIGURATION: |
        datasources:
          postgres:
            url: jdbc:postgresql://postgres:5432/kestra
            driverClassName: org.postgresql.Driver
            username: kestra
            password: k3str4
        kestra:
          server:
            basicAuth:
              enabled: false
              username: "[email protected]"
              password: kestra
          repository:
            type: postgres
          storage:
            type: local
            local:
              basePath: "/app/storage"
          queue:
            type: postgres
          tasks:
            tmpDir:
              path: /tmp/kestra-wd/tmp
          tutorialFlows:
            enabled: false
          url: http://localhost:8080/
        micronaut:
            io:
              watch:
                enabled: true
                paths:
                  - /flows
    ports:
      - "8080:8080"
      - "8081:8081"
    depends_on:
      postgres:
        condition: service_started

  postgres:
    image: postgres
    volumes:
      - postgres-data:/var/lib/postgresql/data
    environment:
      POSTGRES_DB: kestra
      POSTGRES_USER: kestra
      POSTGRES_PASSWORD: k3str4
    healthcheck:
      test: ["CMD-SHELL", "pg_isready -d $${POSTGRES_DB} -U $${POSTGRES_USER}"]
      interval: 30s
      timeout: 10s
      retries: 10

volumes:
  postgres-data:
    driver: local
  kestra-data:
    driver: local