[bug] moon ci results in different actions plan on GHA and local

[artsiommiksiuk]

Describe the bug

On CI, with GHA, I'm using this setup:

  docker-build:
    needs: test
    runs-on: ubuntu-latest

    steps:
      - uses: actions/checkout@v4
        with:
          lfs: true
          fetch-depth: 0
      - uses: extractions/setup-just@v2
      - uses: moonrepo/setup-toolchain@v0
      - uses: crazy-max/ghaction-setup-docker@v3
        with:
          daemon-config: |
            {
              "features": {
                "containerd-snapshotter": true
              }
            }

      - uses: aws-actions/configure-aws-credentials@v4
        with:
          aws-region: eu-central-1
      - uses: aws-actions/amazon-ecr-login@v2
        with:
          mask-password: "true"

      - run: moon ci --color :docker-push-ci --cache=read

      - uses: appthrust/moon-ci-retrospect@v1
        if: success() || failure()

Which results moon assembling this info:

All qrmap:* tasks never should've been picked up (nothing was changed from it's dependencies).

Meanwhile on local machine I can't reproduce it even if I specifically setting up MOON_BASE and MOON_HEAD to simulate what CI suppose to see.

It does not picks qrmap as something needing change. Which is correct.

Steps to reproduce

I really don't know how to make good reproducible example, and I'm not excluding that I don't understand something.

Expected behavior

CI moon ci run behaves the same as local.

Or, if it is some configuration error from my side, I want to know how to make moon ci :some-command still honor only affected targets and run only specific command, like run :some-command only for targets been affected by current commit.

Screenshots

Environment

  System:
    OS: macOS 14.5
    CPU: (12) arm64 Apple M3 Pro
    Memory: 117.92 MB / 36.00 GB
    Shell: 5.9 - /bin/zsh
  Binaries:
    Node: 20.11.1 - ~/.nvm/versions/node/v20.11.1/bin/node
    npm: 10.2.4 - ~/.nvm/versions/node/v20.11.1/bin/npm
    pnpm: 9.14.2 - ~/Library/pnpm/pnpm
  Managers:
    Homebrew: 4.4.15 - /opt/homebrew/bin/brew
    pip3: 24.2 - ~/Projects/backend/.venv-apps-qrmap/bin/pip3
    RubyGems: 3.0.3.1 - /usr/bin/gem
  Utilities:
    Bazel: 7.4.0 - /opt/homebrew/bin/bazel
    Make: 3.81 - /usr/bin/make
    GCC: 15.0.0 - /usr/bin/gcc
    Git: 2.39.3 - /usr/bin/git
    Clang: 15.0.0 - /usr/bin/clang
    Curl: 8.6.0 - /usr/bin/curl
    OpenSSL: 3.4.0 - /opt/homebrew/bin/openssl
  Servers:
    Apache: 2.4.58 - /usr/sbin/apachectl
  Virtualization:
    Docker: 27.4.0 - /usr/local/bin/docker
    Docker Compose: 2.31.0 - /usr/local/bin/docker-compose
  IDEs:
    VSCode: 1.96.0 - /usr/local/bin/code
    Vim: 9.0 - /usr/bin/vim
    Xcode: /undefined - /usr/bin/xcodebuild
  Languages:
    Bash: 3.2.57 - /bin/bash
    Go: 1.23.4 - /opt/homebrew/bin/go
    Perl: 5.34.1 - /usr/bin/perl
    Protoc: 28.3 - /opt/homebrew/bin/protoc
    Python: 3.10.15 - /Users/artsiommiksiuk/Projects/backend/.venv-apps-qrmap/bin/python
    Python3: 3.10.15 - /Users/artsiommiksiuk/Projects/backend/.venv-apps-qrmap/bin/python3
    Ruby: 2.6.10 - /usr/bin/ruby
  Databases:
    PostgreSQL: 14.13 - /opt/homebrew/bin/postgres
    SQLite: 3.43.2 - /usr/bin/sqlite3
  Browsers:
    Chrome: 131.0.6778.265
    Safari: 17.5

Additional context

[artsiommiksiuk]

Okay, what I figured also, that moon still rebuilds everything even if touched files are not related to moon handled projects.

Can I have somehow behaviour that moons executes it's commands only if one of the moon managed projects has touched files?

[milesj]

@artsiommiksiuk If you pass --log trace, it will tell you why each task was affected, and by what.

[artsiommiksiuk]

Okay, I figured, that this is because it affected by GITHUB_SHA set in GHA CI. And from the documentation, if env set moon always adds this task into affected list.

This is one of the tasks is defined.

stack: "infrastructure"
type: "library"

env:
  MT_ECR_REGISTRY: registry_address
  MT_ECR_REPOSITORY: repo_name
  MT_LOCAL_IMAGE_NAME: local_image_name

tasks:
  ...

  docker-push-ci:
    type: run
    script: >
      docker tag $MT_LOCAL_IMAGE_NAME:latest $MT_ECR_REGISTRY/$MT_ECR_REPOSITORY:latest &&
      docker tag $MT_LOCAL_IMAGE_NAME:latest $MT_ECR_REGISTRY/$MT_ECR_REPOSITORY:$(git rev-parse --short $GITHUB_SHA) &&
      docker push $MT_ECR_REGISTRY/$MT_ECR_REPOSITORY --all-tags
    deps:
      - docker-build-ci
    options:
      cache: false

So, how can I achieve rebuild only affected packages in this case?

[milesj]

Youll have to set inferInputs to false for any task referencing that env var.

[artsiommiksiuk]

Okay, that's unpredictable, but yeah, it should work.

[artsiommiksiuk]

But I still think that this is an incorrect behavior. I'd assume that running CI, first filters by touched, then from the graph of touched cached actions should be eliminated.
In other words, CI logic filtering must not interfer with caching (inputs or outputs). Because in this case it seems it is.

[artsiommiksiuk]

Or, I need a way to specify filtering priority flag. Like if --affected-first or --touched-first is set, then it suppose to construct actions graph based on touched files first.

So, with or without caching, because implicit inputs still are just inputs, and inputs are part of a caching mechanism, moon ci must rely on touched files in priority and not mix things.

If user wants to execute full graph, he should just use moon instead.

[milesj]

Environment variables work just the same as touched files, so there's not much difference here. They are both considered inputs and feed into the affected tracker.

This just happens to be an unfortunate scenario where an env var that constantly changes is used as an input. We probably need a blacklist of env vars to ignore, but in the meantime, turning inferInputs off should work.

[artsiommiksiuk]

I've added this and it's working.

But I really still have a feeling that this shouldn't work like this anyway. I understand why ENV's might be implemented the same way as files, but this becomes misleading in the boundaries of git.

Git operates with files, and assumption is that touched files supposed to be the ones that git tracks. Git doesn't track envs, therefore, envs shouldn't behave the same way when doing affected checks in this matter.

I'd say, better would be if moon would be more specific. Affected for the VCS must be based on what VCS is tracking. Most VCS tracking FS entities, therefore only FS must be considered. If there will be VCS for FS, ENV and Remote resources, then it would make sense to track them the same way calculating diff.

Otherwise, the rest shouldn't make a difference. I'm not sure also how this would behave with conjunction to cache? If we have something not being affected by the VCS diff, but cache, especially remote is invalid, shouldn't it be rerun then as well?

I hope I wasn't too vague giving this example.

[milesj]

Affected uses inputs to determine whether something is affected or not, and inputs are more than just files. This unfortunately will not change.

[milesj]

Making a change in v1.32 that doesn't infer from CI/CD vars. #1811

[milesj]

v1.32 out.