CI: Migrate test-new to meson

[tobiasdiez]

Migrates the test-new workflow to use Meson. The Meson build system rebuilds faster and leverages standard tools like ccache for caching Cython file compilation. The github workflow is also considerably easier to read and maintain now.

To ensure reliability, pushing the Docker image in the build workflow has been disabled, as it was continuously clogging the GitHub cache (limited to 10GB, while each cached image consumes several gigabytes). This change also resolves intermittent workflow failures caused by unsuccessful image pushes.

📝 Checklist

• The title is concise and informative.
• The description explains in detail what this PR is about.
• I have linked a relevant issue or discussion.
• I have created tests covering the changes.
• I have updated the documentation and checked the documentation preview.

⌛ Dependencies

[user202729]

Doesn't remove this defeat the whole point of test-new? (that you do a fast-fail when there's a trivial error to notify the pull request author to fix it, so you don't burn CPU on testing the rest of the files)

[tobiasdiez]

The main point of test-new is that it provides quick feedback for devs. But I'm planning to also move these "long" tests to meson in a follow-up and then it's indeed a good idea to make run only after all other normal tests were passing.

[user202729]

leverages standard tools like ccache for caching Cython file compilation

By the way, the only benefit of ccache I can see is when you git checkout a far away branch, does something, then git checkout back — the reason why meson would have taken longer without ccache is that the file modification time are bumped.

If you're already using docker then ccache should be useless. Also there's a risk of some hidden cache stale issue (there'd better be a rebuild from scratch every release at least)

[tobiasdiez]

leverages standard tools like ccache for caching Cython file compilation

By the way, the only benefit of ccache I can see is when you git checkout a far away branch, does something, then git checkout back — the reason why meson would have taken longer without ccache is that the file modification time are bumped.
If you're already using docker then ccache should be useless. Also there's a risk of some hidden cache stale issue (there'd better be a rebuild from scratch every release at least)

The meson workflow is not using docker and thus ccache is used to cache the compilation (more precisely: cython is still run on all cython files, but the compilation of the resulting c/c++ is cached).

[github-actions]

Documentation preview for this PR (built with commit 87caf60; changes) is ready! 🎉
This preview will update shortly after each push to this PR.

[user202729]

Sure why not, at least it's not slower than the current test-new.

The missing feature is "only run test-long (& full test with meson) if test-new succeeds" but… the only disadvantage is wasted electricity, I suppose.

[tobiasdiez]

Thanks @user202729 for the review. I'm setting this now to "positive review", hope you are okay with this.

[user202729]

Wait a minute…

codecov/project — 54.14% (-35.74%) compared to f280b7d

Any idea why it's like this?

[tobiasdiez]

Shouldn't be caused by this PR; in fact, the checks for the commit for the last merge commit were okay.

It is because the test-long runs fail due to out-of-space errors. This later issue comes most likely from me merging the latest develop branch into this branch before the base image was build in ci-linux. The whole setup is sadly rather unstable.

[user202729]

So… it's random? I guess if you retrigger CI it should be fine then?

(why does the amount of disk usage fluctuate anyway? Sound like a bug somewhere…)