Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Rec use meson to create dist file #15076

Open
wants to merge 10 commits into
base: master
Choose a base branch
from

Conversation

omoerbeek
Copy link
Member

@omoerbeek omoerbeek commented Jan 22, 2025

Short description

Draft as I need to double check the differences between the the autotool generated tarball and meson generated tarball.

meson dist -C yourbuilddir --formats bztar gets you the tarball.

Meson dist uses a partial checkout of the tree for subprojects (as rec is), so symlinks into the parent are dangling. There is a dist script to fix those.

To bootstrap, a few symlinks are replaced in git by actual files, as the project clause needs them before the dist script is run.

Next, after the dist dir is created, we replace the remaining symlinks into the rest of the tree by the actual files using some tar magic in the dist script.

I checked the produced tarbal can be used to build both using meson and autotools.

version.sh script from meson docs: https://mesonbuild.com/Creating-releases.html#cement-a-version-obtained-from-vcs

Checklist

I have:

  • read the CONTRIBUTING.md document
  • compiled this code
  • tested this code
  • included documentation (including possible behaviour changes)
  • documented the code
  • added or modified regression test(s)
  • added or modified unit test(s)

@omoerbeek omoerbeek changed the title Rec meson dist Rec use meson to create dist file Jan 22, 2025
@omoerbeek omoerbeek marked this pull request as draft January 22, 2025 14:19
@omoerbeek omoerbeek added the rec label Jan 22, 2025
@coveralls
Copy link

coveralls commented Jan 22, 2025

Pull Request Test Coverage Report for Build 12985846512

Warning: This coverage report may be inaccurate.

This pull request's base commit is no longer the HEAD commit of its target branch. This means it includes changes from outside the original pull request, including, potentially, unrelated coverage changes.

Details

  • 0 of 0 changed or added relevant lines in 0 files are covered.
  • 54 unchanged lines in 13 files lost coverage.
  • Overall coverage decreased (-0.01%) to 64.689%

Files with Coverage Reduction New Missed Lines %
pdns/tcpiohandler.cc 1 68.24%
pdns/recursordist/syncres.cc 2 80.1%
pdns/recursordist/aggressive_nsec.cc 2 66.32%
pdns/sstuff.hh 2 56.83%
pdns/stubresolver.cc 3 77.58%
pdns/packethandler.cc 3 72.53%
pdns/tsigverifier.cc 3 77.22%
pdns/opensslsigners.cc 3 61.34%
pdns/rcpgenerator.cc 3 90.73%
pdns/dnsdistdist/dnsdist-carbon.cc 6 61.6%
Totals Coverage Status
Change from base Build 12983429925: -0.01%
Covered Lines: 127814
Relevant Lines: 166470

💛 - Coveralls

@omoerbeek
Copy link
Member Author

omoerbeek commented Jan 22, 2025

Dir compare shows that the meson tarball has some extra files.
autotools show a few generated files extra that are currenlty missing in the meson tarball.
I marked them to stand out

The question is if we want to have these files in the meson tarball as well. They would require ragel and python as abuild dep. I'd say include in tarball for ragel and nope for python generated files.

Only in meson: .gitignore
Only in meson: Caddyfile
Only in meson: README.md
Only in meson: autom4te.cache
Only in meson/build-aux: gen-version
*** Only in autotools: dnslabeltext.cc (handled)
Only in meson: docs
*** Only in autotools: effective_tld_names.dat
Only in meson: examples
Only in meson/ext/arc4random: .gitignore
Only in meson/ext/arc4random: meson.build
Only in meson/ext/json11: .gitignore
Only in meson/ext/json11: meson.build
Only in meson/ext/luawrapper: meson.build
Only in meson/ext/probds: .gitignore
Only in meson/ext/probds: meson.build
Only in meson/ext/protozero: CHANGELOG.md
Only in meson/ext/protozero: LICENSE.from_folly
Only in meson/ext/protozero: LICENSE.md
Only in meson/ext/protozero: meson.build
Only in meson/ext/yahttp: .gitignore
Only in meson/ext/yahttp: meson.build
Only in meson/ext/yahttp/yahttp: meson.build
Only in meson/m4: pdns_enable_p11kit.m4
Only in meson/m4: pdns_with_gnutls.m4
Only in meson: make-ext-symlinks.py
Only in meson: meson
Only in meson: meson-dist-script.sh
Only in meson: meson.build
Only in meson: meson_options.txt
Only in meson: recursor-lsan.supp
Only in meson: recursor-tsan.supp
Only in meson/settings: .gitignore
*** Only in autotools/settings: cxxsettings-generated.cc (not needed)
Only in meson/settings: meson.build
Only in meson/settings/rust: .gitignore
Only in meson/settings/rust: build_settings
Only in meson/settings/rust: meson.build
*** Only in autotools/settings/rust/src: lib.rs (not needed)
Only in meson: version.sh

@eli-schwartz
Copy link
Contributor

Dir compare shows that the meson tarball has some extra files.

gitignore files can be excluded pretty easily: #15078

pdns/recursordist/meson-dist-script.sh Outdated Show resolved Hide resolved
@omoerbeek
Copy link
Member Author

omoerbeek commented Jan 24, 2025

Current state is that this shows there are a few files missing form the autotools tarball (files not essential for building, but still...). For the rest extra files in meson tarball are 1) files needed for meson to work 2) files that wil be skipped by #15078
or files added by this PR

Only in meson: .gitattributes
Only in meson: .gitignore
Only in meson/build-aux: gen-version
Only in meson/ext/arc4random: .gitignore
Only in meson/ext/arc4random: meson.build
Only in meson/ext/json11: .gitignore
Only in meson/ext/json11: meson.build
Only in meson/ext/luawrapper: meson.build
Only in meson/ext/probds: .gitignore
Only in meson/ext/probds: meson.build
Only in meson/ext/protozero: CHANGELOG.md
Only in meson/ext/protozero: LICENSE.from_folly
Only in meson/ext/protozero: LICENSE.md
Only in meson/ext/protozero: meson.build
Only in meson/ext/yahttp: .gitignore
Only in autotools/ext/yahttp: README.md
Only in meson/ext/yahttp: meson.build
Only in meson/ext/yahttp/yahttp: meson.build
Only in meson/m4: pdns_enable_p11kit.m4
Only in meson/m4: pdns_with_gnutls.m4
Only in meson: meson
Only in meson: meson-dist-script.sh
Only in meson: meson.build
Only in meson: meson_options.txt
Only in autotools: pubsuffix.cc (generated using shell script)
Only in autotools: rec-metrics-gen.h (generated using python script))
Only in autotools: rec-oids-gen.h (idem)
Only in autotools: rec-prometheus-gen.h (idem)
Only in autotools: rec-snmp-gen.h (idem)
Only in meson/settings: .gitignore
Only in autotools/settings: cxxsettings-generated.cc (idem)
Only in meson/settings: meson.build
Only in meson/settings/rust: .gitignore
Only in meson/settings/rust: build_settings
Only in meson/settings/rust: meson.build
Only in autotools/settings/rust/src: lib.rs (idem)
Only in meson: version.sh

@omoerbeek omoerbeek marked this pull request as ready for review January 24, 2025 08:57
@omoerbeek omoerbeek requested a review from Habbie January 24, 2025 08:57
@omoerbeek
Copy link
Member Author

omoerbeek commented Jan 27, 2025

And after a rebase to pick up #15078:

Only in meson/build-aux: gen-version
Only in meson/ext/arc4random: meson.build
Only in meson/ext/json11: meson.build
Only in meson/ext/luawrapper: meson.build
Only in meson/ext/probds: meson.build
Only in meson/ext/protozero: CHANGELOG.md
Only in meson/ext/protozero: LICENSE.from_folly
Only in meson/ext/protozero: LICENSE.md
Only in meson/ext/protozero: meson.build
Only in meson/ext/yahttp: meson.build
Only in meson/ext/yahttp/yahttp: meson.build
Only in meson/m4: pdns_enable_p11kit.m4
Only in meson/m4: pdns_with_gnutls.m4
Only in meson: meson
Only in meson: meson-dist-script.sh
Only in meson: meson.build
Only in meson: meson_options.txt
Only in autotools: pubsuffix.cc
Only in autotools: rec-metrics-gen.h
Only in autotools: rec-oids-gen.h
Only in autotools: rec-prometheus-gen.h
Only in autotools: rec-snmp-gen.h
Only in autotools/settings: cxxsettings-generated.cc
Only in meson/settings: meson.build
Only in meson/settings/rust: build_settings
Only in meson/settings/rust: meson.build
Only in autotools/settings/rust/src: lib.rs
Only in meson: version.sh

All mentioned Only in autotools files are generated by calling a python or shell script.

@rgacogne
Copy link
Member

It looks quite good, nice work!

Comment on lines +6 to +7
recursor-lsan.supp export-ignore
recursor-tsan.supp export-ignore
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is it worth keeping these in and setting it up so that meson sanitizer builds automatically use them?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

At some point yes, but for now my goal is to produce a valid distfile. Extra features can be done later.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants