Update monit tools version

[nikodemas]

Fix the issue with static executable (see #8682 (comment), dmwm/CMSMonitoring#239 and dmwm/CMSMonitoring#238).

FYI @vkuznet @leggerf @brij01

[cmsbuild]

A new Pull Request was created by @nikodemas for branch IB/CMSSW_13_3_X/master.

@cmsbuild, @smuzaffar, @aandvalenzuela, @iarspider can you please review it and eventually sign? Thanks.
@rappoccio, @antoniovilela, @sextonkennedy you are the release manager for this.
cms-bot commands are listed here

[smuzaffar]

please test

[cmsbuild]

Pull request #8697 was updated.

[smuzaffar]

@nikodemas , it still fails on lxplus/lxplus8 [a]

[a]

[muzaffar@lxplus800 ~]$ /cvmfs/cms-ci.cern.ch/week0/PR_3774b2b4/cmsmon/monit --help
/cvmfs/cms-ci.cern.ch/week0/PR_3774b2b4/cmsmon/../el8_amd64_gcc11/cms/cmsmon-tools/0.6.5/monit: /lib64/libc.so.6: version `GLIBC_2.34' not found (required by /cvmfs/cms-ci.cern.ch/week0/PR_3774b2b4/cmsmon/../el8_amd64_gcc11/cms/cmsmon-tools/0.6.5/monit)
/cvmfs/cms-ci.cern.ch/week0/PR_3774b2b4/cmsmon/../el8_amd64_gcc11/cms/cmsmon-tools/0.6.5/monit: /lib64/libc.so.6: version `GLIBC_2.32' not found (required by /cvmfs/cms-ci.cern.ch/week0/PR_3774b2b4/cmsmon/../el8_amd64_gcc11/cms/cmsmon-tools/0.6.5/monit)

[smuzaffar]

@vkuznet , the old version 0.5.35 also links against libc but that does not fail [a]. I think previously the release was built on CentOS7 node while new version 0.6.X is built on may be rhel9/alma9?

[muzaffar@lxplus800 ~]$ ldd /cvmfs/cms.cern.ch/el8_amd64_gcc11/cms/cmsmon-tools/0.5.35-9b563cabfb89b1b9a4f4444339006648/monit
        linux-vdso.so.1 (0x00007ff91d877000)
        libpthread.so.0 => /lib64/libpthread.so.0 (0x00007ff91d42b000)
        libc.so.6 => /lib64/libc.so.6 (0x00007ff91d066000)
        /lib64/ld-linux-x86-64.so.2 (0x00007ff91d64b000)

[vkuznet]

@smuzaffar I did not follow how PR was made but I gave instruction to @nikodemas to built static executables. If you still see dependency on GLIC it means that the built still did not produced static executables and @nikodemas should fix that. I think spec file should properly updated to pick up latest release tag which should be in place after CI/CD pipeline in CMSMonitoring. @nikodemas I suggest you manually download release tar ball and test monit tool on lxplus. You should be able to run ldd /path/monit command and it should says that it is statically linked executable. Once it is the case, you must change release tag and hash(es) (I think) to pick up in a spec which should provide only statically linked executables.

[smuzaffar]

@vkuznet , as I wrote earlier, the libc dependency was always there even the old version of these tools links against libc. So I think problem is not the static build. Looks like it was built on a newer OS (with newer glibc)
[a]

[muzaffar@lxplus7125 cmsmon-tools]$ ldd /cvmfs/cms.cern.ch/slc7_amd64_gcc10/cms/cmsmon-tools/0.5.32/monit
        linux-vdso.so.1 =>  (0x00007ffced16c000)
        libpthread.so.0 => /lib64/libpthread.so.0 (0x00007f68e608a000)
        libc.so.6 => /lib64/libc.so.6 (0x00007f68e5cbc000)
        /lib64/ld-linux-x86-64.so.2 (0x00007f68e62a6000)

[vkuznet]

yes, I understand, and my point is that we should build fully static executable. The one you referring is not a static executable as it depends on system libraries, so it is dynamically linked executable (even though it is small subset of system libs). I pointed @nikodemas that there is a way to built fully statically linked executables and he should just find proper set of options to go compiler. For instance, auth-proxy-server release (https://github.com/dmwm/auth-proxy-server/releases/download/0.2.47/auth-proxy-tools_amd64.tar.gz) provides such executables and I built them using this Makefile. Here is a proof:

# on lxplus
curl -ksLO https://github.com/dmwm/auth-proxy-server/releases/download/0.2.47/auth-proxy-tools_amd64.tar.gz
tar xfz auth-proxy-tools_amd64.tar.gz
ldd auth-proxy-tools_amd64/auth-proxy-server
        not a dynamic executable

So, it is a question of flags for specific compiler version. Since I do not know now which OS and GoLang compiler is used in CMS Monitoring I can't say much, but it is possible to build pure statically linked executable and I provided all examples.

[cmsbuild]

+1

Summary: https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-04e698/34724/summary.html
COMMIT: 59f2439
CMSSW: CMSSW_13_3_X_2023-09-12-1100/el8_amd64_gcc11
User test area: For local testing, you can use /cvmfs/cms-ci.cern.ch/week0/cms-sw/cmsdist/8697/34724/install.sh to create a dev area with all the needed externals and cmssw changes.

Comparison Summary

Summary:

• You potentially removed 1 lines from the logs
• Reco comparison results: 338 differences found in the comparisons
• DQMHistoTests: Total files compared: 50
• DQMHistoTests: Total histograms compared: 3348561
• DQMHistoTests: Total failures: 128
• DQMHistoTests: Total nulls: 0
• DQMHistoTests: Total successes: 3348411
• DQMHistoTests: Total skipped: 22
• DQMHistoTests: Total Missing objects: 0
• DQMHistoSizes: Histogram memory added: 0.0 KiB( 49 files compared)
• Checked 214 log files, 167 edm output root files, 50 DQM output files
• TriggerResults: no differences found

[nikodemas]

Hi,

thank you for your help @smuzaffar @vkuznet. I have managed to build a static executable by adding CGO_ENABLED=0 before the go build command (as was done in Valentin's make files), have tested it out by downloading the tar files to the all versions of lxplus and recreated the release. Is it ok if I leave the same version number (0.6.5)?

[smuzaffar]

@nikodemas , I would suggest to update the version.

[cmsbuild]

Pull request #8697 was updated.

[smuzaffar]

please test

[cmsbuild]

+1

Summary: https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-04e698/34749/summary.html
COMMIT: a0f27da
CMSSW: CMSSW_13_3_X_2023-09-13-2300/el8_amd64_gcc11
User test area: For local testing, you can use /cvmfs/cms-ci.cern.ch/week0/cms-sw/cmsdist/8697/34749/install.sh to create a dev area with all the needed externals and cmssw changes.

Comparison Summary

Summary:

• You potentially removed 1 lines from the logs
• ROOTFileChecks: Some differences in event products or their sizes found
• Reco comparison results: 1790 differences found in the comparisons
• DQMHistoTests: Total files compared: 50
• DQMHistoTests: Total histograms compared: 3348561
• DQMHistoTests: Total failures: 301
• DQMHistoTests: Total nulls: 0
• DQMHistoTests: Total successes: 3348238
• DQMHistoTests: Total skipped: 22
• DQMHistoTests: Total Missing objects: 0
• DQMHistoSizes: Histogram memory added: 11.238000000000001 KiB( 49 files compared)
• DQMHistoSizes: changed ( 11634.0,... ): 1.166 KiB Physics/NanoAODDQM
• DQMHistoSizes: changed ( 13234.0,... ): 0.769 KiB Physics/NanoAODDQM
• Checked 214 log files, 167 edm output root files, 50 DQM output files
• TriggerResults: no differences found

[smuzaffar]

+externals

[cmsbuild]

This pull request is fully signed and it will be integrated in one of the next IB/CMSSW_13_3_X/master IBs (tests are also fine). This pull request will now be reviewed by the release team before it's merged. @rappoccio, @antoniovilela, @sextonkennedy (and backports should be raised in the release meeting by the corresponding L2)