add hltPrintMenuVersions and utilities for HLT-menu spreadsheets [13_2_X]

[missirol]

backport of #42967
backport of #42976

PR description:

From the description of #42967:

This PR adds a few utilities related to the maintenance of the HLT menus.

• hltPrintMenuVersions is a modified version of a script shared by @silviodonato to print to stdout metadata ("save comment" and "release template") of each version of a given HLT menu in ConfDB. I have added some command-line options to it using argparse (see --help for more info). Some examples below.

  hltPrintMenuVersions /dev/CMSSW_13_2_0/Fake2 -c 'cms_hlt_v3_r/convertMe!@cmsr'
  hltPrintMenuVersions /dev/CMSSW_13_2_0/Fake2 --db run3                        
  hltPrintMenuVersions /dev/CMSSW_10_0_0/Fake2 --db run2
  hltPrintMenuVersions /cdaq/special/PilotBeamTest2021/Circulating --db adg                        

• Upon the release of a new HLT menu for data-taking (or MC production), TSG produces a spreadsheet which contains information on the content of the menu. The tools used to create this spreadsheet are added in this PR to HLTrigger/Configuration/scripts/utils (see the readme file in scripts/utils for further details).

Unit tests for hltPrintMenuVersions and hltMenuContentToCSVs are included. (These new tests rely on connecting to ConfDB, so I fear they might occasionally fail due to connection glitches, based on what I remember from #37598 (comment).)

From the description of #42976:

This PR restricts the execution of the unit test of hltPrintMenuVersions (added in #42967) to the x86-64 architecture, given that the python module cx_Oracle is not fully working in CMSSW builds with non-x86-64 architecture (e.g. Aarch64 and PowerPC).

The implementation is tentatively copied from other unit tests in CMSSW.

PR validation:

The new unit tests pass.

If this PR is a backport, please specify the original PR and why you need to backport that PR. If this PR will be backported, please specify to which release cycle the backport is meant for:

#42967
#42976

Utilities for HLT-menu development.

[cmsbuild]

A new Pull Request was created by @missirol (Marino Missiroli) for CMSSW_13_2_X.

It involves the following packages:

• HLTrigger/Configuration (hlt)

@missirol, @cmsbuild, @Martin-Grunewald, @mmusich can you please review it and eventually sign? Thanks.
@silviodonato, @Martin-Grunewald this is something you requested to watch as well.
@sextonkennedy, @antoniovilela, @rappoccio you are the release manager for this.

cms-bot commands are listed here

• Backported from add hltPrintMenuVersions and utilities for HLT-menu spreadsheets #42967

[mmusich]

please test

[cmsbuild]

+1

Summary: https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-c5a997/35095/summary.html
COMMIT: ffa2eee
CMSSW: CMSSW_13_2_X_2023-10-08-2300/el8_amd64_gcc11
User test area: For local testing, you can use /cvmfs/cms-ci.cern.ch/week0/cms-sw/cmssw/42969/35095/install.sh to create a dev area with all the needed externals and cmssw changes.

Comparison Summary

Summary:

• You potentially added 3 lines to the logs
• Reco comparison results: 12 differences found in the comparisons
• DQMHistoTests: Total files compared: 48
• DQMHistoTests: Total histograms compared: 3198423
• DQMHistoTests: Total failures: 9
• DQMHistoTests: Total nulls: 0
• DQMHistoTests: Total successes: 3198392
• DQMHistoTests: Total skipped: 22
• DQMHistoTests: Total Missing objects: 0
• DQMHistoSizes: Histogram memory added: 0.0 KiB( 47 files compared)
• Checked 207 log files, 159 edm output root files, 48 DQM output files
• TriggerResults: no differences found

[mmusich]

+hlt

• verbatim backport of add hltPrintMenuVersions and utilities for HLT-menu spreadsheets #42967

[cmsbuild]

This pull request is fully signed and it will be integrated in one of the next CMSSW_13_2_X IBs (tests are also fine) and once validation in the development release cycle CMSSW_13_3_X is complete. This pull request will now be reviewed by the release team before it's merged. @rappoccio, @sextonkennedy, @antoniovilela (and backports should be raised in the release meeting by the corresponding L2)

[mmusich]

hold

• see add hltPrintMenuVersions and utilities for HLT-menu spreadsheets #42967 (comment)

[cmsbuild]

Pull request has been put on hold by @mmusich
They need to issue an unhold command to remove the hold state or L1 can unhold it for all

[rappoccio]

+1

[cmsbuild]

Pull request #42969 was updated. @missirol, @mmusich, @Martin-Grunewald, @cmsbuild can you please check and sign again.

[mmusich]

please test for el8_aarch64_gcc11

[mmusich]

unhold

[cmsbuild]

+1

Summary: https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-c5a997/35116/summary.html
COMMIT: 2aea2e7
CMSSW: CMSSW_13_2_X_2023-10-09-2300/el8_aarch64_gcc11
User test area: For local testing, you can use /cvmfs/cms-ci.cern.ch/week0/cms-sw/cmssw/42969/35116/install.sh to create a dev area with all the needed externals and cmssw changes.

[mmusich]

+hlt

• tests pass now in all archs

[cmsbuild]

This pull request is fully signed and it will be integrated in one of the next CMSSW_13_2_X IBs (tests are also fine) and once validation in the development release cycle CMSSW_13_3_X is complete. This pull request will now be reviewed by the release team before it's merged. @sextonkennedy, @rappoccio, @antoniovilela (and backports should be raised in the release meeting by the corresponding L2)

[antoniovilela]

+1