HLT menu development for 12_4_X (4/N) + migration to 12_4_0 + frozen menu V1.2 for 2022

[missirol]

backport of #38482

PR description:

• Development of HLT menus for CMSSW_12_4_X.
• Migration of HLT configs to release template CMSSW_12_4_0.
• Introduction of V1.2 frozen (GRun) menu to be used for MC production in 12_4_X.

Includes the integration of the following JIRA tickets.

• CMSHLT-2267: [EXO] Update of displaced-ditau trigger
• CMSHLT-2287: [TSG] Update of HLTDebug* EventContents with DQMGPUvsCPU output collections
• CMSHLT-2304: [MUO,BPH] Change L1T seed of HLT_Mu15, remove HLT_Mu7p5_TkX
• CMSHLT-2315: [ECAL] Disable energy-recovery in ECAL RecHit producers
• CMSHLT-2326: [EXO] Update of 2 triggers with new displaced-global-muon reconstruction
• CMSHLT-2336: [TSG] Disable unclaimed triggers (90 triggers in total)
• CMSHLT-2343: [TSG] Add DQMGPUvsCPU stream to HIon and PRef menus
• CMSHLT-2344: [BPH] Disabled HLT_DoubleMu2_Jpsi_DoubleTkMu0_Phi_v
• CMSHLT-2346: [BPH] Removed *ToCSCS* single-muon 2018-Parking triggers and dataset
• CMSHLT-2348: [BPH] Disabled single-muon 2018-Parking triggers (integrated partially; Paths not yet removed nor replaced)
• CMSHLT-2353: [L1T] Update of MET L1T seeds

In addition, HLT configs will now use cms.ConditionalTasks instead of cms.Tasks thanks a recent update of ConfDB (V03-04-00). Further details on the reasons behind this change can be found in #36938.

PR validation:

TSG tests.

If this PR is a backport, please specify the original PR and why you need to backport that PR:

#38482

Development of HLT menus for 2022 pp data-taking.

[cmsbuild]

A new Pull Request was created by @missirol (Marino Missiroli) for CMSSW_12_4_X.

It involves the following packages:

• Configuration/HLT (hlt)
• Configuration/PyReleaseValidation (pdmv, upgrade)
• HLTrigger/Configuration (hlt)

@Martin-Grunewald, @jordan-martins, @AdrianoDee, @bbilin, @cmsbuild, @missirol, @srimanob, @kskovpen can you please review it and eventually sign? Thanks.
@makortel, @kpedro88, @fabiocos, @Martin-Grunewald, @silviodonato, @trtomei, @beaucero, @slomeo this is something you requested to watch as well.
@perrotta, @dpiparo, @qliphy you are the release manager for this.

cms-bot commands are listed here

• Backported from HLT menu development for 12_4_X (4/N) + migration to 12_4_0 + frozen menu V1.2 for 2022 [12_5_X] #38482

[missirol]

enable gpu

[missirol]

please test

[cmsbuild]

+1

Summary: https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-0377ad/25735/summary.html
COMMIT: f6ab4f3
CMSSW: CMSSW_12_4_X_2022-06-23-1100/el8_amd64_gcc10
Additional Tests: GPU
User test area: For local testing, you can use /cvmfs/cms-ci.cern.ch/week0/cms-sw/cmssw/38483/25735/install.sh to create a dev area with all the needed externals and cmssw changes.

GPU Comparison Summary

Summary:

• No significant changes to the logs found
• ROOTFileChecks: Some differences in event products or their sizes found
• Reco comparison results: 0 differences found in the comparisons
• Reco comparison had 3 failed jobs
• DQMHistoTests: Total files compared: 4
• DQMHistoTests: Total histograms compared: 19864
• DQMHistoTests: Total failures: 8
• DQMHistoTests: Total nulls: 0
• DQMHistoTests: Total successes: 19856
• DQMHistoTests: Total skipped: 0
• DQMHistoTests: Total Missing objects: 0
• DQMHistoSizes: Histogram memory added: 0.0 KiB( 3 files compared)
• Checked 12 log files, 9 edm output root files, 4 DQM output files
• TriggerResults: found differences in 3 / 3 workflows

Comparison Summary

Summary:

• No significant changes to the logs found
• ROOTFileChecks: Some differences in event products or their sizes found
• Reco comparison results: 4 differences found in the comparisons
• DQMHistoTests: Total files compared: 50
• DQMHistoTests: Total histograms compared: 3675221
• DQMHistoTests: Total failures: 1853
• DQMHistoTests: Total nulls: 10
• DQMHistoTests: Total successes: 3673336
• DQMHistoTests: Total skipped: 22
• DQMHistoTests: Total Missing objects: 0
• DQMHistoSizes: Histogram memory added: -10252.140000000001 KiB( 49 files compared)
• DQMHistoSizes: changed ( 11634.0,... ): -1706.259 KiB HLT/BPH
• DQMHistoSizes: changed ( 11634.0,... ): -2.432 KiB HLT/Filters
• Checked 208 log files, 45 edm output root files, 50 DQM output files
• TriggerResults: found differences in 4 / 49 workflows

[cmsbuild]

Pull request #38483 was updated. @Martin-Grunewald, @jordan-martins, @AdrianoDee, @bbilin, @cmsbuild, @missirol, @srimanob, @kskovpen can you please check and sign again.

[missirol]

please test

See #38482 (comment).

[cmsbuild]

-1

Failed Tests: RelVals-INPUT
Summary: https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-0377ad/25750/summary.html
COMMIT: 38e157c
CMSSW: CMSSW_12_4_X_2022-06-23-2300/el8_amd64_gcc10
Additional Tests: GPU
User test area: For local testing, you can use /cvmfs/cms-ci.cern.ch/week0/cms-sw/cmssw/38483/25750/install.sh to create a dev area with all the needed externals and cmssw changes.

RelVals-INPUT

The relvals timed out after 4 hours.

GPU Comparison Summary

Summary:

• No significant changes to the logs found
• ROOTFileChecks: Some differences in event products or their sizes found
• Reco comparison results: 0 differences found in the comparisons
• Reco comparison had 3 failed jobs
• DQMHistoTests: Total files compared: 4
• DQMHistoTests: Total histograms compared: 19864
• DQMHistoTests: Total failures: 8
• DQMHistoTests: Total nulls: 0
• DQMHistoTests: Total successes: 19856
• DQMHistoTests: Total skipped: 0
• DQMHistoTests: Total Missing objects: 0
• DQMHistoSizes: Histogram memory added: 0.0 KiB( 3 files compared)
• Checked 12 log files, 9 edm output root files, 4 DQM output files
• TriggerResults: found differences in 3 / 3 workflows

Comparison Summary

Summary:

• No significant changes to the logs found
• ROOTFileChecks: Some differences in event products or their sizes found
• Reco comparison results: 4 differences found in the comparisons
• DQMHistoTests: Total files compared: 50
• DQMHistoTests: Total histograms compared: 3675221
• DQMHistoTests: Total failures: 1858
• DQMHistoTests: Total nulls: 11
• DQMHistoTests: Total successes: 3673330
• DQMHistoTests: Total skipped: 22
• DQMHistoTests: Total Missing objects: 0
• DQMHistoSizes: Histogram memory added: -10252.136 KiB( 49 files compared)
• DQMHistoSizes: changed ( 11634.0,... ): -1706.259 KiB HLT/BPH
• DQMHistoSizes: changed ( 11634.0,... ): -2.432 KiB HLT/Filters
• DQMHistoSizes: changed ( 312.0 ): 0.004 KiB MessageLogger/Warnings
• Checked 208 log files, 45 edm output root files, 50 DQM output files
• TriggerResults: found differences in 4 / 49 workflows

[missirol]

+hlt

• the RelVal-INPUT error is most likely unrelated to this PR (we can rerun the tests, if others prefer)
• outputs of PR tests are in line with changes to the HLT menus, even though some differences are not fully understood by me (see HLT menu development for 12_4_X (4/N) + migration to 12_4_0 + frozen menu V1.2 for 2022 [12_5_X] #38482 (comment))
• in particular, it is not clear what causes the differences in GlobalRecHitsV/HCals, and I don't see how HLT changes can induce those diffs (I see nothing to be changed on the HLT side, and I don't really know how those plots are filled) (clarified in HLT menu development for 12_4_X (4/N) + migration to 12_4_0 + frozen menu V1.2 for 2022 [12_5_X] #38482 (comment))

[missirol]

@cms-sw/orp-l2

Just FYI, this is the PR with the HLT menu for the upcoming MC production in 12_4_X.

[srimanob]

+Upgrade

Backport.

[srimanob]

@cmsbuild please test

Hope we remove relval input error from the test.

[perrotta]

urgent

[cmsbuild]

+1

Summary: https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-0377ad/25765/summary.html
COMMIT: 38e157c
CMSSW: CMSSW_12_4_X_2022-06-24-1100/el8_amd64_gcc10
Additional Tests: GPU
User test area: For local testing, you can use /cvmfs/cms-ci.cern.ch/week0/cms-sw/cmssw/38483/25765/install.sh to create a dev area with all the needed externals and cmssw changes.

GPU Comparison Summary

Summary:

• No significant changes to the logs found
• ROOTFileChecks: Some differences in event products or their sizes found
• Reco comparison results: 0 differences found in the comparisons
• DQMHistoTests: Total files compared: 4
• DQMHistoTests: Total histograms compared: 19864
• DQMHistoTests: Total failures: 9
• DQMHistoTests: Total nulls: 0
• DQMHistoTests: Total successes: 19855
• DQMHistoTests: Total skipped: 0
• DQMHistoTests: Total Missing objects: 0
• DQMHistoSizes: Histogram memory added: 0.0 KiB( 3 files compared)
• Checked 12 log files, 9 edm output root files, 4 DQM output files
• TriggerResults: found differences in 3 / 3 workflows

Comparison Summary

Summary:

• No significant changes to the logs found
• ROOTFileChecks: Some differences in event products or their sizes found
• Reco comparison results: 4 differences found in the comparisons
• DQMHistoTests: Total files compared: 50
• DQMHistoTests: Total histograms compared: 3675221
• DQMHistoTests: Total failures: 1853
• DQMHistoTests: Total nulls: 10
• DQMHistoTests: Total successes: 3673336
• DQMHistoTests: Total skipped: 22
• DQMHistoTests: Total Missing objects: 0
• DQMHistoSizes: Histogram memory added: -10252.140000000001 KiB( 49 files compared)
• DQMHistoSizes: changed ( 11634.0,... ): -1706.259 KiB HLT/BPH
• DQMHistoSizes: changed ( 11634.0,... ): -2.432 KiB HLT/Filters
• Checked 208 log files, 45 edm output root files, 50 DQM output files
• TriggerResults: found differences in 4 / 49 workflows

[kskovpen]

+pdmv

[cmsbuild]

This pull request is fully signed and it will be integrated in one of the next CMSSW_12_4_X IBs (tests are also fine) and once validation in the development release cycle CMSSW_12_5_X is complete. This pull request will now be reviewed by the release team before it's merged. @perrotta, @dpiparo, @qliphy (and backports should be raised in the release meeting by the corresponding L2)

[perrotta]

+1