2025 Wfs Skeleton and Updates to Data Wfs (2024,2023,2022) #46445

AdrianoDee · 2024-10-18T14:24:37Z

PR description:

This PR proposes the addition of 2025 wfs to the matrix. For the moment they are a copy of what we run currently for 2024 with the idea that once 2025 parameters (conditions, HLT) come we will simply need to fill the dummy ones (keeping them in sync one with the other). Here I add also:

@relval2025 pointing to GRun;
auto:phase1_2025_* symbolic GTs;

The era is Era_2025 that for now is simply a copy of Era_2024. A TTbar 2025 noPU wf is added to the limited matrix to be tested with PRs.

The rationale is that I would like to avoid (with some anticipation) that we fall again in a situation such as #44028 or #41410.

Additional changes:

2023 data wfs GT moved to the offline one (auto:run3_data_relval);
2024 data wfs added to the PR tests reducing the number of 2022,2023 wfs;
small fixes for ZeroBias 2024 wfs to have the RECO datatier produced for them (as it was in 2023 and 2022) and to run the proper DQM sequence.
2024 HLT moved to relval2025 to keep GRun tested;
relval_2024 moved to Fake2;
2023 data wfs HLT moved to relval_2023 (Fake2);

PR validation:

New wfs run.

cmsbuild · 2024-10-18T14:24:52Z

@AdrianoDee, CMSSW_14_2_X branch is closed for direct updates. cms-bot is going to move this PR to master branch.
In future, please use cmssw master branch to submit your changes.

cmsbuild · 2024-10-18T14:25:10Z

cms-bot internal usage

AdrianoDee · 2024-10-18T14:25:16Z

please test

cmsbuild · 2024-10-18T14:27:39Z

+code-checks

Logs: https://cmssdt.cern.ch/SDT/code-checks/cms-sw-PR-46445/42304

There are other open Pull requests which might conflict with changes you have proposed:
- File Configuration/PyReleaseValidation/python/relval_steps.py modified in PR(s): New Inputs for D110 And Enabling HIN Phase2 Wf #46355
- File Configuration/PyReleaseValidation/python/upgradeWorkflowComponents.py modified in PR(s): Add run3 tracking low pu era #33532, CMSSW Integration of LST #45117, Introduce support for cluster splitting and JetCore iteration in Phase-2 track reconstruction #46001

cmsbuild · 2024-10-18T14:27:55Z

A new Pull Request was created by @AdrianoDee for master.

It involves the following packages:

Configuration/AlCa (alca)
Configuration/HLT (hlt)
Configuration/PyReleaseValidation (upgrade, pdmv)

@AdrianoDee, @Martin-Grunewald, @Moanwar, @atpathak, @consuegs, @kskovpen, @miquork, @mmusich, @perrotta, @srimanob, @subirsarkar, @sunilUIET can you please review it and eventually sign? Thanks.
@Martin-Grunewald, @fabiocos, @makortel, @missirol, @mmusich, @rsreds, @silviodonato, @slomeo, @tocheng, @yuanchao this is something you requested to watch as well.
@antoniovilela, @mandrenguyen, @rappoccio, @sextonkennedy you are the release manager for this.

cms-bot commands are listed here

mmusich · 2024-10-18T14:30:47Z

Configuration/HLT/python/autoHLT.py

@@ -13,6 +13,7 @@
  'relval2022' : 'Fake2',
  'relval2023' : 'Fake2',
  'relval2024' : 'GRun',


are we going to get 2025 relvals to run in IBs and PRs?

IBs no, PRs I've added a single ttbar noPu.

(explaining myself, if we overhaul completely 2024 in favor of 2025, we can demote relval2024 to Fake2 in this cycle as the purpose of having GRun here is to keep having workflows running it somewhere)

Meaning: I didn't add the in this PR but it would be doable.

(explaining myself, if we overhaul completely 2024 in favor of 2025, we can demote relval2024 to Fake2 in this cycle as the purpose of having GRun here is to keep having workflows running it somewhere)

Then yes we could do that.

we're not going to re-run the 2024 HLT in any other release than 14.0.X in production. I don't see the reason to validate the 2024 HLT in this cycle. Of course "offline" validators can continue to validate offline reconstruction.

This is clear, I wanted to understand if it's anyway useful to keep an eye on the 2024 GT (for the "offline" validations).

If it's not, I'll move everything to 2025 and that's it.

I've moved @relval2024 to Fake and added a couple of 2025 wfs to IB tests.

This is clear, I wanted to understand if it's anyway useful to keep an eye on the 2024 GT (for the "offline" validations).

2024 pp GTs are supposed to be "completed" by now.
There is still some work in progress for the 2024 HI GTs, but they are validated via different workflows than the ones addressed here.

cmsbuild · 2024-10-18T15:24:22Z

-1

Failed Tests: RelVals
Size: This PR adds an extra 92KB to repository
Summary: https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-758440/42298/summary.html
COMMIT: e6ad2e9
CMSSW: CMSSW_14_2_X_2024-10-17-2300/el8_amd64_gcc12
User test area: For local testing, you can use /cvmfs/cms-ci.cern.ch/week1/cms-sw/cmssw/46445/42298/install.sh to create a dev area with all the needed externals and cmssw changes.

RelVals

ValueError: Undefined workflows: 16834.0

AdrianoDee · 2024-10-19T17:35:09Z

please test

cmsbuild · 2024-10-31T10:29:38Z

Pull request #46445 was updated. @AdrianoDee, @Moanwar, @atpathak, @consuegs, @DickyChant, @miquork, @perrotta, @srimanob, @subirsarkar can you please check and sign again.

AdrianoDee · 2024-10-31T10:30:04Z

please abort

AdrianoDee · 2024-10-31T10:44:24Z

please test with #46559

cmsbuild · 2024-10-31T19:02:46Z

+1

Size: This PR adds an extra 16KB to repository
Summary: https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-758440/42503/summary.html
COMMIT: 7f14a11
CMSSW: CMSSW_14_2_X_2024-10-30-2300/el8_amd64_gcc12
User test area: For local testing, you can use /cvmfs/cms-ci.cern.ch/week1/cms-sw/cmssw/46445/42503/install.sh to create a dev area with all the needed externals and cmssw changes.

Comparison Summary

Summary:

You potentially removed 162 lines from the logs
ROOTFileChecks: Some differences in event products or their sizes found
Reco comparison results: 3539 differences found in the comparisons
DQMHistoTests: Total files compared: 38
DQMHistoTests: Total histograms compared: 2641417
DQMHistoTests: Total failures: 65273
DQMHistoTests: Total nulls: 1
DQMHistoTests: Total successes: 2576123
DQMHistoTests: Total skipped: 20
DQMHistoTests: Total Missing objects: 0
DQMHistoSizes: Histogram memory added: -700067.382 KiB( 37 files compared)
DQMHistoSizes: changed ( 12834.0,... ): -43568.391 KiB HLT/JME
DQMHistoSizes: changed ( 12834.0,... ): -17546.239 KiB HLT/Tracking
DQMHistoSizes: changed ( 12834.0,... ): -16130.179 KiB HLT/BTV
DQMHistoSizes: changed ( 12834.0,... ): -14352.967 KiB HLT/EGM
DQMHistoSizes: changed ( 12834.0,... ): -10467.421 KiB HLT/EXO
DQMHistoSizes: changed ( 12834.0,... ): -10390.469 KiB HLT/SiStrip
DQMHistoSizes: changed ( 12834.0,... ): -10237.354 KiB HLT/HCAL
DQMHistoSizes: changed ( 12834.0,... ): -9248.691 KiB HLT/Vertexing
DQMHistoSizes: changed ( 12834.0,... ): -9093.510 KiB HLT/Muon
DQMHistoSizes: changed ( 12834.0,... ): -9066.871 KiB HLT/Pixel
DQMHistoSizes: changed ( 12834.0 ): ...
Checked 169 log files, 139 edm output root files, 38 DQM output files
TriggerResults: found differences in 4 / 36 workflows

AdrianoDee · 2024-10-31T20:24:35Z

+pdmv

perrotta · 2024-10-31T21:21:17Z

+alca

Moanwar · 2024-11-03T15:14:27Z

+Upgrade

cmsbuild · 2024-11-03T15:14:55Z

This pull request is fully signed and it will be integrated in one of the next master IBs (tests are also fine). This pull request will now be reviewed by the release team before it's merged. @mandrenguyen, @sextonkennedy, @rappoccio, @antoniovilela (and backports should be raised in the release meeting by the corresponding L2)

mandrenguyen · 2024-11-04T10:15:03Z

+1

AdrianoDee marked this pull request as ready for review October 18, 2024 14:24

cmsbuild changed the base branch from CMSSW_14_2_X to master October 18, 2024 14:24

cmsbuild added this to the CMSSW_14_2_X milestone Oct 18, 2024

cmsbuild added hlt-pending alca-pending pending-signatures tests-pending orp-pending pdmv-pending upgrade-pending code-checks-pending labels Oct 18, 2024

cmsbuild added tests-started and removed tests-pending labels Oct 18, 2024

cmsbuild added code-checks-approved and removed code-checks-pending labels Oct 18, 2024

mmusich reviewed Oct 18, 2024

View reviewed changes

cmsbuild added tests-rejected tests-pending code-checks-pending and removed tests-started tests-rejected code-checks-approved labels Oct 18, 2024

AdrianoDee force-pushed the 2025_wfs branch from e4df2ee to 49a82d8 Compare October 19, 2024 17:31

cmsbuild added tests-pending and removed tests-started labels Oct 31, 2024

cmsbuild added tests-started and removed tests-pending labels Oct 31, 2024

cmsbuild added tests-approved and removed tests-started labels Oct 31, 2024

cmsbuild added pdmv-approved and removed pdmv-pending labels Oct 31, 2024

cmsbuild added alca-approved and removed alca-pending labels Oct 31, 2024

This was referenced Nov 1, 2024

Rename Phase-2 "2026" geometries and workflows to "Run4" #46582

Merged

Add WF for 2024 PbPb skim #46589

Merged

cmsbuild added fully-signed upgrade-approved and removed pending-signatures upgrade-pending labels Nov 3, 2024

cmsbuild added orp-approved and removed orp-pending labels Nov 4, 2024

cmsbuild merged commit 1a3f98b into cms-sw:master Nov 4, 2024
11 checks passed

AdrianoDee deleted the 2025_wfs branch November 4, 2024 10:17

cmsbuild mentioned this pull request Nov 4, 2024

Fixing Wf Names for ParkingDouble* #46601

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

2025 Wfs Skeleton and Updates to Data Wfs (2024,2023,2022) #46445

2025 Wfs Skeleton and Updates to Data Wfs (2024,2023,2022) #46445

AdrianoDee commented Oct 18, 2024 •

edited

Loading

cmsbuild commented Oct 18, 2024

cmsbuild commented Oct 18, 2024 •

edited

Loading

AdrianoDee commented Oct 18, 2024

cmsbuild commented Oct 18, 2024

cmsbuild commented Oct 18, 2024

mmusich Oct 18, 2024

AdrianoDee Oct 18, 2024

mmusich Oct 18, 2024 •

edited

Loading

AdrianoDee Oct 18, 2024

AdrianoDee Oct 18, 2024

mmusich Oct 18, 2024

AdrianoDee Oct 18, 2024

AdrianoDee Oct 18, 2024

AdrianoDee Oct 19, 2024

perrotta Oct 24, 2024

cmsbuild commented Oct 18, 2024

AdrianoDee commented Oct 19, 2024

cmsbuild commented Oct 31, 2024

AdrianoDee commented Oct 31, 2024

AdrianoDee commented Oct 31, 2024

cmsbuild commented Oct 31, 2024

AdrianoDee commented Oct 31, 2024

perrotta commented Oct 31, 2024

Moanwar commented Nov 3, 2024

cmsbuild commented Nov 3, 2024

mandrenguyen commented Nov 4, 2024

2025 Wfs Skeleton and Updates to Data Wfs (2024,2023,2022) #46445

2025 Wfs Skeleton and Updates to Data Wfs (2024,2023,2022) #46445

Conversation

AdrianoDee commented Oct 18, 2024 • edited Loading

PR description:

PR validation:

cmsbuild commented Oct 18, 2024

cmsbuild commented Oct 18, 2024 • edited Loading

AdrianoDee commented Oct 18, 2024

cmsbuild commented Oct 18, 2024

cmsbuild commented Oct 18, 2024

Choose a reason for hiding this comment

Choose a reason for hiding this comment

mmusich Oct 18, 2024 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

cmsbuild commented Oct 18, 2024

RelVals

AdrianoDee commented Oct 19, 2024

cmsbuild commented Oct 31, 2024

AdrianoDee commented Oct 31, 2024

AdrianoDee commented Oct 31, 2024

cmsbuild commented Oct 31, 2024

Comparison Summary

AdrianoDee commented Oct 31, 2024

perrotta commented Oct 31, 2024

Moanwar commented Nov 3, 2024

cmsbuild commented Nov 3, 2024

mandrenguyen commented Nov 4, 2024

AdrianoDee commented Oct 18, 2024 •

edited

Loading

cmsbuild commented Oct 18, 2024 •

edited

Loading

mmusich Oct 18, 2024 •

edited

Loading