Introduce Unified Particle Transformer v2 #47173

AlexDeMoor · 2025-01-23T21:41:17Z

PR description:

This PR opens the update of the Unified Particle Transformer bringing substantial performance improvement for flavour tagging performance and model inference time. Link of the XPOG meeting : https://indico.cern.ch/event/1504557/#3-upart-training-updates

This PR has to be tested with the following PR of the model :cms-data/RecoBTag-Combined#64

Please note the final model is being finalized for training and validation.

cmsbuild · 2025-01-23T21:41:42Z

cms-bot internal usage

cmsbuild · 2025-01-23T21:44:21Z

-code-checks

Logs: https://cmssdt.cern.ch/SDT/code-checks/cms-sw-PR-47173/43412

Code check has found code style and quality issues which could be resolved by applying following patch(s)

code-format:
https://cmssdt.cern.ch/SDT/code-checks/cms-sw-PR-47173/43412/code-format.patch
e.g. curl -k https://cmssdt.cern.ch/SDT/code-checks/cms-sw-PR-47173/43412/code-format.patch | patch -p1
You can also run scram build code-format to apply code format directly

cmsbuild · 2025-01-23T21:56:02Z

+code-checks

Logs: https://cmssdt.cern.ch/SDT/code-checks/cms-sw-PR-47173/43413

cmsbuild · 2025-01-23T21:56:24Z

A new Pull Request was created by @AlexDeMoor for master.

It involves the following packages:

RecoBTag/FeatureTools (reconstruction)
RecoBTag/ONNXRuntime (reconstruction)

@cmsbuild, @jfernan2, @mandrenguyen can you please review it and eventually sign? Thanks.
@AlexDeMoor, @Ming-Yan, @Senphy, @andrzejnovak, @castaned, @hqucms, @missirol this is something you requested to watch as well.
@antoniovilela, @mandrenguyen, @rappoccio, @sextonkennedy you are the release manager for this.

cms-bot commands are listed here

hqucms · 2025-01-24T00:37:34Z

test parameters:

pull_request = Add UParT v2 cms-data/RecoBTag-Combined#64
enable = nano,profiling

hqucms · 2025-01-24T00:38:07Z

please test

cmsbuild · 2025-01-24T06:59:55Z

+1

Size: This PR adds an extra 28KB to repository
Summary: https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-8e554d/43935/summary.html
COMMIT: 33e0cf4
CMSSW: CMSSW_15_0_X_2025-01-23-1100/el8_amd64_gcc12
Additional Tests: NANO,PROFILING
User test area: For local testing, you can use /cvmfs/cms-ci.cern.ch/week1/cms-sw/cmssw/47173/43935/install.sh to create a dev area with all the needed externals and cmssw changes.

The following merge commits were also included on top of IB + this PR after doing git cms-merge-topic:

@perrotta Update online GTs for 150X and GT for 2025 MC conditions in autoCond #47163

You can see more details here:
https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-8e554d/43935/git-recent-commits.json
https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-8e554d/43935/git-merge-result

Comparison Summary

Summary:

You potentially added 15 lines to the logs
Reco comparison results: 4138 differences found in the comparisons
DQMHistoTests: Total files compared: 49
DQMHistoTests: Total histograms compared: 3819085
DQMHistoTests: Total failures: 17154
DQMHistoTests: Total nulls: 0
DQMHistoTests: Total successes: 3801911
DQMHistoTests: Total skipped: 20
DQMHistoTests: Total Missing objects: 0
DQMHistoSizes: Histogram memory added: 0.0 KiB( 48 files compared)
Checked 214 log files, 184 edm output root files, 49 DQM output files
TriggerResults: found differences in 1 / 47 workflows

NANO Comparison Summary

Summary:

You potentially removed 70 lines from the logs
ROOTFileChecks: Some differences in event products or their sizes found
Reco comparison results: 1314 differences found in the comparisons
DQMHistoTests: Total files compared: 21
DQMHistoTests: Total histograms compared: 75127
DQMHistoTests: Total failures: 3489
DQMHistoTests: Total nulls: 0
DQMHistoTests: Total successes: 71638
DQMHistoTests: Total skipped: 0
DQMHistoTests: Total Missing objects: 0
DQMHistoSizes: Histogram memory added: 0.0 KiB( 20 files compared)
Checked 105 log files, 60 edm output root files, 21 DQM output files
TriggerResults: no differences found

Nano size comparison Summary:

Sample	kb/ev	ref kb/ev	diff kb/ev	ev/s/thd	ref ev/s/thd	diff rate	mem/thd	ref mem/thd
2500.001	3.105	3.114	-0.009 ( -0.3% )	6.00	6.40	-6.3%	2.577	2.562
2500.002	3.223	3.230	-0.007 ( -0.2% )	5.44	5.72	-5.0%	3.011	2.586
2500.003	3.161	3.171	-0.010 ( -0.3% )	5.65	5.95	-5.0%	2.982	2.604
2500.011	1.638	1.644	-0.006 ( -0.4% )	10.45	10.15	+3.0%	2.663	2.660
2500.012	2.176	2.184	-0.008 ( -0.4% )	5.84	5.97	-2.1%	2.839	2.473
2500.013	1.994	2.000	-0.006 ( -0.3% )	8.17	8.45	-3.3%	2.767	2.464
2500.021	0.022	0.022	0.000 ( +0.0% )	1.97	2.06	-4.3%	2.641	2.632
2500.022	0.022	0.022	0.000 ( +0.0% )	1.76	2.00	-11.8%	2.639	2.632
2500.023	0.022	0.022	0.000 ( +0.0% )	1.74	1.90	-8.3%	2.500	2.484
2500.024	0.022	0.022	0.000 ( +0.0% )	1.44	1.55	-7.1%	2.733	2.732
2500.031	0.035	0.035	0.000 ( +0.0% )	1.62	1.76	-7.8%	2.693	2.676
2500.032	0.036	0.036	0.000 ( +0.0% )	1.62	1.77	-8.8%	2.642	2.645
2500.033	0.037	0.037	0.000 ( +0.1% )	1.56	1.70	-8.3%	2.734	2.732
2500.034	0.036	0.036	0.000 ( +0.0% )	1.60	1.70	-5.6%	2.706	2.703
2500.101	2.844	2.844	0.000 ( +0.0% )	13.11	16.45	-20.3%	2.681	2.676
2500.111	1.463	1.463	0.000 ( +0.0% )	26.55	31.87	-16.7%	2.363	2.372
2500.112	1.883	1.883	0.000 ( +0.0% )	19.40	26.28	-26.2%	2.440	2.438
2500.131	0.750	0.750	0.000 ( +0.0% )	31.54	37.72	-16.4%	1.509	1.509
2500.201	2.674	2.674	0.000 ( +0.0% )	10.76	13.87	-22.4%	2.250	2.234
2500.211	1.806	1.806	0.000 ( +0.0% )	21.95	27.23	-19.4%	2.427	2.433
2500.212	2.203	2.203	0.000 ( +0.0% )	17.74	22.68	-21.8%	2.532	2.514
2500.221	2.038	2.038	0.000 ( +0.0% )	11.06	14.47	-23.6%	2.166	2.150
2500.222	3.479	3.479	0.000 ( +0.0% )	9.90	13.13	-24.6%	2.251	2.253
2500.223	9.431	9.444	-0.013 ( -0.1% )	4.17	4.27	-2.4%	2.274	2.269
2500.224	6.289	6.304	-0.015 ( -0.2% )	1.48	1.39	+5.9%	2.261	2.270
2500.225	6.334	6.350	-0.016 ( -0.2% )	1.25	1.30	-3.7%	2.465	2.457
2500.226	3.172	3.172	0.000 ( +0.0% )	10.82	13.79	-21.6%	2.192	2.243
2500.227	1.442	1.442	0.000 ( +0.0% )	19.80	24.12	-17.9%	1.484	1.482
2500.228	3.957	3.957	0.000 ( +0.0% )	7.07	9.20	-23.2%	2.333	2.280
2500.231	1.456	1.456	0.000 ( +0.0% )	18.78	22.73	-17.4%	2.334	2.057
2500.232	2.462	2.462	0.000 ( +0.0% )	17.07	21.65	-21.1%	2.434	2.433
2500.233	4.946	4.954	-0.008 ( -0.2% )	6.48	6.23	+4.0%	2.514	2.493
2500.234	3.833	3.842	-0.009 ( -0.2% )	1.89	1.80	+5.1%	2.455	2.217
2500.235	3.864	3.873	-0.010 ( -0.2% )	1.78	1.69	+5.5%	2.666	2.405
2500.236	2.252	2.252	0.000 ( +0.0% )	17.64	22.36	-21.1%	2.425	2.418
2500.237	1.018	1.018	0.000 ( +0.0% )	29.41	35.70	-17.6%	1.503	1.435
2500.238	2.444	2.444	0.000 ( +0.0% )	14.27	17.43	-18.2%	2.508	2.063
2500.241	9.404	9.404	0.000 ( +0.0% )	6.05	4.20	+44.1%	1.956	1.869
2500.242	10.331	10.331	0.000 ( +0.0% )	1.27	1.61	-21.1%	1.755	1.552
2500.243	2.712	2.712	0.000 ( +0.0% )	13.46	15.84	-15.0%	1.092	1.088
2500.244	486.016	486.016	0.000 ( +0.0% )	0.89	1.16	-22.9%	1.723	1.712
2500.245	826.413	826.413	0.000 ( +0.0% )	1.30	1.50	-13.0%	1.735	1.699
2500.901	1.819	1.819	0.000 ( +0.0% )	33.08	47.17	-29.9%	1.477	1.476
2500.902	1.665	1.665	0.000 ( +0.0% )	38.17	47.77	-20.1%	1.370	1.372
2500.911	14.345	14.345	0.000 ( +0.0% )	6.36	8.42	-24.4%	1.122	1.119
2500.912	0.199	0.199	0.000 ( +0.0% )	2.21	3.83	-42.1%	0.882	0.878
2500.913	0.110	0.110	0.000 ( +0.0% )	1.82	2.70	-32.7%	0.881	0.880

jfernan2 · 2025-01-24T10:22:54Z

@AlexDeMoor do I understand correctly from your PR description that performance should be improved with this PR? The rate ev/s/thd is decreased for all the NANO workflows tested

hqucms · 2025-01-24T22:10:10Z

assign xpog

cmsbuild · 2025-01-24T22:10:29Z

New categories assigned: xpog

@ftorrresd,@hqucms you have been requested to review this Pull request/Issue and eventually sign? Thanks

hqucms · 2025-01-27T13:32:23Z

@AlexDeMoor do I understand correctly from your PR description that performance should be improved with this PR? The rate ev/s/thd is decreased for all the NANO workflows tested

I think there are some fluctuations in the NANO tests -- let's run again and see.

hqucms · 2025-01-27T13:32:28Z

please test

cmsbuild · 2025-02-03T12:48:16Z

+code-checks

Logs: https://cmssdt.cern.ch/SDT/code-checks/cms-sw-PR-47173/43537

Found files with invalid states:
- RecoBTag/Combined/data/UParTAK4/PUPPI/V01/UParTAK4_v2.onnx:
  - Added: 4c3ec0d
  - Deleted: 570033a
There are other open Pull requests which might conflict with changes you have proposed:
- File RecoBTag/Configuration/python/RecoBTag_cff.py modified in PR(s): Adding UParT scores to offline HLT DQM #47012

cmsbuild · 2025-02-03T12:48:43Z

Pull request #47173 was updated. @cmsbuild, @ftorrresd, @hqucms, @jfernan2, @mandrenguyen can you please check and sign again.

hqucms · 2025-02-03T12:52:57Z

please test

cmsbuild · 2025-02-03T19:59:10Z

+1

Size: This PR adds an extra 28KB to repository
Summary: https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-8e554d/44157/summary.html
COMMIT: eff5b69
CMSSW: CMSSW_15_0_X_2025-02-03-1100/el8_amd64_gcc12
Additional Tests: NANO,PROFILING
User test area: For local testing, you can use /cvmfs/cms-ci.cern.ch/week1/cms-sw/cmssw/47173/44157/install.sh to create a dev area with all the needed externals and cmssw changes.

The following merge commits were also included on top of IB + this PR after doing git cms-merge-topic:

@SegmentLinking Dynamic Memory Allocation for LST Objects #47232

You can see more details here:
https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-8e554d/44157/git-recent-commits.json
https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-8e554d/44157/git-merge-result

Comparison Summary

Summary:

You potentially added 17 lines to the logs
ROOTFileChecks: Some differences in event products or their sizes found
Reco comparison results: 2949 differences found in the comparisons
DQMHistoTests: Total files compared: 50
DQMHistoTests: Total histograms compared: 4016938
DQMHistoTests: Total failures: 17640
DQMHistoTests: Total nulls: 0
DQMHistoTests: Total successes: 3999278
DQMHistoTests: Total skipped: 20
DQMHistoTests: Total Missing objects: 0
DQMHistoSizes: Histogram memory added: 0.0 KiB( 49 files compared)
Checked 218 log files, 189 edm output root files, 50 DQM output files
TriggerResults: no differences found

NANO Comparison Summary

Summary:

You potentially removed 781 lines from the logs
ROOTFileChecks: Some differences in event products or their sizes found
Reco comparison results: 1455 differences found in the comparisons
DQMHistoTests: Total files compared: 21
DQMHistoTests: Total histograms compared: 75219
DQMHistoTests: Total failures: 3188
DQMHistoTests: Total nulls: 0
DQMHistoTests: Total successes: 72031
DQMHistoTests: Total skipped: 0
DQMHistoTests: Total Missing objects: 0
DQMHistoSizes: Histogram memory added: 0.0 KiB( 20 files compared)
Checked 106 log files, 61 edm output root files, 21 DQM output files
TriggerResults: no differences found

Nano size comparison Summary:

Sample	kb/ev	ref kb/ev	diff kb/ev	ev/s/thd	ref ev/s/thd	diff rate	mem/thd	ref mem/thd
2500.001	3.108	3.114	-0.006 ( -0.2% )	6.98	6.42	+8.6%	2.576	2.550
2500.002	3.225	3.230	-0.005 ( -0.2% )	6.22	5.72	+8.7%	3.007	2.984
2500.003	3.166	3.171	-0.005 ( -0.2% )	6.52	6.01	+8.5%	2.991	2.956
2500.011	1.641	1.644	-0.003 ( -0.2% )	11.87	10.27	+15.6%	2.666	2.627
2500.012	2.179	2.184	-0.005 ( -0.2% )	6.56	6.00	+9.3%	2.843	2.818
2500.013	1.996	2.000	-0.003 ( -0.2% )	9.30	8.44	+10.3%	2.751	2.727
2500.021	0.022	0.022	0.000 ( +0.0% )	1.92	2.08	-8.1%	2.679	2.603
2500.022	0.022	0.022	0.000 ( +0.0% )	1.83	1.99	-8.5%	2.682	2.604
2500.023	0.022	0.022	0.000 ( +0.0% )	1.77	1.89	-6.8%	2.547	2.470
2500.024	0.022	0.022	0.000 ( +0.0% )	1.47	1.56	-5.9%	2.780	2.703
2500.031	0.035	0.035	0.000 ( +0.0% )	1.69	1.76	-4.0%	2.736	2.657
2500.032	0.036	0.036	0.000 ( +0.0% )	1.71	1.82	-6.0%	2.701	2.620
2500.033	0.037	0.037	0.000 ( +0.0% )	1.60	1.70	-6.0%	2.774	2.707
2500.034	0.036	0.036	0.000 ( +0.0% )	1.61	1.71	-6.1%	2.761	2.682
2500.101	2.850	2.847	0.003 ( +0.1% )	16.56	16.54	+0.1%	2.636	2.642
2500.111	1.468	1.465	0.003 ( +0.2% )	31.45	31.89	-1.4%	2.344	2.326
2500.112	1.889	1.885	0.004 ( +0.2% )	26.03	25.59	+1.7%	2.413	2.408
2500.131	0.750	0.750	0.000 ( +0.0% )	37.45	37.66	-0.6%	1.501	1.508
2500.201	2.679	2.676	0.003 ( +0.1% )	13.73	13.90	-1.2%	2.211	2.208
2500.211	1.836	1.833	0.003 ( +0.2% )	27.21	27.53	-1.2%	2.399	2.400
2500.212	2.233	2.229	0.004 ( +0.2% )	22.90	22.50	+1.8%	2.491	2.495
2500.221	2.038	2.038	0.000 ( +0.0% )	14.67	14.61	+0.5%	2.128	2.120
2500.222	3.486	3.482	0.005 ( +0.1% )	13.24	13.25	-0.1%	2.224	2.220
2500.223	9.489	9.493	-0.004 ( -0.0% )	4.87	4.30	+13.2%	2.351	2.290
2500.224	6.604	6.547	0.056 ( +0.9% )	1.29	1.42	-8.8%	2.293	2.288
2500.225	6.651	6.594	0.056 ( +0.9% )	1.20	1.31	-8.3%	2.500	2.505
2500.226	3.180	3.175	0.005 ( +0.1% )	13.69	13.76	-0.5%	2.219	2.212
2500.227	1.442	1.442	0.000 ( +0.0% )	23.97	23.99	-0.1%	1.450	1.442
2500.228	3.995	3.959	0.037 ( +0.9% )	9.15	9.24	-1.0%	2.312	2.321
2500.231	1.457	1.457	0.000 ( +0.0% )	22.72	23.00	-1.2%	2.302	2.290
2500.232	2.492	2.489	0.004 ( +0.1% )	21.23	21.47	-1.1%	2.400	2.399
2500.233	4.985	4.988	-0.003 ( -0.1% )	7.33	6.31	+16.2%	2.523	2.468
2500.234	3.918	3.884	0.034 ( +0.9% )	1.63	1.80	-9.5%	2.243	2.434
2500.235	3.949	3.916	0.034 ( +0.9% )	1.55	1.69	-8.5%	2.423	2.634
2500.236	2.282	2.278	0.004 ( +0.2% )	22.23	22.72	-2.1%	2.393	2.394
2500.237	1.018	1.018	0.000 ( +0.0% )	35.57	35.60	-0.1%	1.439	1.460
2500.238	2.468	2.466	0.003 ( +0.1% )	17.51	17.51	-0.0%	2.471	2.487
2500.241	9.404	9.404	0.000 ( +0.0% )	7.26	6.43	+12.8%	1.922	1.927
2500.242	10.331	10.331	0.000 ( +0.0% )	1.57	1.58	-0.8%	1.727	1.727
2500.243	2.712	2.712	0.000 ( +0.0% )	15.99	15.82	+1.1%	1.065	1.061
2500.244	486.016	486.016	0.000 ( +0.0% )	1.14	1.15	-0.9%	1.701	1.685
2500.245	826.413	826.413	0.000 ( +0.0% )	1.54	1.55	-0.7%	1.681	1.695
2500.251	645.314	645.314	0.000 ( +0.0% )	1.68	1.68	-0.2%	1.778	1.788
2500.901	1.819	1.819	0.000 ( +0.0% )	45.50	45.66	-0.3%	1.443	1.447
2500.902	1.665	1.665	0.000 ( +0.0% )	48.90	49.19	-0.6%	1.337	1.336
2500.911	14.345	14.345	0.000 ( +0.0% )	8.85	8.39	+5.5%	1.085	1.087
2500.912	0.240	0.240	0.000 ( +0.0% )	2.71	3.55	-23.8%	0.851	0.844
2500.913	0.110	0.110	0.000 ( +0.0% )	2.62	2.63	-0.4%	0.849	0.851

hqucms · 2025-02-03T22:26:53Z

+1

jfernan2 · 2025-02-04T10:24:53Z

+1

cmsbuild · 2025-02-04T10:25:19Z

This pull request is fully signed and it will be integrated in one of the next master IBs (tests are also fine). This pull request will now be reviewed by the release team before it's merged. @mandrenguyen, @rappoccio, @antoniovilela, @sextonkennedy (and backports should be raised in the release meeting by the corresponding L2)
Notice This PR was tested with additional Pull Request(s), please also merge them if necessary: cms-data/RecoBTag-Combined#64

mandrenguyen · 2025-02-04T15:05:13Z

+1

cmsbuild added this to the CMSSW_15_0_X milestone Jan 23, 2025

cmsbuild added reconstruction-pending pending-signatures tests-pending orp-pending code-checks-pending labels Jan 23, 2025

cmsbuild added code-checks-rejected code-checks-pending and removed code-checks-pending code-checks-rejected labels Jan 23, 2025

cmsbuild added code-checks-approved and removed code-checks-pending labels Jan 23, 2025

cmsbuild added the requires-external label Jan 24, 2025

cmsbuild added tests-started and removed tests-pending labels Jan 24, 2025

cmsbuild added tests-approved and removed tests-started labels Jan 24, 2025

cmsbuild added the xpog-pending label Jan 24, 2025

Update UParT v1 jet neut regression description in Nano

eff5b69

cmsbuild added code-checks-pending and removed code-checks-approved labels Feb 3, 2025

cmsbuild added code-checks-approved and removed code-checks-pending labels Feb 3, 2025

cmsbuild added tests-started and removed tests-pending labels Feb 3, 2025

cmsbuild added tests-approved and removed tests-started labels Feb 3, 2025

hqucms mentioned this pull request Feb 3, 2025

[JME,Nano] Refactor constituents table, add GloParT WvsQCD score #47206

Merged

cmsbuild added xpog-approved and removed xpog-pending labels Feb 3, 2025

cmsbuild added reconstruction-approved fully-signed and removed reconstruction-pending pending-signatures labels Feb 4, 2025

cmsbuild mentioned this pull request Feb 4, 2025

Add UParT v2 cms-data/RecoBTag-Combined#64

Merged

cmsbuild added orp-approved and removed orp-pending labels Feb 4, 2025

cmsbuild merged commit e767d8d into cms-sw:master Feb 4, 2025
16 checks passed

This was referenced Feb 4, 2025

libunwind master with fix for uninitialized access cms-sw/cmsdist#9639

Merged

[GCC14] template-id not allowed for constructor in C++20 #47265

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Introduce Unified Particle Transformer v2 #47173

Introduce Unified Particle Transformer v2 #47173

AlexDeMoor commented Jan 23, 2025

cmsbuild commented Jan 23, 2025 •

edited

Loading

cmsbuild commented Jan 23, 2025

cmsbuild commented Jan 23, 2025

cmsbuild commented Jan 23, 2025

hqucms commented Jan 24, 2025

hqucms commented Jan 24, 2025

cmsbuild commented Jan 24, 2025

jfernan2 commented Jan 24, 2025

hqucms commented Jan 24, 2025

cmsbuild commented Jan 24, 2025

hqucms commented Jan 27, 2025

hqucms commented Jan 27, 2025

cmsbuild commented Feb 3, 2025

cmsbuild commented Feb 3, 2025

hqucms commented Feb 3, 2025

cmsbuild commented Feb 3, 2025

hqucms commented Feb 3, 2025

jfernan2 commented Feb 4, 2025

cmsbuild commented Feb 4, 2025

mandrenguyen commented Feb 4, 2025

Introduce Unified Particle Transformer v2 #47173

Introduce Unified Particle Transformer v2 #47173

Conversation

AlexDeMoor commented Jan 23, 2025

PR description:

cmsbuild commented Jan 23, 2025 • edited Loading

cmsbuild commented Jan 23, 2025

cmsbuild commented Jan 23, 2025

cmsbuild commented Jan 23, 2025

hqucms commented Jan 24, 2025

hqucms commented Jan 24, 2025

cmsbuild commented Jan 24, 2025

Comparison Summary

NANO Comparison Summary

jfernan2 commented Jan 24, 2025

hqucms commented Jan 24, 2025

cmsbuild commented Jan 24, 2025

hqucms commented Jan 27, 2025

hqucms commented Jan 27, 2025

cmsbuild commented Feb 3, 2025

cmsbuild commented Feb 3, 2025

hqucms commented Feb 3, 2025

cmsbuild commented Feb 3, 2025

Comparison Summary

NANO Comparison Summary

hqucms commented Feb 3, 2025

jfernan2 commented Feb 4, 2025

cmsbuild commented Feb 4, 2025

mandrenguyen commented Feb 4, 2025

cmsbuild commented Jan 23, 2025 •

edited

Loading