Introduce autoNANO #42238
Introduce autoNANO #42238
Conversation
|
+code-checks Logs: https://cmssdt.cern.ch/SDT/code-checks/cms-sw-PR-42238/36255
|
|
A new Pull Request was created by @simonepigazzini for master. It involves the following packages:
@perrotta, @rappoccio, @vlimant, @cmsbuild, @simonepigazzini, @fabiocos, @davidlange6 can you please review it and eventually sign? Thanks. cms-bot commands are listed here |
|
enable nano |
|
please test |
|
-1 Failed Tests: RelVals-INPUT RelVals-INPUT
Expand to see more relval errors ...Comparison SummarySummary:
NANO Comparison SummarySummary:
Nano size comparison Summary:
|
|
please test |
|
+1 Summary: https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-b20932/33845/summary.html Comparison SummaryThere are some workflows for which there are errors in the baseline: Summary:
NANO Comparison SummarySummary:
Nano size comparison Summary:
|
|
please test |
|
+1 Summary: https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-b20932/35500/summary.html Comparison SummarySummary:
NANO Comparison SummarySummary:
Nano size comparison Summary:
|
|
+1 |
|
+1 |
cmsbuild
added
operations-approved
fully-signed
orp-approved
and removed
operations-pending
pending-signatures
orp-pending
labels
|
This pull request is fully signed and it will be integrated in one of the next master IBs (tests are also fine). This pull request will be automatically merged. |
| @@ -1776,9 +1777,30 @@ def prepare_NANO(self, stepSpec = '' ): | |||
| print(f"in prepare_nano {stepSpec}") | |||
| ''' Enrich the schedule with NANO ''' | |||
| _,_nanoSeq,_nanoCff = self.loadDefaultOrSpecifiedCFF(stepSpec,self.NANODefaultCFF,self.NANODefaultSeq) | |||
| self.scheduleSequence(_nanoSeq,'nanoAOD_step') | |||
| custom = "nanoAOD_customizeCommon" | |||
| self._options.customisation_file.insert(0,'.'.join([_nanoCff,custom])) | |||
There was a problem hiding this comment.
NB : inserting at 0, versus appending has undesirable effects. #44908 helps once all is in autoNano
PR description:
Introduce a new utility to configure the NANO step using a mapping technique similar to the one used by DQM to load only selected modules.
autoNANO allows one to customize the NANO step using a syntax like:
-s NANO:@PHYS+@L1Fullwhere the@is then mapped to sequences and customize function and loaded into the processing. Previously we used the standard-s NANO:my_cff.my_sq1+sq2syntax to customize the NANO step, this is still supported, but the two syntaxes cannot be mixed.We also introduce a different mapping function than the one used by DQM, mainly because the latter uses a list of lists to specify different modules (pre, main, post) while we prefer to use a dictionary to avoid ordering related bugs (we can discuss if we actually want to port the DQM implementation to use a dictionary instead of a list of lists as well).
In this PR we insert a minimal set of workflows into autoNANO, the list will be expanded soon collecting the other existing workflow that are currently configured in a different way.
PR validation:
Ran few test jobs for all possible combination. Successfully reproduced PromptNano 2023 and standard NANO ntuples.
No backport foreseen at the moment.
Further details on the autoNANO syntax
The default NANO sequence (which includes a "default customize function") is loaded by default if no step modifer is specified (
-s NANO) or if the mapping points to empty strings (`'PHYS': {'sequence': '', 'customize': ''},). Missing dictionary keys are instead used to specify a workflow that do not require a dedicated "sequence" or "customize".