Use a quick union algorithm to speed up PFBlockAlgo (factor of 2-3x)

[lgray]

Use a quick union algorithm + knowledge of ranges of block element types to speed up PFBlockAlgo.

Minor regressions expected due to the order-sensitivity of PFAlgo.

Regressions appear as changes in the particle flow candidate list, and so can affect PF-met and PF-jets kinematic observables. Effect is statistical in nature and should be quantifiable using a large stats sample.

[lgray]

@cmsbuild please test

[cmsbuild]

The tests are being triggered in jenkins.
https://cmssdt.cern.ch/jenkins/job/ib-any-integration/12469/console

[cmsbuild]

A new Pull Request was created by @lgray (Lindsey Gray) for CMSSW_8_1_X.

It involves the following packages:

RecoParticleFlow/PFProducer

@cmsbuild, @cvuosalo, @slava77, @davidlange6 can you please review it and eventually sign? Thanks.
@mmarionncern, @rafaellopesdesa, @bachtis this is something you requested to watch as well.
@slava77, @Degano, @smuzaffar you are the release manager for this.

cms-bot commands are list here #13028

[cmsbuild]

+1
Tested at: 11f9bef
https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-14138/12469/summary.html

[VinInn]

@fwyzard , measured factor 3 speedup of PFBlockProducer for HLT at PU22 corresponding to 4% of total time.

[slava77]

On 4/19/16 3:28 PM, Vincenzo Innocente wrote:

@fwyzard https://github.com/fwyzard , measured factor 3 speedup of
PFBlockProducer for HLT at PU22 corresponding to 4% of total time.

Hi Vincenzo,

Please post a link with HLT time instructions (or forward by email)

Thank you.

—
You are receiving this because you were mentioned.
Reply to this email directly or view it on GitHub
#14138 (comment)

[VinInn]

I have with all other changes (including PFRecHit slimming) as well (but PFBlock has been touched only here)
http://innocent.web.cern.ch/innocent/perfResults/igprof-navigator/hlt_benchmark_SlimPFHit22PU_CMSSW_8_1_X_2016-04-15-2300_slc6_amd64_gcc530/25
wrt
http://innocent.web.cern.ch/innocent/perfResults/igprof-navigator/hlt_benchmark_VeryFastHLT22PU_CMSSW_8_1_X_2016-04-13-2300_slc6_amd64_gcc530/25
(includes #14058 and #14048 as well)
or
http://innocent.web.cern.ch/innocent/perfResults/igprof-navigator/hlt_benchmark_ori22PU_CMSSW_8_0_3_patch1_slc6_amd64_gcc530/25

[VinInn]

this summary
https://twiki.cern.ch/twiki/bin/view/CMS/ViHLTPerf52016
corresponds to current IB+ #14058 + #14048

[cmsbuild]

Comparison is ready
https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-14138/12469/summary.html

[lgray]

Any movement on this?

[cvuosalo]

@lgray: Review should be done Thursday unless problems are found.
More details on the expected regressions would be helpful.

[cvuosalo]

auto_ptr has been deprecated. When adding new code, wouldn't it be better to use shared_ptr or unique_ptr instead?

[lgray]

blocksNew_ was from debugging and implementation anyway, I'll fix it.

[cvuosalo]

+1

For #14138 11f9bef

Speeding up PFBlockAlgo.

The code changes are satisfactory. Jenkins tests against baseline CMSSW_8_1_X_2016-04-18-2300 show numerous, very tiny differences at the level of statistical fluctuations in PF quantities. An extended test of workflow 25202.0_TTbar_13 with 70 events against baseline CMSSW_8_1_X_2016-04-17-0000 also shows numerous, tiny, insignificant differences in PF quantities. CPU time shrinks about 50-80% for the affected modules, though the effect on total time per event is harder to measure.

For HLT (excluding the first event):

  delta/mean delta/orJob     original                   new       module name
  ---------- ------------     --------                  ----       ------------
   -1.413673      -0.40%       207.18 ms/ev ->        35.58 ms/ev hltParticleFlowBlockReg
   -1.389132      -0.42%       218.73 ms/ev ->        39.42 ms/ev hltParticleFlowBlock
   -1.383895      -0.40%       212.92 ms/ev ->        38.77 ms/ev hltParticleFlowBlockForTaus
  ---------- ------------     --------                  ----       ------------

For RECO:

        particleFlowBlock        119.404 ms/ev -> 53.8104 ms/ev
        pfNoPileUpIso    19.7848 ms/ev -> 7.09613 ms/ev
        pfNoPileUp       19.7443 ms/ev -> 7.02833 ms/ev
        pfNoPileUpJME    24.8046 ms/ev -> 14.0714 ms/ev
         Total in detailed printout:     191.904 ms/ev -> 87.8453 ms/ev          delta: -104.058

[cmsbuild]

This pull request is fully signed and it will be integrated in one of the next CMSSW_8_1_X IBs (tests are also fine). This pull request requires discussion in the ORP meeting before it's merged. @slava77, @davidlange6, @Degano, @smuzaffar