Bugfix: Run forward and backwards when looking for links in PFBA

[lgray]

Ensure by brute force that all block orderings are taken care of in PFBA and links between objects are never missed.

Incurs a minor performance penalty since inner loop for link-finding is larger, minor jittery regressions expected.

This fixes the bug reported in Heavy Ions relvals.

[lgray]

@cmsbuild please test

[cmsbuild]

The tests are being triggered in jenkins.
https://cmssdt.cern.ch/jenkins/job/ib-any-integration/13346/console

[cmsbuild]

A new Pull Request was created by @lgray (Lindsey Gray) for CMSSW_8_1_X.

It involves the following packages:

RecoParticleFlow/PFProducer

@cmsbuild, @cvuosalo, @slava77, @davidlange6 can you please review it and eventually sign? Thanks.
@mmarionncern, @rafaellopesdesa, @bachtis, @cbernet this is something you requested to watch as well.
@slava77, @Degano, @smuzaffar you are the release manager for this.

cms-bot commands are list here #13028

[cmsbuild]

+1
Tested at: f2cdf47
https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-14754/13346/summary.html

[cmsbuild]

Comparison is ready
https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-14754/13346/summary.html

[lgray]

Anyone looking at this?

[slava77]

+1

for #14754 f2cdf47

• code change is in line with the description; this PR is a follow up to crashes reported in relvals in 810pre5 https://hypernews.cern.ch/HyperNews/CMS/get/relval/5014.html
• jenkins tests pass and comparisons with baseline show small differences starting from PFBlock
• same inputs as in the relval were tested in CMSSW_8_1_X_2016-05-31-1100 and lead to a crash. After changes in this PR the crash is gone.
• local tests with larger samples confirm there are only small differences without significant deviations.

timing is now worse by ~x.16 for PFBA and x1.3 for pfNoPileup (this one has N*m loop over PFCandidates which terminates sooner or later depending on how close the candidates are)
workflow 25202:

106.50 ms/ev ->       174.51 ms/ev particleFlowBlock
10.30 ms/ev ->        13.05 ms/ev pfNoPileUpIso
10.30 ms/ev ->        12.96 ms/ev pfNoPileUp
 12.32 ms/ev ->        15.46 ms/ev pfNoPileUpIsoPFBRECO
  12.43 ms/ev ->        15.40 ms/ev pfNoPileUpEI
21.43 ms/ev ->        23.76 ms/ev pfNoPileUpJMEEI
20.85 ms/ev ->        22.49 ms/ev pfNoPileUpJME

compare this to the results in the initial PR #14138 (comment) (different machines were used for testing

        particleFlowBlock        119.404 ms/ev -> 53.8104 ms/ev
        pfNoPileUpIso    19.7848 ms/ev -> 7.09613 ms/ev
        pfNoPileUp       19.7443 ms/ev -> 7.02833 ms/ev
        pfNoPileUpJME    24.8046 ms/ev -> 14.0714 ms/ev

The net effect of updates to PFBA since #14138 is still positive, roughly 0.5% of total reco time.

[cmsbuild]

This pull request is fully signed and it will be integrated in one of the next CMSSW_8_1_X IBs (tests are also fine). This pull request requires discussion in the ORP meeting before it's merged. @slava77, @davidlange6, @Degano, @smuzaffar

[lgray]

Taking a look at it in igprof/perf stat to see if there's any way to keep the total speed up.
That can some as a separate PR though.

[davidlange6]

+1