JP-3510: Use all groups to flag bad IRS2 reference pixels

[melanieclarke]

Resolves JP-3510

Closes #8195

This PR improves on-the-fly bad reference pixel flagging by using all groups from all integrations to compile mean and standard deviation statistics. This should catch reference pixels that are intermittently bad and happened to be okay in the last group.

Additionally, this PR moves the flagging outside of the IRS2 correction method, so that it can be applied before any other reference pixel handling is done, and updates the pixel replacement method for both newly flagged bad pixels and known bad pixels. Replacement values come from the nearest reference groups, respecting parity if possible; 0.0 is used as a replacement value only as a last resort.

Checklist for maintainers

• added entry in CHANGES.rst within the relevant release section
• updated or added relevant tests
• updated relevant documentation
• added relevant milestone
• added relevant label(s)
• ran regression tests, post a link to the Jenkins job below.
  How to run regression tests on a PR
• Make sure the JIRA ticket is resolved properly

[codecov]

Codecov Report

Attention: 15 lines in your changes are missing coverage. Please review.

Comparison is base (dac66be) 75.25% compared to head (d159f51) 75.25%.

❗ Current head d159f51 differs from pull request most recent head bc149ba. Consider uploading reports for the commit bc149ba to get more accurate results

Files	Patch %	Lines
jwst/refpix/irs2_subtract_reference.py	91.50%	9 Missing ⚠️
jwst/refpix/refpix_step.py	0.00%	6 Missing ⚠️

Additional details and impacted files

@@            Coverage Diff             @@
##           master    #8197      +/-   ##
==========================================
- Coverage   75.25%   75.25%   -0.01%     
==========================================
  Files         470      470              
  Lines       38435    38466      +31     
==========================================
+ Hits        28923    28946      +23     
- Misses       9512     9520       +8     

Flag	Coverage Δ		*Carryforward flag
nightly	77.37% <ø> (+<0.01%)	⬆️	Carriedforward from 649b28f

*This pull request uses carry forward flags. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

[hbushouse]

Code updates look fine. Just a couple minor comments.

[hbushouse]

Have you done a regression test run or do you not have permission to set those up on our Jenkins server?

[melanieclarke]

@hbushouse - I have permission to see regression tests but not launch them.

[hbushouse]

Regtest run started at https://plwishmaster.stsci.edu:8081/job/RT/job/JWST-Developers-Pull-Requests/1164

[hbushouse]

Regression test results look good. Several tests involving NIRSpec IRS2 data that show expected differences starting at the refpix step, as well as several unrelated differences due to other recent mergers into the master branch.