Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

JP-3719: Fix names for intermediate spectral outlier detection files #8735

Merged
merged 4 commits into from
Aug 29, 2024

Conversation

melanieclarke
Copy link
Collaborator

@melanieclarke melanieclarke commented Aug 28, 2024

Resolves JP-3719

Closes #8734

For spectral modes with resampling, intermediate files from outlier detection are not being written correctly. There should be files written to the output directory with suffix 'outlier_s2d'. Instead, there are currently files written to the current directory with suffix 'outlier_i2d', and some duplicate files written to the output directory with suffix 'outlier__outlier_s2d.fits'.

This PR should fix these issues by:

  • removing the duplicate file saving mechanism in the spectral outlier detection method, in favor of the native one in the resampler
  • modifying the resampling step to store the suffix 'i2d' for imaging and 's2d' for spectral data
  • passing the output directory to the resampler in the spectral outlier detection interface.

I also added a unit test for spectral outlier detection, along with some checks for these kinds of regressions in intermediate file saving.

Checklist for PR authors (skip items if you don't have permissions or they are not applicable)

  • added entry in CHANGES.rst within the relevant release section
  • updated or added relevant tests
  • updated relevant documentation
  • added relevant milestone
  • added relevant label(s)
  • ran regression tests, post a link to the Jenkins job below.
    How to run regression tests on a PR
  • All comments are resolved
  • Make sure the JIRA ticket is resolved properly

@melanieclarke
Copy link
Collaborator Author

melanieclarke commented Aug 28, 2024

Regression tests here:
https://github.com/spacetelescope/RegressionTests/actions/runs/10604279986

Failures are all unrelated.

Copy link

codecov bot commented Aug 28, 2024

Codecov Report

Attention: Patch coverage is 80.00000% with 2 lines in your changes missing coverage. Please review.

Project coverage is 60.64%. Comparing base (cccc198) to head (e43af51).

Files with missing lines Patch % Lines
jwst/outlier_detection/outlier_detection_step.py 0.00% 1 Missing ⚠️
jwst/resample/resample.py 66.66% 1 Missing ⚠️
Additional details and impacted files
@@            Coverage Diff             @@
##           master    #8735      +/-   ##
==========================================
+ Coverage   60.34%   60.64%   +0.29%     
==========================================
  Files         372      372              
  Lines       38374    38371       -3     
==========================================
+ Hits        23158    23269     +111     
+ Misses      15216    15102     -114     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

@melanieclarke melanieclarke marked this pull request as ready for review August 29, 2024 17:03
@melanieclarke melanieclarke requested a review from a team as a code owner August 29, 2024 17:03
@melanieclarke melanieclarke requested a review from emolter August 29, 2024 17:03
@melanieclarke
Copy link
Collaborator Author

Thanks for reviewing @emolter! Since this is a minor technical PR, I'll go ahead and merge when CI completes.

@melanieclarke melanieclarke merged commit 13c47a8 into spacetelescope:master Aug 29, 2024
29 checks passed
@melanieclarke melanieclarke deleted the jp-3719 branch August 29, 2024 21:16
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Fix intermediate filenames for spectral outlier detection
2 participants