Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

JP-3733 remove setting pixel_replace.save to True in calwebb_spec3 #8765

Merged
merged 3 commits into from
Sep 18, 2024

Conversation

jemorrison
Copy link
Collaborator

@jemorrison jemorrison commented Sep 10, 2024

Resolves JP-3733

Closes #8754

This PR addresses removes saving the pixel_replace step output by default. The user must set pixel_replace.save_results=True to get the output written to disk.

Tasks

@jemorrison
Copy link
Collaborator Author

@tapastro Working on trying to figure out how to run regression tests in new system

Copy link

codecov bot commented Sep 10, 2024

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 61.76%. Comparing base (120ee5a) to head (9aef412).

Additional details and impacted files
@@           Coverage Diff           @@
##           master    #8765   +/-   ##
=======================================
  Coverage   61.75%   61.76%           
=======================================
  Files         377      377           
  Lines       38749    38742    -7     
=======================================
- Hits        23931    23929    -2     
+ Misses      14818    14813    -5     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.


# Overriding the Step.save_model method for the following steps.
# These steps save intermediate files, resulting in meta.filename
# being modified. This can affect the filenames of subsequent
# steps.
self.outlier_detection.save_model = invariant_filename(self.outlier_detection.save_model)
self.pixel_replace.save_model = invariant_filename(self.pixel_replace.save_model)
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is this line still needed? Say, if the user manually sets save_results = True, will the subsequent output filenames be affected?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I did not find any difference in the filenames (when pixel_replace.save_results=True) in having this line and or removed.
Since it is not doing any harm maybe we should leave it. Removing it may have unintended effects.
I will add it back in

@jemorrison jemorrison force-pushed the JP-3733_pixel_replace branch 2 times, most recently from 5d2c37b to 4e5d94b Compare September 17, 2024 00:57
@jemorrison jemorrison force-pushed the JP-3733_pixel_replace branch from 4e5d94b to 9aef412 Compare September 17, 2024 20:22
@jemorrison
Copy link
Collaborator Author

jemorrison commented Sep 17, 2024

@melanieclarke @tapastro
The regression test failures all are related to a different PR. I believe we can go ahead and merge this. But I can also re-run the regression test once the regression suite is cleaned up.
https://plwishmaster.stsci.edu:8081/job/RT/job/JWST-Developers-Pull-Requests/1719/

Copy link
Collaborator

@melanieclarke melanieclarke left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM!

Testing locally on MIRI LRS data, which runs pixel_replace by default, no file is saved by default. Manually setting save_results=True for pixel_replace does save the file as expected.

@melanieclarke melanieclarke merged commit fcd3d43 into spacetelescope:master Sep 18, 2024
29 checks passed
@jemorrison jemorrison deleted the JP-3733_pixel_replace branch November 19, 2024 15:23
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

pixel_replace generating files in OPS during Spec3Pipeline
3 participants