Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

deprecate nlow nhigh, fix incorrect spec values in docs #8603

Merged
merged 6 commits into from
Jun 26, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 4 additions & 0 deletions CHANGES.rst
Original file line number Diff line number Diff line change
Expand Up @@ -253,6 +253,10 @@ outlier_detection
- Fixed a bug that led to small total flux offsets between input and blotted
images if the nominal and actual wcs-computed pixel areas were different. [#8553]

- Deprecate ``nlow`` and ``nhigh`` parameters, which no longer have an effect. [#8603]

- Fix errors in documentation describing arguments. [#8603]

pathloss
--------

Expand Down
64 changes: 33 additions & 31 deletions docs/jwst/outlier_detection/arguments.rst
Original file line number Diff line number Diff line change
@@ -1,80 +1,81 @@
.. _outlier_detection_step_args:

For more details about step arguments (including datatypes, possible values
and defaults) see :py:obj:`jwst.outlier_detection.OutlierDetectionStep.spec`.

Step Arguments for Non-IFU data
===============================
The `outlier_detection` step for non-IFU data has the following optional arguments
that control the behavior of the processing:

``--weight_type`` (string, default='exptime')
The type of data weighting to use during resampling;
options are 'exptime', 'error', and 'None'.
``--weight_type``
The type of data weighting to use during resampling.

``--pixfrac`` (float, default=1.0)
``--pixfrac``
The pixel fraction used during resampling;
valid values go from 0.0 to 1.0.

``--kernel`` (string, default='square')
``--kernel``
The form of the kernel function used to distribute flux onto a
resampled image. Options are 'square', 'turbo', 'point', and
'lanczos'.
resampled image.

``--fillval`` (string, default='INDEF')
``--fillval``
The value to assign to resampled image pixels that have zero weight or
do not receive any flux from any input pixels during drizzling.
Any floating-point value, given as a string, is valid.
A value of 'INDEF' will use the last zero weight flux.

``--nlow`` (integer, default=0)
The number of low values in each pixel stack to ignore
when computing the median value.
``--nlow``
Deprecated and has no effect. This parameter will be removed
in a future version.

``--nhigh`` (integer, default=0)
The number of high values in each pixel stack to ignore
when computing the median value.
``--nhigh``
Deprecated and has no effect. This parameter will be removed
in a future version.

``--maskpt`` (float, default=0.7)
``--maskpt``
The percent of maximum weight to use as lower-limit for valid data;
valid values go from 0.0 to 1.0.

``--snr`` (string, default='4.0 3.0')
``--snr``
The signal-to-noise values to use for bad pixel identification.
Since cosmic rays often extend across several pixels the user
must specify two cut-off values for determining whether a pixel should
be masked: the first for detecting the primary cosmic ray, and the
second (typically lower threshold) for masking lower-level bad pixels
adjacent to those found in the first pass. Valid values are a pair of
floating-point values in a single string.
floating-point values in a single string (for example "5.0 4.0").

``--scale`` (string, default='0.5 0.4')
``--scale``
The scaling factor applied to derivative used to identify bad pixels.
Since cosmic rays often extend across several pixels the user
must specify two cut-off values for determining whether a pixel should
be masked: the first for detecting the primary cosmic ray, and the
second (typically lower threshold) for masking lower-level bad pixels
adjacent to those found in the first pass. Valid values are a pair of
floating-point values in a single string.
floating-point values in a single string (for example "1.2 0.7").

``--backg`` (float, default=0.0)
``--backg``
User-specified background value to apply to the median image.

``--rolling_window_width`` (int, default=25)
``--rolling_window_width``
Number of integrations over which to take the median when using rolling-window
median for TSO observations.

``--save_intermediate_results`` (boolean, default=False)
``--save_intermediate_results``
Specifies whether or not to save any intermediate products created
during step processing.

``--resample_data`` (boolean, default=True)
``--resample_data``
Specifies whether or not to resample the input images when
performing outlier detection.

``--good_bits`` (string, default="~DO_NOT_USE")
``--good_bits``
The DQ bit values from the input image DQ arrays
that should be considered 'good' when building the weight mask. See
DQ flag :ref:`dq_parameter_specification` for details.

``--allowed_memory`` (float, default=None)
``--allowed_memory``
Specifies the fractional amount of
free memory to allow when creating the resampled image. If ``None``, the
environment variable ``DMODEL_ALLOWED_MEMORY`` is used. If not defined, no
Expand All @@ -84,7 +85,7 @@ that control the behavior of the processing:
For example, if set to ``0.5``, only resampled images that use less than half
the available memory can be created.

``--in_memory`` (boolean, default=False)
``--in_memory``
Specifies whether or not to load and create all images that are used during
processing into memory. If ``False``, input files are loaded from disk when
needed and all intermediate files are stored on disk, rather than in memory.
Expand All @@ -94,19 +95,20 @@ Step Arguments for IFU data
The `outlier_detection` step for IFU data has the following optional arguments
that control the behavior of the processing:

``--kernel_size`` (string, default='7 7')
``--kernel_size``
The size of the kernel to use to normalize the pixel differences. The kernel size
must only contain odd values.
must only contain odd values. Valid values are a pair of ints in a single string
(for example "7 7").

``--threshold_percent`` (float, default=99.8)
``--threshold_percent``
The threshold (in percent) of the normalized minimum pixel difference used to identify bad pixels.
Pixels with a normalized minimum pixel difference above this percentage are flagged as a outlier.

``--save_intermediate_results`` (boolean, default=False)
``--save_intermediate_results``
Specifies whether or not to save any intermediate products created
during step processing.

``--in_memory`` (boolean, default=False)
``--in_memory``
Specifies whether or not to load and create all images that are used during
processing into memory. If ``False``, input files are loaded from disk when
needed and all intermediate files are stored on disk, rather than in memory.
2 changes: 0 additions & 2 deletions docs/jwst/outlier_detection/outlier_detection_imaging.rst
Original file line number Diff line number Diff line change
Expand Up @@ -69,8 +69,6 @@ Specifically, this routine performs the following operations:

* The median image is created by combining all grouped mosaic images or
non-resampled input data (as planes in a ModelContainer) pixel-by-pixel.
* The ``nlow`` and ``nhigh`` parameters specify how many low and high values
to ignore when computing the median for any given pixel.
* The ``maskpt`` parameter sets the percentage of the weight image values to
use, and any pixel with a weight below this value gets flagged as "bad" and
ignored when resampled.
Expand Down
4 changes: 1 addition & 3 deletions docs/jwst/outlier_detection/outlier_detection_tso.rst
Original file line number Diff line number Diff line change
Expand Up @@ -21,8 +21,6 @@ a few variations to accomodate the nature of these 3D data.
* The ``rolling_window_width`` parameter specifies the number of integrations over
which to compute the median. The default is 25. If the number of integrations
is less than or equal to ``rolling_window_width``, a simple median is used instead.
* The ``nlow`` and ``nhigh`` parameters specify how many low and high values
to ignore when computing the median for any given pixel.
* The ``maskpt`` parameter sets the percentage of the weight image values to
use, and any pixel with a weight below this value gets flagged as "bad" and
ignored when the median is taken.
Expand All @@ -38,4 +36,4 @@ a few variations to accomodate the nature of these 3D data.
#. The input data model DQ arrays are updated with the mask of detected outliers.


.. automodapi:: jwst.outlier_detection.outlier_detection_tso
.. automodapi:: jwst.outlier_detection.outlier_detection_tso
4 changes: 2 additions & 2 deletions jwst/outlier_detection/outlier_detection_step.py
Original file line number Diff line number Diff line change
Expand Up @@ -55,8 +55,8 @@ class OutlierDetectionStep(Step):
pixfrac = float(default=1.0)
kernel = string(default='square') # drizzle kernel
fillval = string(default='INDEF')
nlow = integer(default=0)
nhigh = integer(default=0)
nlow = integer(default=0) # DEPRECATED this setting has no effect and will be removed
nhigh = integer(default=0) # DEPRECATED this setting has no effect and will be removed
maskpt = float(default=0.7)
snr = string(default='5.0 4.0')
scale = string(default='1.2 0.7')
Expand Down