Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Forced photometry doesn't take into account 0 values in the noise and background maps #312

Closed
ajstewart opened this issue Sep 3, 2020 · 4 comments · Fixed by #317
Closed
Assignees
Labels
bug Something isn't working

Comments

@ajstewart
Copy link
Contributor

Current the forced photometry code doesn't seem to be taking into account the border of 0 values you get from the maps selavy produces (this can be in areas where there is image data present).

This causes divide by zero errors:

(vast-tools-dev) ada:~/ada1/temp/forced_phot_testing$ python forced-test.py
WARNING: FITSFixedWarning: 'datfix' made the change 'Invalid parameter values: MJD-OBS and DATE-OBS are inconsistent.
Set MJD-BEG to 58785.569143 from DATE-BEG.
Set MJD-END to 58785.578705 from DATE-END'. [astropy.wcs.wcs]
Removing %i sources that are outside of the image range 0
/suphys/aste7152/.conda/envs/vast-tools-dev/lib/python3.8/site-packages/astropy/units/quantity.py:477: RuntimeWarning: divide by zero encountered in true_divide
  result = super().__array_ufunc__(function, method, *arrays, **kwargs)
/suphys/aste7152/.conda/envs/vast-tools-dev/lib/python3.8/site-packages/astropy/units/quantity.py:477: RuntimeWarning: invalid value encountered in reduce
  result = super().__array_ufunc__(function, method, *arrays, **kwargs)
/suphys/aste7152/.conda/envs/vast-tools-dev/lib/python3.8/site-packages/astropy/units/quantity.py:477: RuntimeWarning: invalid value encountered in true_divide
  result = super().__array_ufunc__(function, method, *arrays, **kwargs)
/import/ada1/aste7152/software/development/forced_phot/forced_phot/__init__.py:807: RuntimeWarning: divide by zero encountered in true_divide
  out = fitter(g, xx[good], yy[good], d[good], weights=1.0 / n[good])
/import/ada1/aste7152/software/development/forced_phot/forced_phot/__init__.py:809: RuntimeWarning: divide by zero encountered in true_divide
  chisq = np.zeros(len(X0)) + (((d[good] - model[good]) / n[good]) ** 2).sum()

One simple fix is to mask these at the data load stage with NaN's so they are caught later on in the NaN check, what do you think @dlakaplan?

        self.bgdata = self.fb[0].data.squeeze()
        self.noisedata = self.fn[0].data.squeeze()
        self.bgdata[self.bgdata == 0] = np.nan
        self.noisedata[self.noisedata == 0] = np.nan
@ajstewart ajstewart added the bug Something isn't working label Sep 3, 2020
@dlakaplan
Copy link

I hadn't encountered this. I think filling with nan is good. I can do that later today.

@dlakaplan
Copy link

However, the data may be entirely valid even if the bg map is 0. is it better to set the values to NaN or to set them to e.g., 0 for the bg and the median noise value for the noise?

@dlakaplan
Copy link

OK, I think this is pushed to my version. And the try/except for #298 as well. But I don't have the files to test so please do that and let me know

@ajstewart
Copy link
Contributor Author

Thanks @dlakaplan, these fixes solve the problems.

ajstewart added a commit that referenced this issue Sep 4, 2020
ajstewart added a commit that referenced this issue Sep 5, 2020
srggrs added a commit that referenced this issue Sep 9, 2020
* master: (126 commits)
  autoformat
  added css and js sourcemaps
  wrapped endpoints in a DRF ViewSet
  remove unused import
  Removed extra line
  Improved suppression and added user option
  made source query card collapsible
  fix reset query form button Now clears all fields and removes validation classes.
  rewrote sesame and coord validator API to use DRF
  use minified datatables-pipeline script
  Filtered out Astropy warnings
  Fixes for #298 and #312
  fix migration file 0001_initial.py
  query form redesign, sesame resolver, coord validator Redesigned the source query form layout. Generalized the object name resolution to use Sesame via Astropy. Created an API endpoint for Sesame queries and coord validation. Upon successful Sesame name resolution, the coord input box is filled with the result. Upon failure, the exception message is shown to the user as a form validation error. 2-epoch metric query fields added but are disabled until PR #305 is merged. Closes #166.
  added new GET params to docstring
  Remove other branch changes
  Fixed measurement count display
  Fixed default no_forced value
  Changed JS9 overlay behaviour
  added currently working tests
  ...

# Conflicts:
#	pipeline/management/commands/runpipeline.py
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working
Projects
None yet
Development

Successfully merging a pull request may close this issue.

2 participants