Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

MOSAIC association candidate id redundant, includes only single observation #7725

Closed
stscijgbot-jp opened this issue Jul 11, 2023 · 16 comments · Fixed by #8798
Closed

MOSAIC association candidate id redundant, includes only single observation #7725

stscijgbot-jp opened this issue Jul 11, 2023 · 16 comments · Fixed by #8798
Labels
AsnGenerator associations mosaic Software Affected: associations team-chartreuse

Comments

@stscijgbot-jp
Copy link
Collaborator

stscijgbot-jp commented Jul 11, 2023 •
edited
Loading

Issue JP-3305 was created on JIRA by Hien Tran:

while reprocessing program 2739 it was noted that the l3 associations for the MOSAIC candidate id {}c1000{}, which as listed in the pool includes observations 10, 12, 13, each makes up of a single observation only – not all three, as would be expected for a mosaic combination. this makes all the c1000 associations redundant, since all of their products would be covered by the individual -o associations.

the targetid is different for each of the observations, likely preventing the full, 3-obs mosaic association from being made. but if such a mosaic is intended, there should be a way to make such a combined product possible.

a summary of c1000 asns, showing the productnames, number of members, and the observation in each, follows

jw02739-c1000_20230710t150016_image3_00001
jw02739-c1000_t007_nircam_clear-f335m
12
02739010


jw02739-c1000_20230710t150016_image3_00002
jw02739-c1000_t007_nircam_f444w-f470n
12
02739010


jw02739-c1000_20230710t150016_image3_00003
jw02739-c1000_t007_nircam_clear-f187n
48
02739010


jw02739-c1000_20230710t150016_image3_00004
jw02739-c1000_t007_nircam_clear-f444w
12
02739010


jw02739-c1000_20230710t150016_image3_00005
jw02739-c1000_t007_nircam_clear-f115w
48
02739010


jw02739-c1000_20230710t150016_image3_00006
jw02739-c1000_t007_nircam_clear-f200w
48
02739010


jw02739-c1000_20230710t150016_image3_00007
jw02739-c1000_t009_nircam_clear-f187n
48
02739013


jw02739-c1000_20230710t150016_image3_00008
jw02739-c1000_t009_nircam_f444w-f470n
12
02739013


jw02739-c1000_20230710t150016_image3_00009
jw02739-c1000_t009_nircam_clear-f115w
48
02739013


jw02739-c1000_20230710t150016_image3_00010
jw02739-c1000_t009_nircam_clear-f335m
12
02739013


jw02739-c1000_20230710t150016_image3_00011
jw02739-c1000_t009_nircam_clear-f444w
12
02739013


jw02739-c1000_20230710t150016_image3_00012
jw02739-c1000_t009_nircam_clear-f200w
48
02739013```
the pool file is attached
@stscijgbot-jp
Copy link
Collaborator Author

Comment by Hien Tran on JIRA:

as an example of the redundancy of c1000 asns, compare the two associations:
jw02739-o013_20230710t150016_image3_00006 and
jw02739-c1000_20230710t150016_image3_00012
for products named jw02739-{}o013{}_t009_nircam_clear-f200w and jw02739-{}c1000{}_t009_nircam_clear-f200w, respectively.

their input science members, as listed below, are identical

jw02739013001_02105_00001_nrca1_cal.fits
jw02739013001_02105_00001_nrca2_cal.fits
jw02739013001_02105_00001_nrca3_cal.fits
jw02739013001_02105_00001_nrca4_cal.fits
jw02739013001_02105_00001_nrcb1_cal.fits
jw02739013001_02105_00001_nrcb2_cal.fits
jw02739013001_02105_00001_nrcb3_cal.fits
jw02739013001_02105_00001_nrcb4_cal.fits
jw02739013001_02105_00002_nrca1_cal.fits
jw02739013001_02105_00002_nrca2_cal.fits
jw02739013001_02105_00002_nrca3_cal.fits
jw02739013001_02105_00002_nrca4_cal.fits
jw02739013001_02105_00002_nrcb1_cal.fits
jw02739013001_02105_00002_nrcb2_cal.fits
jw02739013001_02105_00002_nrcb3_cal.fits
jw02739013001_02105_00002_nrcb4_cal.fits
jw02739013001_02105_00003_nrca1_cal.fits
jw02739013001_02105_00003_nrca2_cal.fits
jw02739013001_02105_00003_nrca3_cal.fits
jw02739013001_02105_00003_nrca4_cal.fits
jw02739013001_02105_00003_nrcb1_cal.fits
jw02739013001_02105_00003_nrcb2_cal.fits
jw02739013001_02105_00003_nrcb3_cal.fits
jw02739013001_02105_00003_nrcb4_cal.fits
jw02739013001_02105_00004_nrca1_cal.fits
jw02739013001_02105_00004_nrca2_cal.fits
jw02739013001_02105_00004_nrca3_cal.fits
jw02739013001_02105_00004_nrca4_cal.fits
jw02739013001_02105_00004_nrcb1_cal.fits
jw02739013001_02105_00004_nrcb2_cal.fits
jw02739013001_02105_00004_nrcb3_cal.fits
jw02739013001_02105_00004_nrcb4_cal.fits
jw02739013001_02105_00005_nrca1_cal.fits
jw02739013001_02105_00005_nrca2_cal.fits
jw02739013001_02105_00005_nrca3_cal.fits
jw02739013001_02105_00005_nrca4_cal.fits
jw02739013001_02105_00005_nrcb1_cal.fits
jw02739013001_02105_00005_nrcb2_cal.fits
jw02739013001_02105_00005_nrcb3_cal.fits
jw02739013001_02105_00005_nrcb4_cal.fits
jw02739013001_02105_00006_nrca1_cal.fits
jw02739013001_02105_00006_nrca2_cal.fits
jw02739013001_02105_00006_nrca3_cal.fits
jw02739013001_02105_00006_nrca4_cal.fits
jw02739013001_02105_00006_nrcb1_cal.fits
jw02739013001_02105_00006_nrcb2_cal.fits
jw02739013001_02105_00006_nrcb3_cal.fits
jw02739013001_02105_00006_nrcb4_cal.fits ```

@stscijgbot-jp
Copy link
Collaborator Author

Comment by Rosa Diaz on JIRA:

Why I cannot find the APT for this program or any data in MAST? I find this very strange. Is 2739 a real program?

@stscijgbot-jp
Copy link
Collaborator Author

Comment by Tony Roman on JIRA:

2739 is a real program. It is a Director's Discretionary program for the purpose of obtaining public outreach images. Information about it is deliberately hidden at the direction of the Director's Office. The program information will be published after all of the press releases associated with this observing program have been completed.

@stscijgbot-jp
Copy link
Collaborator Author

Comment by Rosa Diaz on JIRA:

Hien Tran  Tony tells me that each of the mentioned observations are from different targets, so it is ok these are not associated. The question is now, why it got a mosaic candidate ID.

We might have to put this on hold until the data is public, however, if this is an issue that cannot wait because it is creating more work for DMS, we need to request for some members of the DMS to be able to see this proposal to further investigate the issue.

@stscijgbot-jp
Copy link
Collaborator Author

Comment by Jonathan Eisenhamer on JIRA:

Hien Tran Standard request: What was the list of candidates passed to the generator to run?

@stscijgbot-jp
Copy link
Collaborator Author

Comment by Hien Tran on JIRA:

from the log /ifs/archive/ops/jwst/store/owlmgr/processing_logs/asn_02739_generator_2023191154132.err

asn_generate /ifs/archive/ops/jwst/info/owlmgr/paths/sdp/asn_creation/pools/jw02739_20230710t150016_pool.csv --DMS -v -p /ifs/archive/ops/jwst/info/owlmgr/paths/sdp/asn_creation/asn_tables/jw02739_20230710t150016/o012_c1000/ --version-id 20230710t150016 -i o012 c1000 ```

@stscijgbot-jp
Copy link
Collaborator Author

Comment by Jonathan Eisenhamer on JIRA:

Following from Rosa Diaz question: From the association end, it is not clear how a the MOSAIC candidate comes about at all. What in the APT interface creates the MOSAIC candidate?

@stscijgbot-jp
Copy link
Collaborator Author

stscijgbot-jp commented Oct 16, 2023 •
edited
Loading

Comment by Ernie Morse on JIRA:

The mosaic association is used when tiles from a mosaic are split off into new observations.  This was first implemented in APT-75132.  The ticket quotes this PPS requirement:

PPS-2272: Mosaic Tile Removal to New Observation

When one or more tiles are removed from a mosaic observation, the PDT shall
accept Observer requests to place the affected visits in a new observation that
is associated with the original observation for data processing purposes.

The split-off tiles have a new target, centered at the offset of the tile.

@stscijgbot-jp
Copy link
Collaborator Author

Comment by Jonathan Eisenhamer on JIRA:

Thanks! At this moment, MOSAIC candidates are not handled correctly by the association rules, nor can it be handled due to the naming conventions in use. This is because of the differing targets. A higher-level meeting, such as the reformed Association meeting, will be required to determine best course of action.

@stscijgbot-jp
Copy link
Collaborator Author

Comment by Jesse Doggett on JIRA:

I filed a ticket for a similar case, JP-3453 before I saw this one, so I withdrew.  It does include copies of the association files generated in case that might be helpful.

@stscijgbot-jp
Copy link
Collaborator Author

Comment by Howard Bushouse on JIRA:

A proposed solution, discussed in an Associations meeting on Jan 22, 2024, would be to eliminate the grouping by target-id for mosaic candidates, because there are some proposals that use different target-ids for different sections/tiles of a single mosaic. Right now the different target-ids are preventing a Lev-3 association that contains all of the mosaic tiles. Removing the constraint on target-id (only for mosaic candidates) would then allow all of the mosaic tiles to be included in a single Lev-3 ASN, as desired.

@stscijgbot-jp
Copy link
Collaborator Author

Comment by Jonathan Eisenhamer on JIRA:

Note: The pure-parallels work has introduced the concept of the target id of "0", or "t0" as it would be in Level 3 naming. The "0" designation in this case is when there is no defined target. It may be possible to use this precedent to resolve the level 3 naming issue.

@tapastro tapastro mentioned this issue Sep 17, 2024
Merged
10 tasks
@stscijgbot-jp
Copy link
Collaborator Author

Comment by Tyler Pauly on JIRA:

PR has been submitted for review - an example output of the new rule is attached, demonstrating the rule type and the members of such an association for the provided pool.

@stscijgbot-jp
Copy link
Collaborator Author

Comment by Melanie Clarke on JIRA:

Fixed by #8798

@stscijgbot-jp
Copy link
Collaborator Author

Comment by Tyler Pauly on JIRA:

The fix that made it into Build 11.1 generates the expected associations that include all targets of a MOSAIC candidate (across observations), but failed to remove the duplicate associations generated under both o-type and c-type candidates. These duplicates were removed in #8843 , which will be part of Build 11.2.

@stscijgbot-jp
Copy link
Collaborator Author

Comment by Hien Tran on JIRA:

the c1000 image3 mosaics for this program were generated as expected during the latest repro with b11.1.1. fixed, closing.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
AsnGenerator associations mosaic Software Affected: associations team-chartreuse
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

JP-3305: New association candidate type to group mosaic tiles across targets tapastro/jwst
1 participant
@stscijgbot-jp