Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add GNSSRO bufr2ioda and json files #1448

Merged
merged 10 commits into from
Feb 19, 2025
Merged

Conversation

XuanliLi-NOAA
Copy link
Collaborator

Description

Add JSON and bufr2ioda python scripts for GNSSRO data. Each GNSSRO mission has a separate ioda obs file.

Companion PRs

N/A

Issues

Refs #750

Automated CI tests to run in Global Workflow

  • atm_jjob
  • C96C48_ufs_hybatmDA
  • C96C48_hybatmaerosnowDA
  • C48mx500_3DVarAOWCDA
  • C48mx500_hybAOWCDA
  • C96C48_hybatmDA

@XuanliLi-NOAA XuanliLi-NOAA self-assigned this Jan 16, 2025
@CoryMartin-NOAA
Copy link
Contributor

Should these go in here or in spoc? @emilyhcliu what are your thoughts?

@emilyhcliu
Copy link
Collaborator

@CoryMartin-NOAA
The json and Python scripts in this PR are written in bufr-query low-level Python API.
These are good to use with the current setup in GDASApp. So, they should be checked in here in GDASApp so Xuanli can keep track of changes and repeat her tests.

In the meanwhile, @nicholasesposito will create these low-level Pythons into one Python script with high-level API and mapping files. These will be checked into SPOC.

Once we have most of the Python Script, mapping files in SPOC, we will work on GDASApp to use them from SPOC.

@CoryMartin-NOAA
Copy link
Contributor

ok sounds good thanks @emilyhcliu I just wanted to double check

Copy link
Contributor

@CoryMartin-NOAA CoryMartin-NOAA left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

a few minor comments, and I didn't closely look at the converters but assume they have been thoroughly tested

@emilyhcliu
Copy link
Collaborator

@XuanliLi-NOAA one question.
Is it possible to write one Python to convert all platforms?
GPSRO data is in one subset. It should be possible to write one Python to handle all platforms.
Are these platforms using the same BUFR table?

No worries, what you have in this PR is OK. But. we should try to create one Python script with high-level Python API for GPSRO.
@nicholasesposito will work with you on this.

@XuanliLi-NOAA
Copy link
Collaborator Author

@emilyhcliu , it would be great to have one Python script to convert all platforms. Yes, let's do that with the with high-level Python API.

@emilyhcliu
Copy link
Collaborator

@emilyhcliu , it would be great to have one Python script to convert all platforms. Yes, let's do that with the with high-level Python API.

Do all GPSRO platform use the same bufr table?

@XuanliLi-NOAA
Copy link
Collaborator Author

XuanliLi-NOAA commented Jan 17, 2025

@emilyhcliu, yes, for the test date 00z 2024-02-19, they use the same bufr table. But after June 2024, we have new EUMETSAT Spire data, this has an additional BUFR mnemonic "SASBID". Should we consider that in the high level python API?

@XuanliLi-NOAA Yes, we should include the new one. It is good that all platforms share the same bufr table mnemonics. We can have one mapping file to handle all of them. So, the one Python script with high-level API should be doable for GPSRO.
Thanks for the information.

@emilyhcliu Sounds good, thank you for confirming that.

Copy link
Collaborator

@nicholasesposito nicholasesposito left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Recommend changing the global attributes, but @emilyhcliu should comment on that. The rest looked fine with the exception of I think changing the data_format description.

@XuanliLi-NOAA
Copy link
Collaborator Author

XuanliLi-NOAA commented Jan 24, 2025

@emilyhcliu and @nicholasesposito data_provider has been corrected in the json files to reflect the actual originating center. data_format has been changed to dataOriginalFormatSpec in the python scripts. I'm not sure if we should replace subsets with platforms, as subsets is a bufr message for the name of the subset.

Copy link
Collaborator

@nicholasesposito nicholasesposito left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think just clean up some of the comments unless they are needed. Otherwise no glaring issues from me. Looks very complicated so thank you for taking the time to do all this!

@XuanliLi-NOAA
Copy link
Collaborator Author

XuanliLi-NOAA commented Jan 31, 2025

@nicholasesposito @emilyhcliu Thank you so much, Nick. I cleaned up the python scripts and also uncommented mefr1 to maintain consistency with the other variables.

Copy link
Collaborator

@nicholasesposito nicholasesposito left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Approved.

@CoryMartin-NOAA CoryMartin-NOAA merged commit bec351a into develop Feb 19, 2025
5 checks passed
@CoryMartin-NOAA CoryMartin-NOAA deleted the feature/gnssro_bufr2ioda branch February 19, 2025 20:24
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants