Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

HWRF ingestion issue #41

Closed
webgisdeveloper opened this issue Nov 9, 2022 · 4 comments
Closed

HWRF ingestion issue #41

webgisdeveloper opened this issue Nov 9, 2022 · 4 comments
Assignees
@webgisdeveloper
Labels
bug Something isn't working fixed fixed issues incident something wrong from the data source question Further information is requested

Comments

@webgisdeveloper
Copy link
Contributor

HWRF data comes after MoM output is already generated.
#38 may introduce a side-effect and generate MoM outputs too early
@webgisdeveloper webgisdeveloper added improvement for better coding question Further information is requested labels Nov 9, 2022
@webgisdeveloper webgisdeveloper self-assigned this Nov 9, 2022
@webgisdeveloper webgisdeveloper mentioned this issue Nov 9, 2022
Closed
@webgisdeveloper webgisdeveloper added the inprogress working on it label Nov 9, 2022
@webgisdeveloper
Copy link
Contributor Author

it seems it is related HWRF data listing issue:
in the new HWRF, the folder is prelisted, it will fill in the data later:

Index of /data/nccf/com/hwrf/prod/hwrf.20221109

Name                      Last modified      Size  [Parent Directory](https://ftpprd.ncep.noaa.gov/data/nccf/com/hwrf/prod/)                               -   
[00/](https://ftpprd.ncep.noaa.gov/data/nccf/com/hwrf/prod/hwrf.20221109/00/)                       09-Nov-2022 06:19    -   
[06/](https://ftpprd.ncep.noaa.gov/data/nccf/com/hwrf/prod/hwrf.20221109/06/)                       09-Nov-2022 12:18    -   
[12/](https://ftpprd.ncep.noaa.gov/data/nccf/com/hwrf/prod/hwrf.20221109/12/)                       09-Nov-2022 18:17    -   
[18/](https://ftpprd.ncep.noaa.gov/data/nccf/com/hwrf/prod/hwrf.20221109/18/)                       09-Nov-2022 22:33    -

@webgisdeveloper webgisdeveloper added bug Something isn't working and removed improvement for better coding labels Nov 9, 2022
@webgisdeveloper
Copy link
Contributor Author

???
for the previous day, process
if there are rainfall data, running time +3 hour delay
if there is no rainfall data, running time (at UTC) + 8 hours

webgisdeveloper pushed a commit that referenced this issue Nov 10, 2022
#41 run full data in HWRF_tool
565ccd6
webgisdeveloper pushed a commit that referenced this issue Nov 10, 2022
#41 check if it is too early to process
d2d9391
webgisdeveloper pushed a commit that referenced this issue Nov 10, 2022
#41 use the latest hwrf summary in 16 hours
bc1244f
webgisdeveloper pushed a commit that referenced this issue Nov 10, 2022
#41 disable debug
dfb63b9
webgisdeveloper pushed a commit that referenced this issue Nov 10, 2022
#41 6 hours delay
2fa81eb
webgisdeveloper pushed a commit that referenced this issue Nov 10, 2022
#41 extra logging in hwrf
ded8ea5
@webgisdeveloper
Copy link
Contributor Author

from log, the issue is actually in DFO_MoM code
DFO shall not update HWRF output unless the DFO data is complete for the day.

webgisdeveloper added a commit that referenced this issue Nov 10, 2022
@webgisdeveloper
#41 change log position
1b8bdd8
webgisdeveloper added a commit that referenced this issue Nov 10, 2022
@webgisdeveloper
#41 fix issue in DFO_MoM
71ffc89
webgisdeveloper added a commit that referenced this issue Nov 12, 2022
@webgisdeveloper
#41 chech if hwrf has date
d80c732
webgisdeveloper added a commit that referenced this issue Nov 12, 2022
@webgisdeveloper
#41 update on hwrf today
2cb84a7
webgisdeveloper added a commit that referenced this issue Nov 12, 2022
@webgisdeveloper
#41 generate hwrf for 00 hour if no data
272bcb3
@webgisdeveloper webgisdeveloper added the incident something wrong from the data source label Nov 17, 2022
@webgisdeveloper webgisdeveloper mentioned this issue Nov 17, 2022
Closed
@webgisdeveloper
Copy link
Contributor Author

sometimes, the data is only listed for the certain hours (less than 4)

Index of /data/nccf/com/hwrf/prod/hwrf.20230102

Name                      Last modified      Size  [Parent Directory](https://ftpprd.ncep.noaa.gov/data/nccf/com/hwrf/prod/)                               -   
[00/](https://ftpprd.ncep.noaa.gov/data/nccf/com/hwrf/prod/hwrf.20230102/00/)                       02-Jan-2023 06:07    -

webgisdeveloper pushed a commit that referenced this issue Jan 10, 2023
#41 geneate mom when not data produced
1892d61
webgisdeveloper added a commit that referenced this issue Jan 10, 2023
@webgisdeveloper
#41 find out current processing datehour
e619267
webgisdeveloper added a commit that referenced this issue Jan 10, 2023
@webgisdeveloper
#41 processing mom if not hwrf data release
bbe1fe9
This was referenced Jan 10, 2023
Open
Open
webgisdeveloper pushed a commit that referenced this issue Jan 12, 2023
#41 put no data code in HWRF tool
511af80
@webgisdeveloper webgisdeveloper added fixed fixed issues and removed inprogress working on it labels Mar 24, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@webgisdeveloper webgisdeveloper

Labels
bug Something isn't working fixed fixed issues incident something wrong from the data source question Further information is requested
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
1 participant
@webgisdeveloper