Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[develop] Integrate 'online-cmaq' into 'develop' branch: Part 3, workflow set-up and run scripts #613

Merged
merged 140 commits into from
Feb 15, 2023
Merged

[develop] Integrate 'online-cmaq' into 'develop' branch: Part 3, workflow set-up and run scripts #613

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
140 commits
Select commit Hold shift + click to select a range
5b63007
Enable air quality modeling online-cmaq
chan-hoo Oct 26, 2022
e2ac87a
edit codeowners file
chan-hoo Oct 26, 2022
686a805
Merge branch 'develop' of https://github.com/ufs-community/ufs-srweat…
chan-hoo Oct 26, 2022
333ae3a
task name change and add sample script for post_stat
chan-hoo Oct 26, 2022
31cd8a7
Merge branch 'develop' of https://github.com/ufs-community/ufs-srweat…
Oct 27, 2022
41c451f
fix restart issue on cycled runs in nco mode
Oct 28, 2022
f0adc6e
Merge branch 'develop' of https://github.com/ufs-community/ufs-srweat…
Oct 28, 2022
1a427ed
update path to point source data on wcoss2
Oct 31, 2022
2574ab1
update templates for GFS_v15p2 with NRT
Oct 31, 2022
f9ab605
Merge branch 'develop' of https://github.com/ufs-community/ufs-srweat…
Oct 31, 2022
61ddb12
update hash of nexus
Nov 1, 2022
f5db563
remove if-statements from aqm.rc
chan-hoo Nov 1, 2022
0b5d04f
Merge branch 'develop' of https://github.com/ufs-community/ufs-srweat…
chan-hoo Nov 2, 2022
9a3c058
Add EXTRN_MDL_OFFSET_HRS to sample scripts
chan-hoo Nov 3, 2022
c6eb1ad
Add time-offset capability
chan-hoo Nov 7, 2022
71ba5b8
Merge branch 'develop' of https://github.com/ufs-community/ufs-srweat…
chan-hoo Nov 7, 2022
e86269c
update module files for aqm on wcoss2
chan-hoo Nov 7, 2022
4bb4e11
restore missing parm while updating
Nov 7, 2022
d4a687e
remove native from xml for dev_transer on wcoss2
Nov 7, 2022
282915d
add deepcopy to retrieve_data.py
Nov 7, 2022
a866c00
correct data locations on wcoss2
Nov 7, 2022
02bc2b7
update sample script for real-time run
Nov 7, 2022
c4fe435
replace cyc with hh in data location on wcoss2
Nov 8, 2022
3178f6a
make lbcs condition general in retrieve_data.py
Nov 9, 2022
1074481
Merge branch 'develop' of https://github.com/ufs-community/ufs-srweat…
chan-hoo Nov 9, 2022
42a35fe
update hash of ufs weather model
chan-hoo Nov 9, 2022
3ef50eb
Merge branch 'develop' of https://github.com/ufs-community/ufs-srweat…
chan-hoo Nov 10, 2022
5c21a46
add nexus_post_split task
Nov 16, 2022
56e2931
remove nexus no-split option
Nov 16, 2022
41efe40
make num_split_nexus=1 work
Nov 16, 2022
585ac13
change dependency of run_fcst
Nov 16, 2022
e66eeff
turn fensha off in sample rrfs_conus_13km
Nov 17, 2022
4b97743
[online-cmaq] Fix/canopy fengsha config (#449)
zmoon Nov 17, 2022
a657e1e
Merge branch 'develop' of https://github.com/ufs-community/ufs-srweat…
Nov 17, 2022
5d16018
update diag_table_aqm to remove warnings
Nov 17, 2022
0ab0e8b
change back to years for fv3_history
Nov 17, 2022
8be1606
add a task for RAVE fire emission
Nov 19, 2022
6990782
fix unbound variable on post_stat due to missing command
Nov 21, 2022
e46aedc
move pt_src config to machine files
Nov 23, 2022
fcdf0de
[online-cmaq] Feature/nexus post fix (#494)
bbakernoaa Nov 30, 2022
e444616
update num_nexus_split in sample scripts
Nov 30, 2022
1bc57d0
Merge branch 'develop' of https://github.com/ufs-community/ufs-srweat…
Nov 30, 2022
d961caa
update hash of ufs weather model and associated scripts
Nov 30, 2022
8ac2d71
allow arbitrary numbers to restart_interval
Dec 2, 2022
c233676
remove error check for restart_interval
Dec 5, 2022
fafbdcd
update gfs version
Dec 6, 2022
fa2f495
replace post_stat and add pre_post_stat
Dec 8, 2022
ce3cb7c
change same to c3b for wgrib2
Dec 8, 2022
2ab01b3
fix missing for-loop
Dec 8, 2022
deab79e
add flag for BUILD_POST_STAT
chan-hoo Dec 9, 2022
22ced19
fix fire_file_fp and typo
chan-hoo Dec 9, 2022
d099d9f
update hash of aqm_utils
Dec 10, 2022
424719b
update modules for aqm_utils
Dec 10, 2022
2c4db89
update bias-correction scripts
Dec 11, 2022
931fa60
update bias-correction scripts
Dec 12, 2022
f6b110a
fix missing command and parm path
Dec 12, 2022
bbf9956
add mem to bias-correction xml
Dec 12, 2022
740463d
fix typo
Dec 12, 2022
ca9e0a2
remove use of 2020 data from nexus-emission
Dec 12, 2022
469dd94
change path to aqm_lbcs in sample
Dec 12, 2022
28b2944
add gfs_v16.3 to file list on hpss
Dec 12, 2022
e769b69
exclude load build modules for nexus
Dec 13, 2022
3e1f48e
split module list for each component
Dec 14, 2022
7b942b3
fix moule list issue on wcoss2
Dec 14, 2022
5482060
copy ufs_common to extrn_mod_dir
Dec 14, 2022
15eb7f4
fix missing link issue on bias-correction
Dec 14, 2022
63ed133
Merge branch 'develop' of https://github.com/ufs-community/ufs-srweat…
chan-hoo Dec 15, 2022
7e5bb4d
add missing nco module to aqm_ics on hera
chan-hoo Dec 15, 2022
034b79f
add nexus_post_split to module load list
Dec 15, 2022
62a688f
add nexus_gfs_sfc task
Dec 16, 2022
dfd6e90
add module file for nexus_gfs_sfc on hera
Dec 16, 2022
66e2a91
add missing env line to xml
Dec 17, 2022
ed6e9aa
add link to gfs data to nexus_emission
Dec 17, 2022
92650bb
add use_gfs_sfc flag to nexus_emission
Dec 17, 2022
5b6e2e8
add missing output copy
Dec 17, 2022
010046b
modify diag_table to add o3_ave for post-stat and bias-correction
Dec 18, 2022
ccac0b4
update hash of ufs weather model
Dec 18, 2022
0025890
Merge branch 'develop' of https://github.com/ufs-community/ufs-srweat…
Dec 19, 2022
021e2e2
Merge branch 'develop' of https://github.com/ufs-community/ufs-srweat…
Dec 19, 2022
2204e57
Merge branch 'online-cmaq' of https://github.com/ufs-community/ufs-sr…
Dec 19, 2022
95a2c52
update hash of aqm-utils
Dec 19, 2022
0c05183
add local_dir to nexus_gfs_sfc for real-time run
Dec 20, 2022
0235b36
remove duplicate
Dec 20, 2022
aac0173
remove aqm_rc_tmpl_fn from sample scripts
Dec 20, 2022
55b3248
turn off run_post in sample scripts
Dec 21, 2022
725dd75
[online-cmaq] Adding changes to use GFS driven biogenics (#523)
bbakernoaa Dec 21, 2022
974cf88
update module version for wcoss2
Dec 22, 2022
2c32f3e
update module version for hera
chan-hoo Dec 22, 2022
2567351
fix ncap2 issue on hera
chan-hoo Dec 22, 2022
3ef2441
[Online-cmaq] Feature/dust aqmrc (#520)
bbakernoaa Dec 22, 2022
a43a9b0
remove pt from aqm.rc and update sample scripts
chan-hoo Dec 22, 2022
3027910
clean up sample scripts
Jan 3, 2023
f12cb5e
[online-cmaq] Fix incorrect date in NEXUS emission (#530)
bbakernoaa Jan 6, 2023
05c5ca4
Enable varying fcst_len in multiple cycles
Jan 11, 2023
0bde866
Merge branch 'develop' of https://github.com/ufs-community/ufs-srweat…
Jan 11, 2023
4d9f79f
remove duplicates
Jan 11, 2023
97a0014
fix file transfer in commnity mode and minor update
Jan 11, 2023
d811d7e
move external_aqm.cfg and reduce build steps
Jan 12, 2023
98235db
back to previous hash of ufs weather app
Jan 12, 2023
7357820
reviewers comments on ufs-srw-app
Jan 14, 2023
f9caf9e
fix typo
Jan 14, 2023
1b372ba
separate ufs dependency for fv3-lam
Jan 16, 2023
45b066a
Merge branch 'develop' of https://github.com/ufs-community/ufs-srweat…
Jan 17, 2023
18be4ac
add codeowners to gitignore
Jan 17, 2023
d75705b
fix typo
Jan 17, 2023
d1aebf4
remove gitignore change
Jan 17, 2023
2a19b16
[online-cmaq] Add point sources changes needed for workflow (#524)
bbakernoaa Jan 18, 2023
dfc8d1d
correct point-source pr issues
Jan 18, 2023
2c6f428
change branch of ufs weather model for canopy
Jan 19, 2023
5fdee93
rebuild externals and sample config files
Jan 24, 2023
4997506
add dev switch for real-time
chan-hoo Jan 25, 2023
5863c06
add missing module for pre_post_stat on hera
chan-hoo Jan 25, 2023
7f6e21b
update hash of aqm-utils
Feb 1, 2023
1ec72e7
store bias-correction training data
Feb 2, 2023
56c8170
Merge branch 'develop' of https://github.com/ufs-community/ufs-srweat…
Feb 2, 2023
2c7928b
add diag field tables for gfs_v17_p8
Feb 2, 2023
0802062
add gfs_v17_p8 to scripts
Feb 2, 2023
1e8787b
separate gfs_v16 from update
Feb 2, 2023
277ce48
add gfs_v17_p8 to build list
Feb 2, 2023
004d763
add gfs_v17_p8 to scripts
Feb 3, 2023
bc04485
update field_table for gfs_v17_p8
Feb 7, 2023
466f80b
reduce n_sponge for gfs_v17_p8
Feb 7, 2023
3d1e9a1
increase k_split for 3km domain
chan-hoo Feb 8, 2023
7c43cc2
Merge branch 'develop' of https://github.com/ufs-community/ufs-srweat…
chan-hoo Feb 8, 2023
73d9c13
add missing endif
chan-hoo Feb 8, 2023
5c563dc
add ccpp option to aqm.rc
chan-hoo Feb 10, 2023
8764a47
Merge branch 'develop' of https://github.com/ufs-community/ufs-srweat…
chan-hoo Feb 10, 2023
ef7050f
Merge branch 'develop' of https://github.com/ufs-community/ufs-srweat…
chan-hoo Feb 10, 2023
7ff7157
clean up
chan-hoo Feb 10, 2023
3e24b1d
update valid_param_vals
chan-hoo Feb 11, 2023
7ad2712
change ccpp v17_p8 to v16 in sample scripts
chan-hoo Feb 11, 2023
a9da112
fix typo in create_model_configure_file
chan-hoo Feb 11, 2023
c51b3b7
change to meet reviewer comments
chan-hoo Feb 12, 2023
6d7329f
add upp-aqm to parm
chan-hoo Feb 12, 2023
73f83f1
add pdym and flag for airnow data
chan-hoo Feb 12, 2023
7eb20cd
fix typo
chan-hoo Feb 12, 2023
c054e82
change path to airnow history data on wcoss2
chan-hoo Feb 13, 2023
f9457e5
remove commented out lines
chan-hoo Feb 13, 2023
3466c95
activte opsroot in sample
chan-hoo Feb 13, 2023
234970d
Merge branch 'develop' of https://github.com/ufs-community/ufs-srweat…
chan-hoo Feb 14, 2023
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion Externals.cfg
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -58,7 +58,7 @@ protocol = git
repo_url = https://github.com/NOAA-EMC/AQM-utils
# Specify either a branch name or a hash but not both.
#branch = develop
hash = 81a99ca
hash = 5ee63d2
local_path = sorc/AQM-utils
required = True

Expand Down
2 changes: 1 addition & 1 deletion jobs/JREGIONAL_AQM_ICS
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -17,7 +17,7 @@
#-----------------------------------------------------------------------
#
. $USHdir/source_util_funcs.sh
source_config_for_task "cpl_aqm_parm" ${GLOBAL_VAR_DEFNS_FP}
source_config_for_task "task_aqm_ics" ${GLOBAL_VAR_DEFNS_FP}
. $USHdir/job_preamble.sh
#
#-----------------------------------------------------------------------
Expand Down
2 changes: 1 addition & 1 deletion jobs/JREGIONAL_AQM_LBCS
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -17,7 +17,7 @@
#-----------------------------------------------------------------------
#
. $USHdir/source_util_funcs.sh
source_config_for_task "task_get_extrn_lbcs|task_make_orog|task_make_lbcs|cpl_aqm_parm" ${GLOBAL_VAR_DEFNS_FP}
source_config_for_task "task_get_extrn_lbcs|task_make_orog|task_make_lbcs|cpl_aqm_parm|task_aqm_lbcs" ${GLOBAL_VAR_DEFNS_FP}
. $USHdir/job_preamble.sh
#
#-----------------------------------------------------------------------
Expand Down
2 changes: 1 addition & 1 deletion jobs/JREGIONAL_BIAS_CORRECTION_O3
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -16,7 +16,7 @@
#-----------------------------------------------------------------------
#
. $USHdir/source_util_funcs.sh
source_config_for_task "cpl_aqm_parm|task_run_post" ${GLOBAL_VAR_DEFNS_FP}
source_config_for_task "cpl_aqm_parm|task_run_post|task_bias_correction_o3" ${GLOBAL_VAR_DEFNS_FP}
. $USHdir/job_preamble.sh
#
#-----------------------------------------------------------------------
Expand Down
2 changes: 1 addition & 1 deletion jobs/JREGIONAL_BIAS_CORRECTION_PM25
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -16,7 +16,7 @@
#-----------------------------------------------------------------------
#
. $USHdir/source_util_funcs.sh
source_config_for_task "cpl_aqm_parm|task_run_post" ${GLOBAL_VAR_DEFNS_FP}
source_config_for_task "cpl_aqm_parm|task_run_post|task_bias_correction_pm25" ${GLOBAL_VAR_DEFNS_FP}
. $USHdir/job_preamble.sh
#
#-----------------------------------------------------------------------
Expand Down
2 changes: 1 addition & 1 deletion jobs/JREGIONAL_FIRE_EMISSION
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -16,7 +16,7 @@
#-----------------------------------------------------------------------
#
. $USHdir/source_util_funcs.sh
source_config_for_task "cpl_aqm_parm" ${GLOBAL_VAR_DEFNS_FP}
source_config_for_task "cpl_aqm_parm|task_fire_emission" ${GLOBAL_VAR_DEFNS_FP}
. $USHdir/job_preamble.sh
#
#-----------------------------------------------------------------------
Expand Down
2 changes: 1 addition & 1 deletion jobs/JREGIONAL_NEXUS_EMISSION
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -16,7 +16,7 @@
#-----------------------------------------------------------------------
#
. $USHdir/source_util_funcs.sh
source_config_for_task "task_run_fcst|cpl_aqm_parm" ${GLOBAL_VAR_DEFNS_FP}
source_config_for_task "task_run_fcst|cpl_aqm_parm|task_nexus_emission" ${GLOBAL_VAR_DEFNS_FP}
. $USHdir/job_preamble.sh
#
#-----------------------------------------------------------------------
Expand Down
2 changes: 1 addition & 1 deletion jobs/JREGIONAL_NEXUS_GFS_SFC
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -16,7 +16,7 @@
#-----------------------------------------------------------------------
#
. $USHdir/source_util_funcs.sh
source_config_for_task "cpl_aqm_parm" ${GLOBAL_VAR_DEFNS_FP}
source_config_for_task "cpl_aqm_parm|task_nexus_gfs_sfc" ${GLOBAL_VAR_DEFNS_FP}
. $USHdir/job_preamble.sh
#
#-----------------------------------------------------------------------
Expand Down
2 changes: 1 addition & 1 deletion jobs/JREGIONAL_NEXUS_POST_SPLIT
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -8,7 +8,7 @@
#-----------------------------------------------------------------------
#
. $USHdir/source_util_funcs.sh
source_config_for_task "cpl_aqm_parm" ${GLOBAL_VAR_DEFNS_FP}
source_config_for_task "cpl_aqm_parm|task_nexus_post_split" ${GLOBAL_VAR_DEFNS_FP}
. $USHdir/job_preamble.sh
#
#-----------------------------------------------------------------------
Expand Down
5 changes: 2 additions & 3 deletions jobs/JREGIONAL_POINT_SOURCE
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -8,7 +8,7 @@
#-----------------------------------------------------------------------
#
. $USHdir/source_util_funcs.sh
source_config_for_task "task_run_fcst|cpl_aqm_parm" ${GLOBAL_VAR_DEFNS_FP}
source_config_for_task "task_run_fcst|cpl_aqm_parm|task_point_source" ${GLOBAL_VAR_DEFNS_FP}
. $USHdir/job_preamble.sh
#
#-----------------------------------------------------------------------
Expand Down Expand Up @@ -43,8 +43,7 @@ print_info_msg "
Entering script: \"${scrfunc_fn}\"
In directory: \"${scrfunc_dir}\"

This is the J-job script for the task that generates the emission files
using NEXUS which will output for FV3 (in NetCDF format).
This is the J-job script for the task that generates the point source files.
========================================================================"
#
#-----------------------------------------------------------------------
Expand Down
2 changes: 1 addition & 1 deletion jobs/JREGIONAL_POST_STAT_O3
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -16,7 +16,7 @@
#-----------------------------------------------------------------------
#
. $USHdir/source_util_funcs.sh
source_config_for_task "cpl_aqm_parm|task_run_post" ${GLOBAL_VAR_DEFNS_FP}
source_config_for_task "cpl_aqm_parm|task_run_post|task_post_stat_o3" ${GLOBAL_VAR_DEFNS_FP}
. $USHdir/job_preamble.sh
#
#-----------------------------------------------------------------------
Expand Down
2 changes: 1 addition & 1 deletion jobs/JREGIONAL_POST_STAT_PM25
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -16,7 +16,7 @@
#-----------------------------------------------------------------------
#
. $USHdir/source_util_funcs.sh
source_config_for_task "cpl_aqm_parm|task_run_post" ${GLOBAL_VAR_DEFNS_FP}
source_config_for_task "cpl_aqm_parm|task_run_post|task_post_stat_pm25" ${GLOBAL_VAR_DEFNS_FP}
. $USHdir/job_preamble.sh
#
#-----------------------------------------------------------------------
Expand Down
2 changes: 1 addition & 1 deletion jobs/JREGIONAL_PRE_POST_STAT
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -16,7 +16,7 @@
#-----------------------------------------------------------------------
#
. $USHdir/source_util_funcs.sh
source_config_for_task "cpl_aqm_parm" ${GLOBAL_VAR_DEFNS_FP}
source_config_for_task "task_pre_post_stat" ${GLOBAL_VAR_DEFNS_FP}
. $USHdir/job_preamble.sh
#
#-----------------------------------------------------------------------
Expand Down
2 changes: 0 additions & 2 deletions parm/FV3.input.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -99,7 +99,6 @@ FV3_GFS_2017_gfdlmp:
agrid_vel_rst: False
d4_bg: 0.15
delt_max: 0.008
dnats: 1
do_sat_adj: True
fv_debug: False
k_split: 6
Expand Down Expand Up @@ -227,7 +226,6 @@ FV3_GFS_v15p2:
agrid_vel_rst: False
d2_bg_k1: 0.15
d2_bg_k2: 0.02
dnats: 1
do_sat_adj: True
fv_debug: False
fv_sg_adj: 600
Expand Down
Loading