Replace UM um_env.py configuration file with yaml file

[blimlim]

Closes #454

um.py currently sources a python file um_env.py in order to set environment variables by the UM, which might cause security issues. This pull request makes payu read the data from a yaml file instead.

If this ends up being merged, we'll need to also convert the um_env.py files in the configurations to yaml.

---

Printing out the environment through a run stage userscript shows that the relevant environment variables are the same with and without the changes:

Printed UM environment variable values without the changes:

AINITIAL=
ASTART=restart_dump.astart
AUSCOM_CPL=true
DATAW=/scratch/tm70/sw6175/access-esm/work/base-run-print-env-pre-industrial-c170a89c/atmosphere
FASTRUN=true
IDEALISE=
PAREXE=parexe
PRINT_STATUS=PrStatus_Diag
RPSEED=
RUNID=PI-01
TYPE=NRUN
UM_ATM_NPROCX=16
UM_ATM_NPROCY=12
UM_NAM_MAX_SECONDS=300
UM_NPES=192
UM_SECTOR_SIZE=2048
UM_STDOUT_FILE=atm.fort6.pe
VN=7.3

UNIT01=
UNIT02=prefix
UNIT04=STASHC
UNIT05=namelists
UNIT07=PI-01.out2
UNIT08=/dev/null
UNIT09=CONTCNTL
UNIT10=xhist
UNIT11=ihist
UNIT12=thist
UNIT14=errflag
UNIT15=
UNIT58=
UNIT22=INPUT/STASHmaster
UNIT57=INPUT/spec3a_sw_hadgem1_6on
UNIT80=INPUT/spec3a_lw_hadgem1_6on
STASETS_DIR=INPUT/stasets
VERT_LEV=INPUT/vertlevs_G3

ARCLBIOG=INPUT/biogenic_351sm.N96L38
BIOMASS=INPUT/Bio_1850_ESM1.anc
CHEMOXID=INPUT/sulpc_oxidants_N96_L38
DMSCONC=INPUT/DMS_conc.N96
NDEPFIL=INPUT/Ndep_1850_ESM1.anc
OCFFEMIS=INPUT/OCFF_1850_ESM1.anc
OZONE=INPUT/ozone_1850_ESM1.anc
SOOTEMIS=INPUT/BC_hi_1850_ESM1.anc
SULPEMIS=INPUT/scycl_1850_ESM1_v4.anc
VEGINIT=INPUT/cable_vegfunc_N96.anc

Printed UM environment variable values with the changes:

AINITIAL=
ASTART=restart_dump.astart
AUSCOM_CPL=true
DATAW=/scratch/tm70/sw6175/access-esm/work/swap-to-yaml-pre-industrial-463fc0f6/atmosphere
FASTRUN=true
IDEALISE=
PAREXE=parexe
PRINT_STATUS=PrStatus_Diag
RPSEED=
RUNID=PI-01
TYPE=NRUN
UM_ATM_NPROCX=16
UM_ATM_NPROCY=12
UM_NAM_MAX_SECONDS=300
UM_NPES=192
UM_SECTOR_SIZE=2048
UM_STDOUT_FILE=atm.fort6.pe
VN=7.3

UNIT01=
UNIT02=prefix
UNIT04=STASHC
UNIT05=namelists
UNIT07=PI-01.out2
UNIT08=/dev/null
UNIT09=CONTCNTL
UNIT10=xhist
UNIT11=ihist
UNIT12=thist
UNIT14=errflag
UNIT15=
UNIT58=
UNIT22=INPUT/STASHmaster
UNIT57=INPUT/spec3a_sw_hadgem1_6on
UNIT80=INPUT/spec3a_lw_hadgem1_6on
STASETS_DIR=INPUT/stasets
VERT_LEV=INPUT/vertlevs_G3

ARCLBIOG=INPUT/biogenic_351sm.N96L38
BIOMASS=INPUT/Bio_1850_ESM1.anc
CHEMOXID=INPUT/sulpc_oxidants_N96_L38
DMSCONC=INPUT/DMS_conc.N96
NDEPFIL=INPUT/Ndep_1850_ESM1.anc
OCFFEMIS=INPUT/OCFF_1850_ESM1.anc
OZONE=INPUT/ozone_1850_ESM1.anc
SOOTEMIS=INPUT/BC_hi_1850_ESM1.anc
SULPEMIS=INPUT/scycl_1850_ESM1_v4.anc
VEGINIT=INPUT/cable_vegfunc_N96.anc

Just to be sure, the simulation is also identical with and without the changes:

[pep8speaks]

Hello @blimlim! Thanks for updating this PR. We checked the lines you've touched for PEP 8 issues, and found:

• In the file payu/models/um.py:

Line 114:73: W291 trailing whitespace
Line 118:80: E501 line too long (82 > 79 characters)
Line 121:80: E501 line too long (81 > 79 characters)
Line 124:80: E501 line too long (110 > 79 characters)
Line 126:14: W291 trailing whitespace

Comment last updated at 2024-07-16 05:40:11 UTC

[jo-basevi]

The changes look good to me

[blimlim]

I'll hold off on merging this for the moment – I just realised it might be worth adding in a depreciation error when a configuration contains a um_env.py instead of um_env.yaml file for anyone running existing configurations.

[CodeGat]

Just a few spelling checks and a question.