Develop Bash CI self test [do not merge] #2365

TerrenceMcGuinness-NOAA · 2024-02-29T22:14:29Z

CI Bash Self test on Orion

type labeled on new line

emcbot · 2024-02-29T23:19:30Z

Automated global-workflow Testing Results:

Machine: Orion
Start: Thu Feb 29 16:20:28 CST 2024 on Orion-login-1.HPC.MsState.Edu
---------------------------------------------------
Build: Completed at 02/29/24 05:19:02 PM
Case setup: Completed for experiment C48_ATM_1a7bf9dd
Case setup: Skipped for experiment C48C48_ufs_hybatmDA_1a7bf9dd
Case setup: Skipped for experiment C48mx500_3DVarAOWCDA_1a7bf9dd
Case setup: Completed for experiment C48_S2SWA_gefs_1a7bf9dd
Case setup: Completed for experiment C48_S2SW_1a7bf9dd
Case setup: Completed for experiment C96_atm3DVar_1a7bf9dd
Case setup: Skipped for experiment C96_atmsnowDA_1a7bf9dd
Case setup: Completed for experiment C96C48_hybatmDA_1a7bf9dd

emcbot · 2024-02-29T23:36:19Z

Experiment C48_S2SWA_gefs_1a7bf9dd  *** FAILED *** on Orion
Experiment C48_S2SWA_gefs_1a7bf9dd  with 3 tasks failed at 02/29/24 05:36:18 PM
Error logs:
/work2/noaa/stmp/GFS_CI_ROOT/ORION/PR/2365/RUNTESTS/COMROOT/C48_S2SWA_gefs_1a7bf9dd/logs/2021032312/fcst_mem000.log
/work2/noaa/stmp/GFS_CI_ROOT/ORION/PR/2365/RUNTESTS/COMROOT/C48_S2SWA_gefs_1a7bf9dd/logs/2021032312/fcst_mem001.log
/work2/noaa/stmp/GFS_CI_ROOT/ORION/PR/2365/RUNTESTS/COMROOT/C48_S2SWA_gefs_1a7bf9dd/logs/2021032312/fcst_mem002.log

TerrenceMcGuinness-NOAA · 2024-03-01T01:07:26Z

24:        Type 8 : Restart files second request
24:       -----------------------------------------
24:             From     : 2021/03/23 12:00:00 UTC
24:             To       : 2021/03/28 12:00:00 UTC
24:             Interval :            12:00:00
24:
24:        Wave model ...
 0:  zeroing coupling accumulated fields at kdt=            1
 0:  zeroing coupling accumulated fields at kdt=            1
25: [Orion-04-53:72055:0:72055] Caught signal 11 (Segmentation fault: address not mapped to object at address 0x4)
27: [Orion-04-53:72057:0:72057] Caught signal 11 (Segmentation fault: address not mapped to object at address 0x4)
28: [Orion-04-53:72058:0:72058] Caught signal 11 (Segmentation fault: address not mapped to object at address 0x4)
29: [Orion-04-53:72059:0:72059] Caught signal 11 (Segmentation fault: address not mapped to object at address 0x4)
31: [Orion-04-53:72061:0:72061] Caught signal 11 (Segmentation fault: address not mapped to object at address 0x4)
33: [Orion-04-53:72063:0:72063] Caught signal 11 (Segmentation fault: address not mapped to object at address 0x4)
35: [Orion-04-53:72065:0:72065] Caught signal 11 (Segmentation fault: address not mapped to object at address 0x4)
30: [Orion-04-53:72060:0:72060] Caught signal 11 (Segmentation fault: address not mapped to object at address 0x4)
34: [Orion-04-53:72064:0:72064] Caught signal 11 (Segmentation fault: address not mapped to object at address 0x4)
24: [Orion-04-53:72054:0:72054] Caught signal 11 (Segmentation fault: address not mapped to object at address 0x4)
26: [Orion-04-53:72056:0:72056] Caught signal 11 (Segmentation fault: address not mapped to object at address 0x4)
32: [Orion-04-53:72062:0:72062] Caught signal 11 (Segmentation fault: address not mapped to object at address 0x4)
29: ==== backtrace (tid:  72059) ====
29:  0 0x0000000001dfe97e pdlib_field_vec_mp_unst_pdlib_read_from_file_()  /work2/noaa/stmp/GFS_CI_ROOT/ORION/PR/2365/global-workflow/sorc/ufs_model.fd/WW3/model/src/pdlib_field_vec.F90:501
29:  1 0x0000000001c887ad w3iorsmd_mp_w3iors_()  /work2/noaa/stmp/GFS_CI_ROOT/ORION/PR/2365/global-workflow/sorc/ufs_model.fd/WW3/model/src/w3iorsmd.F90:801
29:  2 0x0000000001bd147b w3initmd_mp_w3init_()  /work2/noaa/stmp/GFS_CI_ROOT/ORION/PR/2365/global-workflow/sorc/ufs_model.fd/WW3/model/src/w3initmd.F90:961
29:  3 0x0000000001aef5c1 waveinit_ufs()  /work2/noaa/stmp/GFS_CI_ROOT/ORION/PR/2365/global-workflow/sorc/ufs_model.fd/WW3/model/src/wav_comp_nuopc.F90:1659
29:  4 0x0000000001aef5c1 initializerealize()  /work2/noaa/stmp/GFS_CI_ROOT/ORION/PR/2365/global-workflow/sorc/ufs_model.fd/WW3/model/src/wav_comp_nuopc.F90:696
29:  5 0x0000000000a77134 ESMCI::FTable::callVFuncPtr()  /work/noaa/epic/role-epic/spack-stack/orion/spack-stack-1.5.1/cache/build_stage/spack-stage-esmf-8.5.0-7t7fsxpkw36g4ht6c6qbu4bvviztvaim/spack-src/src/Superstructure/Component/src/ESMCI_FTable.C:2167
29:  6 0x0000000000a7b1ff ESMCI_FTableCallEntryPointVMHop()  /work/noaa/epic/role-epic/spack-stack/orion/spack-stack-1.5.1/cache/build_stage/spack-stage-esmf-8.5.0-7t7fsxpkw36g4ht6c6qbu4bvviztvaim/spack-src/src/Superstructure/Component/src/ESMCI_FTable.C:824
29:  7 0x0000000000bd9337 ESMCI::VMK::enter()  /work/noaa/epic/role-epic/spack-stack/orion/spack-stack-1.5.1/cache/build_stage/spack-stage-esmf-8.5.0-7t7fsxpkw36g4ht6c6qbu4bvviztvaim/spack-src/src/Infrastructure/VM/src/ESMCI_VMKernel.C:1125
29:  8 0x00000000008e9232 ESMCI::VM::enter()  /work/noaa/epic/role-epic/spack-stack/orion/spack-stack-1.5.1/cache/build_stage/spack-stage-esmf-8.5.0-7t7fsxpkw36g4ht6c6qbu4bvviztvaim/spack-src/src/Infrastructure/VM/src/ESMCI_VM.C:1216
29:  9 0x0000000000a7857a c_esmc_ftablecallentrypointvm_()  /work/noaa/epic/role-epic/spack-stack/orion/spack-stack-1.5.1/cache/build_stage/spack-stage-esmf-8.5.0-7t7fsxpkw36g4ht6c6qbu4bvviztvaim/spack-src/src/Superstructure/Component/src/ESMCI_FTable.C:981
29: 10 0x000000000097aaa0 esmf_compmod_mp_esmf_compexecute_()  /work/noaa/epic/role-epic/spack-stack/orion/spack-stack-1.5.1/cache/build_stage/spack-stage-esmf-8.5.0-7t7fsxpkw36g4ht6c6qbu4bvviztvaim/spack-src/src/Superstructure/Component/src/ESMF_Comp.F90:1223
29: 11 0x0000000000d56301 esmf_gridcompmod_mp_esmf_gridcompinitialize_()  /work/noaa/epic/role-epic/spack-stack/orion/spack-stack-1.5.1/cache/build_stage/spack-stage-esmf-8.5.0-7t7fsxpkw36g4ht6c6qbu4bvviztvaim/spack-src/src/Superstructure/Component/src/ESMF_GridComp.F90:1412
29: 12 0x000000000091d5a0 nuopc_driver_mp_loopmodelcompss_()  /work/noaa/epic/role-epic/spack-stack/orion/spack-stack-1.5.1/cache/build_stage/spack-stage-esmf-8.5.0-7t7fsxpkw36g4ht6c6qbu4bvviztvaim/spack-src/src/addon/NUOPC/src/NUOPC_Driver.F90:2886
29: 13 0x000000000093a36f nuopc_driver_mp_initializeipdv02p3_()  /work/noaa/epic/role-epic/spack-stack/orion/spack-stack-1.5.1/cache/build_stage/spack-stage-esmf-8.5.0-7t7fsxpkw36g4ht6c6qbu4bvviztvaim/spack-src/src/addon/NUOPC/src/NUOPC_Driver.F90:1979
29: 14 0x000000000094f7f0 nuopc_driver_mp_initializegeneric_()  /work/noaa/epic/role-epic/spack-stack/orion/spack-stack-1.5.1/cache/build_stage/spack-stage-esmf-8.5.0-7t7fsxpkw36g4ht6c6qbu4bvviztvaim/spack-src/src/addon/NUOPC/src/NUOPC_Driver.F90:484
29: 15 0x0000000000a77134 ESMCI::FTable::callVFuncPtr()  /work/noaa/epic/role-epic/spack-stack/orion/spack-stack-1.5.1/cache/build_stage/spack-stage-esmf-8.5.0-7t7fsxpkw36g4ht6c6qbu4bvviztvaim/spack-src/src/Superstructure/Component/src/ESMCI_FTable.C:2167
29: 16 0x0000000000a7b1ff ESMCI_FTableCallEntryPointVMHop()  /work/noaa/epic/role-epic/spack-stack/orion/spack-stack-1.5.1/cache/build_stage/spack-stage-esmf-8.5.0-7t7fsxpkw36g4ht6c6qbu4bvviztvaim/spack-src/src/Superstructure/Component/src/ESMCI_FTable.C:824
29: 17 0x0000000000bd912a ESMCI::VMK::enter()  /work/noaa/epic/role-epic/spack-stack/orion/spack-stack-1.5.1/cache/build_stage/spack-stage-esmf-8.5.0-7t7fsxpkw36g4ht6c6qbu4bvviztvaim/spack-src/src/Infrastructure/VM/src/ESMCI_VMKernel.C:2321
29: 18 0x00000000008e9232 ESMCI::VM::enter()  /work/noaa/epic/role-epic/spack-stack/orion/spack-stack-1.5.1/cache/build_stage/spack-stage-esmf-8.5.0-7t7fsxpkw36g4ht6c6qbu4bvviztvaim/spack-src/src/Infrastructure/VM/src/ESMCI_VM.C:1216
29: 19 0x0000000000a7857a c_esmc_ftablecallentrypointvm_()  /work/noaa/epic/role-epic/spack-stack/orion/spack-stack-1.5.1/cache/build_stage/spack-stage-esmf-8.5.0-7t7fsxpkw36g4ht6c6qbu4bvviztvaim/spack-src/src/Superstructure/Component/src/ESMCI_FTable.C:981
29: 20 0x000000000097aaa0 esmf_compmod_mp_esmf_compexecute_()  /work/noaa/epic/role-epic/spack-stack/orion/spack-stack-1.5.1/cache/build_stage/spack-stage-esmf-8.5.0-7t7fsxpkw36g4ht6c6qbu4bvviztvaim/spack-src/src/Superstructure/Component/src/ESMF_Comp.F90:1223
29: 21 0x0000000000d56301 esmf_gridcompmod_mp_esmf_gridcompinitialize_()  /work/noaa/epic/role-epic/spack-stack/orion/spack-stack-1.5.1/cache/build_stage/spack-stage-esmf-8.5.0-7t7fsxpkw36g4ht6c6qbu4bvviztvaim/spack-src/src/Superstructure/Component/src/ESMF_GridComp.F90:1412
29: 22 0x0000000000437502 MAIN__()  /work2/noaa/stmp/GFS_CI_ROOT/ORION/PR/2365/global-workflow/sorc/ufs_model.fd/driver/UFS.F90:381

TerrenceMcGuinness-NOAA · 2024-03-01T01:10:32Z

Since CI BASH is not "gefs" ready in that all cases are built agains build_all.sh -guw so wasn't this out come expected?

WalterKolczynski-NOAA · 2024-03-01T01:11:05Z

The GEFS test needs to be off if you aren't doing a dual build.

WalterKolczynski-NOAA · 2024-03-01T01:11:58Z

Simultaneous comments. We both got to the same place.

TerrenceMcGuinness-NOAA · 2024-03-01T01:13:00Z

So let's simply add skip host for now for the gefs test and proceed.

WalterKolczynski-NOAA · 2024-03-01T01:15:07Z

Do it just for this fake PR so we can confirm existing tests still work, because after that I'd like to rapidly move to Jenkins.

emcbot · 2024-03-01T01:56:39Z

CI Update on Orion at 02/29/24 07:56:38 PM
============================================
Cloning and Building global-workflow PR: 2365
with PID: 73578 on host: Orion-login-1

emcbot · 2024-03-01T03:04:08Z

Automated global-workflow Testing Results:

Machine: Orion
Start: Thu Feb 29 20:01:04 CST 2024 on Orion-login-1.HPC.MsState.Edu
---------------------------------------------------
Build: Completed at 02/29/24 09:03:38 PM
Case setup: Completed for experiment C48_ATM_3d31e3fb
Case setup: Skipped for experiment C48C48_ufs_hybatmDA_3d31e3fb
Case setup: Skipped for experiment C48mx500_3DVarAOWCDA_3d31e3fb
Case setup: Skipped for experiment C48_S2SWA_gefs_3d31e3fb
Case setup: Completed for experiment C48_S2SW_3d31e3fb
Case setup: Completed for experiment C96_atm3DVar_3d31e3fb
Case setup: Skipped for experiment C96_atmsnowDA_3d31e3fb
Case setup: Completed for experiment C96C48_hybatmDA_3d31e3fb

emcbot · 2024-03-01T04:22:09Z

Experiment C48_ATM_3d31e3fb SUCCESS on Orion at 02/29/24 10:22:09 PM

emcbot · 2024-03-01T05:36:16Z

Experiment C96_atm3DVar_3d31e3fb SUCCESS on Orion at 02/29/24 11:36:15 PM

emcbot · 2024-03-01T05:36:26Z

Experiment C96C48_hybatmDA_3d31e3fb SUCCESS on Orion at 02/29/24 11:36:26 PM

emcbot · 2024-03-01T05:52:10Z

Experiment C48_S2SW_3d31e3fb SUCCESS on Orion at 02/29/24 11:52:09 PM

emcbot · 2024-03-01T05:54:06Z

All CI Test Cases Passed on Orion:

Experiment C48_ATM_3d31e3fb *** SUCCESS *** at 02/29/24 10:22:09 PM
Experiment C96_atm3DVar_3d31e3fb *** SUCCESS *** at 02/29/24 11:36:15 PM
Experiment C96C48_hybatmDA_3d31e3fb *** SUCCESS *** at 02/29/24 11:36:26 PM
Experiment C48_S2SW_3d31e3fb *** SUCCESS *** at 02/29/24 11:52:09 PM

TerrenceMcGuinness-NOAA and others added 30 commits October 18, 2023 17:29

trying status of CI as a workflow action

9aa3cc8

added PR number as input

92f2306

Update orion_status.yaml

8c4b2f0

type labeled on new line

take out PR on on

74aa734

take out PR on on

dcffb3c

take out PR on on do loops

9b8d071

take out PR on on do loops syntax until

de4fd80

take out PR on on do loops syntax until singleton

3623231

take out PR on on do loops syntax until set faulse

218a084

take out PR on on do loops syntax until direct

83d5c79

take out PR on on do loops needed noop

de55f1a

take out PR jobs with no steps

5b3e6cc

take out PR jobs need steps

022dd28

take out PR jobs need steps

c9c8ae2

added reusable workflow for getting lables for each job

74a5b5a

fixed uses path

9e306ef

added dependances for main jobs and name in getlabels

1430ab3

change order in needs

56b0734

Merge branch 'NOAA-EMC:develop' into workflow_actions

863f0f5

took out poll

7e14adb

took out secrets required line

0615fad

took out secrets required line

a9145c7

added {{}} on inputs.pr_number

9b8d608

fixed other pr_number assiginments

ac1b51c

EOF error working

404a454

EOF error working II

381832a

ambiguous redrirect to GITHUB_OUTPUT

3f6bd46

tryig to get needs output from get_Ready

ab5d110

still not getting outputs

35c4877

quotes on GITHUB_OUTPUT

725e525

emcbot added CI-Orion-Running **Bot use only** CI testing on Orion for this PR is in-progress and removed CI-Orion-Building **Bot use only** CI testing is cloning/building on Orion labels Feb 29, 2024

emcbot added CI-Orion-Failed **Bot use only** CI testing on Orion for this PR has failed and removed CI-Orion-Running **Bot use only** CI testing on Orion for this PR is in-progress labels Feb 29, 2024

added skip host Orion on gefs case for this first pass self test

3d31e3f

TerrenceMcGuinness-NOAA added CI-Orion-Ready **CM use only** PR is ready for CI testing on Orion and removed CI-Orion-Failed **Bot use only** CI testing on Orion for this PR has failed labels Mar 1, 2024

emcbot added CI-Orion-Building **Bot use only** CI testing is cloning/building on Orion and removed CI-Orion-Ready **CM use only** PR is ready for CI testing on Orion labels Mar 1, 2024

emcbot added CI-Orion-Running **Bot use only** CI testing on Orion for this PR is in-progress and removed CI-Orion-Building **Bot use only** CI testing is cloning/building on Orion labels Mar 1, 2024

emcbot added CI-Orion-Passed **Bot use only** CI testing on Orion for this PR has completed successfully and removed CI-Orion-Running **Bot use only** CI testing on Orion for this PR is in-progress labels Mar 1, 2024

TerrenceMcGuinness-NOAA closed this Mar 1, 2024

TerrenceMcGuinness-NOAA deleted the develop_ci-selftest branch March 4, 2024 16:27

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Develop Bash CI self test [do not merge] #2365

Develop Bash CI self test [do not merge] #2365

TerrenceMcGuinness-NOAA commented Feb 29, 2024

emcbot commented Feb 29, 2024

emcbot commented Feb 29, 2024

TerrenceMcGuinness-NOAA commented Mar 1, 2024

TerrenceMcGuinness-NOAA commented Mar 1, 2024 •

edited

Loading

WalterKolczynski-NOAA commented Mar 1, 2024

WalterKolczynski-NOAA commented Mar 1, 2024

TerrenceMcGuinness-NOAA commented Mar 1, 2024

WalterKolczynski-NOAA commented Mar 1, 2024

emcbot commented Mar 1, 2024

emcbot commented Mar 1, 2024

emcbot commented Mar 1, 2024

emcbot commented Mar 1, 2024

emcbot commented Mar 1, 2024

emcbot commented Mar 1, 2024

emcbot commented Mar 1, 2024

Develop Bash CI self test [do not merge] #2365

Develop Bash CI self test [do not merge] #2365

Conversation

TerrenceMcGuinness-NOAA commented Feb 29, 2024

emcbot commented Feb 29, 2024

emcbot commented Feb 29, 2024

TerrenceMcGuinness-NOAA commented Mar 1, 2024

TerrenceMcGuinness-NOAA commented Mar 1, 2024 • edited Loading

WalterKolczynski-NOAA commented Mar 1, 2024

WalterKolczynski-NOAA commented Mar 1, 2024

TerrenceMcGuinness-NOAA commented Mar 1, 2024

WalterKolczynski-NOAA commented Mar 1, 2024

emcbot commented Mar 1, 2024

emcbot commented Mar 1, 2024

emcbot commented Mar 1, 2024

emcbot commented Mar 1, 2024

emcbot commented Mar 1, 2024

emcbot commented Mar 1, 2024

emcbot commented Mar 1, 2024

TerrenceMcGuinness-NOAA commented Mar 1, 2024 •

edited

Loading