Lower constant output #398

twicki · 2022-12-08T17:26:40Z

Purpose

Merge in all code changes made to run the 4k nodes run. Mostly logging, quality of life and long run safety.

Code changes:

Timer for halo ex, default to off
Write rank 0 performance timing at each diagnostics
Move grid save at cleanup
Had SafetyChecker

Checklist

Before submitting this PR, please make sure:

You have followed the coding standards guidelines established at Code Review Checklist.
Docstrings and type hints are added to new and updated routines, as appropriate
All relevant documentation has been updated or added (e.g. README, CONTRIBUTING docs)
For each public change and fix in pace-util, HISTORY has been updated
Unit tests are added or updated for non-stencil code changes

… to raise Add nvtx performance enable/disable to allow for more efficient nsys run Pass the timer to the Comm so halo ex can time

.jenkins/install_virtualenv.sh

driver/pace/driver/driver.py

driver/pace/driver/safety_checks.py

util/pace/util/monitor/netcdf_monitor.py

…nstant_output

Move mark to PerformanceCollector Tag json performance status

…nstant_output

…wer_constant_output

…into lower_constant_output

Revert req change, should work

elynnwu and others added 30 commits August 30, 2022 11:19

update default namelist option

bd86f26

Merge commit 'bd86f26f08f68ba22059a59614d513da17549881' into main

127052e

Merge branch 'main' of github.com:ai2cm/pace into main

3bb8f85

Merge branch 'main' of github.com:ai2cm/pace into main

f6617a4

Merge branch 'main' of github.com:ai2cm/pace into main

8c327a2

pass cp only in orchestration mode

db552f5

add override for dace gpu

e554bd2

physics driver as well

a82784f

dace gpu serial test passing

d435e8b

Merge branch 'fix/microphysics-gpu' into main

04fad28

Merge branch 'main' of github.com:ai2cm/pace into main

d6a8c8c

Merge branch 'main' of github.com:ai2cm/pace into main

7da9512

Merge branch 'main' of github.com:ai2cm/pace into main

76ebfcf

Merge branch 'main' of github.com:ai2cm/pace into main

046d120

Merge branch 'main' of github.com:ai2cm/pace into main

b71a927

Merge branch 'main' of github.com:ai2cm/pace into main

5d7a61f

save one file per tile for netcdf monitor

b2cab68

small cleanups

6ce0c9c

fix typing

35d5eb5

delete dead code

0ac3337

Merge branch 'feature/tile_netcdf_monitor' into main

498c19c

add safety_checks for during runs

62489c8

cleanup

ee32979

lines are too long

2dcb58a

Merge branch 'main' of github.com:ai2cm/pace into main

52485eb

Merge remote-tracking branch 'origin/safety_checker' into main

b4b96f9

reviewers' feedback

b64bbd6

add agrid grid

ce775ff

updated saftey checkter

ce42990

32 bit output

0a60196

tobias and others added 6 commits November 24, 2022 16:18

this is it

e21daa0

working thing

00c76d4

netcdf monitor: delete previous files to avoid netcdf4.ensure_success…

96e2c18

… to raise Add nvtx performance enable/disable to allow for more efficient nsys run Pass the timer to the Comm so halo ex can time

Add global additive timer for HaloEx

de8a328

Comm timer on a config hook. Default to False

b28b847

Better protection for unavailable GPUs

93a62ce

twicki commented Dec 8, 2022

View reviewed changes

tobias and others added 4 commits December 8, 2022 18:37

Merge commit 'b64bbd6392ea6bd2e8f886b2257e90d47723dad9' into lower_co…

6ab6bcc

…nstant_output

env-var

eee1d3d

dce

418da4d

Skip json write for large runs

d2617f7

Move mark to PerformanceCollector Tag json performance status

FlorianDeconinck marked this pull request as ready for review December 8, 2022 20:24

lint

bdffb2d

FlorianDeconinck approved these changes Dec 9, 2022

View reviewed changes

FlorianDeconinck and others added 11 commits December 9, 2022 01:01

Merge commit '06206675c337c6e038e2cc437a43edb23d9445e4' into lower_co…

079600e

…nstant_output

fix safety check tests

e10951d

Merge branch 'lower_constant_output' of github.com:ai2cm/pace into lo…

10de1bf

…wer_constant_output

linting is done

e9a4959

make dype of store constant optional

3d5e24c

Fix halo timer test

8afd6fa

Merge branch 'lower_constant_output' of https://github.com/ai2cm/pace …

b7ae5cb

…into lower_constant_output

Add xarray, netcdf4 for monitor

35c7399

Fix vector halo timer test

69aa1d4

Revert req change, should work

fix util test

d20d660

try again

6b4fef8

FlorianDeconinck merged commit 2d36a9d into main Dec 9, 2022

FlorianDeconinck deleted the lower_constant_output branch December 9, 2022 21:55

FlorianDeconinck mentioned this pull request Dec 21, 2022

constants.nc needs to be removed if you change npx #381

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Lower constant output #398

Lower constant output #398

twicki commented Dec 8, 2022 •

edited by FlorianDeconinck

Loading

Lower constant output #398

Lower constant output #398

Conversation

twicki commented Dec 8, 2022 • edited by FlorianDeconinck Loading

Purpose

Code changes:

Checklist

twicki commented Dec 8, 2022 •

edited by FlorianDeconinck

Loading