Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Get rid of vertical integrals for IncompressibleModel #1443

Closed
ali-ramadhan opened this issue Mar 9, 2021 · 1 comment
Closed

Get rid of vertical integrals for IncompressibleModel #1443

ali-ramadhan opened this issue Mar 9, 2021 · 1 comment
Labels
distributed 🕸️ Our plan for total cluster domination numerics 🧮 So things don't blow up and boil the lobsters alive

Comments

@ali-ramadhan
Copy link
Member

PencilFFTs.jl and the distributed FFT-based Poisson solver in PR #590 uses dimension 1 to do fast FFTs. Oceananigans.jl uses dimension 3 to do vertical integrals so I don't think we want to decompose the domain in z otherwise we would have to start rewriting kernels.

This leaves us with only the ability to do 1D/slab decompositions in y, which won't scale super well. The "surface to volume ratio" of each rank is quite high leading to large communication overhead: #590 (comment)

Discussing with @glwagner it seems like we should be able to get rid of the vertical integral for the hydrostatic pressure anomaly. It's the only vertical integral left in IncompressibleModel so this would allow for yz/pencil decompositions which should scale much better.

@ali-ramadhan ali-ramadhan added numerics 🧮 So things don't blow up and boil the lobsters alive distributed 🕸️ Our plan for total cluster domination labels Mar 9, 2021
@glwagner
Copy link
Member

This isn't pressing for two reasons: (1) #2538 works and (2) we find the quality of solutions degrade when we omit the hydrostatic pressure intergral.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
distributed 🕸️ Our plan for total cluster domination numerics 🧮 So things don't blow up and boil the lobsters alive
Projects
None yet
Development

No branches or pull requests

2 participants