Performance testing harness

[vchuravy]

1. Setup a performance testing harness that compares against the base line implementations in Canary.jl
2. Make it work on Supercloud/Nersc

[vchuravy]

What kind of performance measurements are we interested here? Overall walltime?

[fxgiraldo]

Valentin, exactly! overall walltime is what we should compare.

[jkozdon]

We could also look at roofline plots. In general, we expect our problems to be bandwidth limited (not flop limited). So, in an ideal world the only cost we'd like to see is the cost of loading and writing memory and all the floating point ops should be hidden.

Whether this is achievable is another question....

We haven't done these calculation yet for our more optimized codes, but this is something we plan to do in the coming weeks to get a sense of how much performance we're leaving on the table.

Personally, I (and maybe all of us at NPS?) have more experience optimizing GPU codes than CPU codes so this will be a good exercise for me/us as well.