Tentative: multi-dimensional outputs for pytorch and paddle

[kuangdai]

How this was done

I made it in two steps:

1. moving the strategy classes from tensorflow's deeponet.py to a separate file deeponet_strategy.py so that all backends can share it
2. adding strategeis to pytorch and paddle following the same way (with few synatx adaptions)

Remaining Issue

I noted that in call(self, x_func, x_loc, training=False) of DeepONetStrategy, the argument training is completely unused. It is kept in this PR because I don't know if it serves some other purpose. I suggest removal of it.

Tests

A file test_strat.py (renamed to test_start.txt for uploading) is attached in this description. It tests all the model classes (DeepONet, DeepONetCartesianProd, PODDeepONet) and strategy settings (single, independent, split both, split branch, split trunk), making sure that the forward pass yields the correct output shape. To reproduce the tests, put this file under the top directory and run

DDE_BACKEND=tensorflow python test_strat.py 
DDE_BACKEND=pytorch python test_strat.py 
DDE_BACKEND=paddle python test_strat.py

And you should see no errors.

test_strat.txt

[lululxvi]

Great.

Could you make paddle a separate PR? As I need someone else to check the paddle code. We can focus on the tf and pytorch code first.