[RFC] Future structure #13265
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -1,9 +1,9 @@ | ||
| #!/bin/bash | ||
| # Run this script from the project root. | ||
| URL="https://pl-public-data.s3.amazonaws.com/legacy/checkpoints.zip" | ||
| mkdir -p test/legacy | ||
| # wget is simpler but does not work on Windows | ||
| python -c "from urllib.request import urlretrieve; urlretrieve('$URL', 'test/legacy/checkpoints.zip')" | ||
| ls -l test/legacy/ | ||
| unzip -o test/legacy/checkpoints.zip -d test/legacy/ | ||
| ls -l test/legacy/checkpoints/ |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
|
|
@@ -8,21 +8,20 @@ trigger: | |
| include: | ||
| - "master" | ||
| - "release/*" | ||
| - "future/*" | ||
| - "refs/tags/*" | ||
| pr: | ||
| - "master" | ||
| - "release/*" | ||
| - "future/*" | ||
|
|
||
| jobs: | ||
| - job: testing | ||
| # how long to run the job before automatically cancelling | ||
| timeoutInMinutes: "10" | ||
| # how much time to give 'run always even if cancelled tasks' before stopping them | ||
| cancelTimeoutInMinutes: "2" | ||
| pool: intel-hpus | ||
| workspace: | ||
| clean: all | ||
|
|
||
|
|
@@ -33,25 +32,32 @@ jobs: | |
| displayName: 'Instance HW info' | ||
|
|
||
| - bash: | | ||
| pip install -e .[extra] | ||
| pip install . --requirement requirements/test.txt | ||
| displayName: 'Install dependencies' | ||
|
|
||
| - bash: | | ||
| python -m pytest -sv unittests_pl/accelerators/test_hpu.py --forked --junitxml=hpu1_test-results.xml | ||
| workingDirectory: test | ||
| displayName: 'Single card HPU test' | ||
|
|
||
| - bash: | | ||
| python -m pytest -sv unittests_pl/accelerators/test_hpu.py --forked --hpus 8 --junitxml=hpu8_test-results.xml | ||
| workingDirectory: test | ||
| displayName: 'Multi card(8) HPU test' | ||
|
|
||
| - bash: | | ||
| python -m pytest -sv unittests_pl/plugins/precision/hpu/test_hpu.py --hmp-bf16 \ | ||
| 'unittests_pl/plugins/precision/hpu/ops_bf16.txt' --hmp-fp32 \ | ||
| 'unittests_pl/plugins/precision/hpu/ops_fp32.txt' --forked \ | ||
| --junitxml=hpu1_precision_test-results.xml | ||
| workingDirectory: test | ||
| displayName: 'HPU precision test' | ||
|
|
||
| - bash: | | ||
| export PYTHONPATH="${PYTHONPATH}:$(pwd)" | ||
| python "pl_hpu/mnist_sample.py" | ||
|
There was a problem hiding this comment. The structure here should be |
||
| workingDirectory: examples | ||
| displayName: 'Testing: HPU examples' | ||
|
|
||
| - task: PublishTestResults@2 | ||
|
|
||
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
|
|
@@ -5,23 +5,23 @@ trigger: | |
| branches: | ||
| include: | ||
| - master | ||
| - "release/*" | ||
| - "future/*" | ||
| - "refs/tags/*" | ||
| pr: | ||
| - master | ||
| - "release/*" | ||
| - "future/*" | ||
|
|
||
| variables: | ||
| - name: poplar_sdk | ||
| value: "poplar_sdk-ubuntu_20_04-2.3.1+793-89796d462d" | ||
|
|
||
| jobs: | ||
| - job: testing | ||
| # how long to run the job before automatically cancelling | ||
| timeoutInMinutes: "15" | ||
| pool: graphcore-ipus | ||
| workspace: | ||
| clean: all | ||
|
|
||
|
|
@@ -55,7 +55,7 @@ jobs: | |
| export GIT_TERMINAL_PROMPT=1 | ||
| python ./requirements/adjust-versions.py requirements/extra.txt | ||
| python ./requirements/adjust-versions.py requirements/examples.txt | ||
| pip install -e . --requirement ./requirements/devel.txt | ||
| pip list | ||
| displayName: 'Install dependencies' | ||
|
|
||
|
|
@@ -68,16 +68,23 @@ jobs: | |
| set -eux | ||
| source ${{ variables.poplar_sdk }}/poplar-ubuntu*/enable.sh | ||
| source ${{ variables.poplar_sdk }}/popart-ubuntu*/enable.sh | ||
| python -c "import poptorch; print(poptorch.__version__)" | ||
| displayName: "Check poptorch installation" | ||
|
|
||
| - bash: | | ||
| source ${{ variables.poplar_sdk }}/poplar-ubuntu*/enable.sh | ||
| source ${{ variables.poplar_sdk }}/popart-ubuntu*/enable.sh | ||
| cd src | ||
| python -m pytest pytorch_lightning | ||
| displayName: 'DocTests' | ||
|
There was a problem hiding this comment. I wouldn't add doctesting to these accelerator jobs as it complicates the pipeline for contributors |
||
|
|
||
| - bash: | | ||
| source ${{ variables.poplar_sdk }}/poplar-ubuntu*/enable.sh | ||
| source ${{ variables.poplar_sdk }}/popart-ubuntu*/enable.sh | ||
| cd test | ||
| python -m coverage run --source pytorch_lightning -m pytest unittests_pl -vv --durations=50 | ||
| env: | ||
| MKL_THREADING_LAYER: "GNU" | ||
| POPTORCH_WAIT_FOR_IPU: 1 | ||
| PL_RUN_IPU_TESTS: 1 | ||
| displayName: 'UnitTests' | ||
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
|
|
@@ -13,6 +13,7 @@ trigger: | |
| include: | ||
| - "master" | ||
| - "release/*" | ||
| - "future/*" | ||
| - "refs/tags/*" | ||
| pr: | ||
| - "master" | ||
|
|
@@ -40,7 +41,7 @@ references: | |
| # the image uses python 2.7 by default, force a different version | ||
| pyenv global 3.7.3 | ||
| python --version | ||
| pip install -e . -r requirements/docs.txt | ||
| pip list | ||
| cd docs | ||
| make clean | ||
|
|
||
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
|
|
@@ -16,48 +16,48 @@ | |
| /docs/ @edenlightning @tchaton @borda @awaelchli @RobertLaurella | ||
| /.github/*.md @edenlightning @williamfalcon @borda | ||
| /.github/ISSUE_TEMPLATE/ @edenlightning @borda @tchaton | ||
| /docs/source-PL/conf.py @borda @awaelchli @carmocca | ||
|
There was a problem hiding this comment. Why do we need this
|
||
| /docs/source-PL/index.rst @williamfalcon | ||
| /docs/source-PL/levels @williamfalcon @RobertLaurella | ||
| /docs/source-PL/expertise_levels @williamfalcon @RobertLaurella | ||
|
|
||
| # Packages | ||
| /src/pytorch_lightning/accelerators @williamfalcon @tchaton @SeanNaren @awaelchli @justusschock @kaushikb11 | ||
| /src/pytorch_lightning/callbacks @williamfalcon @tchaton @carmocca @borda @kaushikb11 | ||
| /src/pytorch_lightning/core @tchaton @SeanNaren @borda @carmocca @justusschock @kaushikb11 | ||
| /src/pytorch_lightning/distributed @williamfalcon @tchaton @awaelchli @kaushikb11 | ||
| /src/pytorch_lightning/lite @tchaton @awaelchli @carmocca | ||
| /src/pytorch_lightning/loggers @tchaton @awaelchli @borda | ||
| /src/pytorch_lightning/loggers/wandb.py @borisdayma | ||
| /src/pytorch_lightning/loggers/neptune.py @shnela @HubertJaworski @pkasprzyk @pitercl @Raalsky @aniezurawski @kamil-kaczmarek | ||
| /src/pytorch_lightning/loops @tchaton @awaelchli @justusschock @carmocca | ||
| /src/pytorch_lightning/overrides @tchaton @SeanNaren @borda | ||
| /src/pytorch_lightning/plugins @tchaton @SeanNaren @awaelchli @justusschock | ||
| /src/pytorch_lightning/profiler @williamfalcon @tchaton @borda @carmocca | ||
| /src/pytorch_lightning/profiler/pytorch.py @nbcsm @guotuofeng | ||
| /src/pytorch_lightning/strategies @tchaton @SeanNaren @awaelchli @justusschock @kaushikb11 | ||
| /src/pytorch_lightning/trainer @williamfalcon @borda @tchaton @SeanNaren @carmocca @awaelchli @justusschock @kaushikb11 | ||
| /src/pytorch_lightning/trainer/connectors @tchaton @SeanNaren @carmocca @borda | ||
| /src/pytorch_lightning/tuner @SkafteNicki @borda @awaelchli | ||
| /src/pytorch_lightning/utilities @borda @tchaton @SeanNaren @carmocca | ||
|
|
||
| # Specifics | ||
| /src/pytorch_lightning/trainer/connectors/logger_connector @tchaton @carmocca | ||
| /src/pytorch_lightning/trainer/progress.py @tchaton @awaelchli @carmocca | ||
|
|
||
| # API | ||
| /src/pytorch_lightning/callbacks/base.py @williamfalcon @awaelchli @ananthsub @carmocca | ||
| /src/pytorch_lightning/core/datamodule.py @williamFalcon @awaelchli @ananthsub @carmocca | ||
| /src/pytorch_lightning/trainer/trainer.py @williamfalcon @tchaton @awaelchli | ||
| /src/pytorch_lightning/core/hooks.py @williamfalcon @tchaton @awaelchli @ananthsub @carmocca | ||
| /src/pytorch_lightning/core/lightning.py @williamfalcon @tchaton @awaelchli | ||
|
|
||
| # Testing | ||
| /test/unittests_pl/helpers/boring_model.py @williamfalcon @tchaton @borda | ||
|
|
||
| /.github/CODEOWNERS @williamfalcon | ||
| /.github/approve_config.yml @williamfalcon | ||
| /SECURITY.md @williamfalcon | ||
| /README.md @williamfalcon @edenlightning @borda | ||
| /setup.py @williamfalcon @borda @carmocca | ||
| /src/pytorch_lightning/__about__.py @williamfalcon @borda @carmocca | ||
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This should be removed before merge, right?