From 16054363e7dec4733afc6e6cff55465a9e54b5eb Mon Sep 17 00:00:00 2001 From: Ashwin Vaidya Date: Wed, 3 Aug 2022 10:14:57 +0200 Subject: [PATCH 1/3] Add publish workflow + update references to main --- .github/workflows/docs.yml | 2 +- .github/workflows/nightly.yml | 2 +- .github/workflows/pre_merge.yml | 2 +- .github/workflows/publish.yml | 21 ++++++++++++++ CONTRIBUTING.md | 2 +- docs/source/guides/developing_on_docker.rst | 2 +- .../001_getting_started.ipynb | 20 +++++++------ notebooks/100_datamodules/103_folder.ipynb | 2 +- notebooks/200_models/201_fastflow.ipynb | 2 +- notebooks/README.md | 28 +++++++++---------- 10 files changed, 53 insertions(+), 30 deletions(-) create mode 100644 .github/workflows/publish.yml diff --git a/.github/workflows/docs.yml b/.github/workflows/docs.yml index 05cf906e18..29fa9e9ac1 100644 --- a/.github/workflows/docs.yml +++ b/.github/workflows/docs.yml @@ -2,7 +2,7 @@ name: Docs on: push: - branches: [development] + branches: [main] paths-ignore: - ".github/**" # Ignore changes towards the .github directory workflow_dispatch: # run on request (no need for PR) diff --git a/.github/workflows/nightly.yml b/.github/workflows/nightly.yml index 27b9466265..83cff4bd9b 100644 --- a/.github/workflows/nightly.yml +++ b/.github/workflows/nightly.yml @@ -10,7 +10,7 @@ jobs: runs-on: [self-hosted, linux, x64] strategy: max-parallel: 1 - if: github.ref == 'refs/heads/development' + if: github.ref == 'refs/heads/main' steps: - name: Print GPU status run: nvidia-smi diff --git a/.github/workflows/pre_merge.yml b/.github/workflows/pre_merge.yml index c2db03fa48..6b841de460 100644 --- a/.github/workflows/pre_merge.yml +++ b/.github/workflows/pre_merge.yml @@ -2,7 +2,7 @@ name: Pre-Merge Checks on: push: - branches: [development, master] + branches: [main] pull_request: workflow_dispatch: # run on request (no need for PR) diff --git a/.github/workflows/publish.yml b/.github/workflows/publish.yml new file mode 100644 index 0000000000..73e35b7dff --- /dev/null +++ b/.github/workflows/publish.yml @@ -0,0 +1,21 @@ +name: Upload Python Package +on: + release: + types: [created] +jobs: + deploy: + runs-on: ubuntu-20.04 + steps: + - uses: actions/checkout@v2 + - uses: actions/setup-python@v2 + - name: Install dependencies + run: | + python -m pip install --upgrade pip + pip install setuptools wheel twine + - name: Build and publish + env: + TWINE_USERNAME: __token__ + TWINE_PASSWORD: ${{ secrets.PYPI_PASSWORD }} + run: | + python setup.py sdist bdist_wheel + twine upload dist/* diff --git a/CONTRIBUTING.md b/CONTRIBUTING.md index 66a2fd02b0..3937a4125e 100644 --- a/CONTRIBUTING.md +++ b/CONTRIBUTING.md @@ -26,7 +26,7 @@ We utilize GitHub issues to track the feature requests as well. If you are certa We actively welcome your pull requests: -1. Fork the repo and create your branch from [`development`](https://github.com/openvinotoolkit/anomalib/tree/development). +1. Fork the repo and create your branch from [`main`](https://github.com/openvinotoolkit/anomalib/tree/main). 1. If you've added code that should be tested, add tests. 1. If you've changed APIs, update the documentation. 1. Ensure the test suite passes. diff --git a/docs/source/guides/developing_on_docker.rst b/docs/source/guides/developing_on_docker.rst index c2f01e28fb..0e64414883 100644 --- a/docs/source/guides/developing_on_docker.rst +++ b/docs/source/guides/developing_on_docker.rst @@ -38,7 +38,7 @@ After building the image, you can run it as follows anomalib \ /bin/bash -This creates an interactive bash session inside the Docker container, in which you can run all anomalib commands as described in the `readme `_. +This creates an interactive bash session inside the Docker container, in which you can run all anomalib commands as described in the `readme `_. The source code is mapped into the running container by means of the `-v "$(pwd)":/anomalib` parameter. This facilitates changes to the source code without having to rebuild the Docker image. diff --git a/notebooks/000_getting_started/001_getting_started.ipynb b/notebooks/000_getting_started/001_getting_started.ipynb index 2d07cdf46c..d7796943d3 100644 --- a/notebooks/000_getting_started/001_getting_started.ipynb +++ b/notebooks/000_getting_started/001_getting_started.ipynb @@ -4,7 +4,7 @@ "cell_type": "markdown", "metadata": {}, "source": [ - "
\"Paris\"
\n", + "
\"Paris\"
\n", "\n", "
💙 A library for benchmarking, developing and deploying deep learning anomaly detection algorithms
\n", "\n", @@ -74,7 +74,7 @@ "Currently, there are **7** anomaly detection models available in `anomalib` library. Namely,\n", "\n", "- [CFlow](https://arxiv.org/pdf/2107.12571v1.pdf)\n", - "- [DFKDE](https://github.com/openvinotoolkit/anomalib/tree/development/anomalib/models/dfkde)\n", + "- [DFKDE](https://github.com/openvinotoolkit/anomalib/tree/main/anomalib/models/dfkde)\n", "- [DFM](https://arxiv.org/pdf/1909.11786.pdf)\n", "- [Ganomaly](https://arxiv.org/abs/1805.06725)\n", "- [Padim](https://arxiv.org/pdf/2011.08785.pdf)\n", @@ -109,7 +109,7 @@ "cell_type": "markdown", "metadata": {}, "source": [ - "We could use [get_configurable_parameter](https://github.com/openvinotoolkit/anomalib/blob/development/anomalib/config/config.py#L114) function to read the configs from the path and return them in a dictionary. We use the default config file that comes with Padim implementation, which uses `./datasets/MVTec` as the path to the dataset. We need to overwrite this after loading the config." + "We could use [get_configurable_parameter](https://github.com/openvinotoolkit/anomalib/blob/main/anomalib/config/config.py#L114) function to read the configs from the path and return them in a dictionary. We use the default config file that comes with Padim implementation, which uses `./datasets/MVTec` as the path to the dataset. We need to overwrite this after loading the config." ] }, { @@ -486,11 +486,8 @@ } ], "metadata": { - "interpreter": { - "hash": "f26beec5b578f06009232863ae217b956681fd13da2e828fa5a0ecf8cf2ccd29" - }, "kernelspec": { - "display_name": "Python 3.8.12 ('anomalib')", + "display_name": "Python 3.8.10 ('anomalib')", "language": "python", "name": "python3" }, @@ -504,9 +501,14 @@ "name": "python", "nbconvert_exporter": "python", "pygments_lexer": "ipython3", - "version": "3.8.12" + "version": "3.8.10" }, - "orig_nbformat": 4 + "orig_nbformat": 4, + "vscode": { + "interpreter": { + "hash": "6a17319b3a437b8e0e3041e999c0deb965ce85aa030426c7edd15eb3312fca65" + } + } }, "nbformat": 4, "nbformat_minor": 2 diff --git a/notebooks/100_datamodules/103_folder.ipynb b/notebooks/100_datamodules/103_folder.ipynb index b450342d1d..a2e2b26a14 100644 --- a/notebooks/100_datamodules/103_folder.ipynb +++ b/notebooks/100_datamodules/103_folder.ipynb @@ -12,7 +12,7 @@ "- A dataset with good and bad images as well as mask ground-truths for pixel-wise evaluation.\n", "- A dataset with good and bad images that is already split into training and testing sets.\n", "\n", - "To experiment this setting we provide a toy dataset that could be downloaded from the following [https://github.com/openvinotoolkit/anomalib/blob/development/docs/source/data/hazelnut_toy.zip](link). For the rest of the tutorial, we assume that the dataset is downloaded and extracted to `../../datasets`, located in the `anomalib` directory." + "To experiment this setting we provide a toy dataset that could be downloaded from the following [https://github.com/openvinotoolkit/anomalib/blob/main/docs/source/data/hazelnut_toy.zip](link). For the rest of the tutorial, we assume that the dataset is downloaded and extracted to `../../datasets`, located in the `anomalib` directory." ] }, { diff --git a/notebooks/200_models/201_fastflow.ipynb b/notebooks/200_models/201_fastflow.ipynb index a02f8c9a6e..57e5d792b9 100644 --- a/notebooks/200_models/201_fastflow.ipynb +++ b/notebooks/200_models/201_fastflow.ipynb @@ -59,7 +59,7 @@ "source": [ "## Data Module\n", "\n", - "To train the model end-to-end, we do need to have a dataset. In our [previous notebooks](https://github.com/openvinotoolkit/anomalib/tree/development/notebooks/100_datamodules), we demonstrate how to initialize benchmark- and custom datasets. In this tutorial, we will use MVTec AD DataModule. We assume that `datasets` directory is created in the `anomalib` root directory and `MVTec` dataset is located in `datasets` directory." + "To train the model end-to-end, we do need to have a dataset. In our [previous notebooks](https://github.com/openvinotoolkit/anomalib/tree/main/notebooks/100_datamodules), we demonstrate how to initialize benchmark- and custom datasets. In this tutorial, we will use MVTec AD DataModule. We assume that `datasets` directory is created in the `anomalib` root directory and `MVTec` dataset is located in `datasets` directory." ] }, { diff --git a/notebooks/README.md b/notebooks/README.md index 2625ab725d..c2f52b517c 100644 --- a/notebooks/README.md +++ b/notebooks/README.md @@ -2,26 +2,26 @@ ## 0. Training and Inference -| Notebook | GitHub | Colab | -| --------------- | -------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ | -| Getting Started | [001_getting_started](000_getting_started/001_getting_started.ipynb) | [![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/openvinotoolkit/anomalib/blob/development/notebooks/000_getting_started/001_getting_started.ipynb) | +| Notebook | GitHub | Colab | +| --------------- | -------------------------------------------------------------------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | +| Getting Started | [001_getting_started](000_getting_started/001_getting_started.ipynb) | [![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/openvinotoolkit/anomalib/blob/main/notebooks/000_getting_started/001_getting_started.ipynb) | ## 1. Data Modules -| Notebook | GitHub | Colab | -| -------- | ---------------------------------------------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | -| BTech | [101_btech](100_datamodules/101_btech.ipynb) | [![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/openvinotoolkit/anomalib/blob/development/notebooks/100_datamodules/101_btech.ipynb) | -| MVTec | [102_mvtec](100_datamodules/102_mvtec.ipynb) | [![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/openvinotoolkit/anomalib/blob/development/notebooks/100_datamodules/102_mvtec.ipynb) | -| Folder | [103_folder](100_datamodules/103_folder.ipynb) | [![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/openvinotoolkit/anomalib/blob/development/notebooks/100_datamodules/103_folder.ipynb) | +| Notebook | GitHub | Colab | +| -------- | ---------------------------------------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | +| BTech | [101_btech](100_datamodules/101_btech.ipynb) | [![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/openvinotoolkit/anomalib/blob/main/notebooks/100_datamodules/101_btech.ipynb) | +| MVTec | [102_mvtec](100_datamodules/102_mvtec.ipynb) | [![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/openvinotoolkit/anomalib/blob/main/notebooks/100_datamodules/102_mvtec.ipynb) | +| Folder | [103_folder](100_datamodules/103_folder.ipynb) | [![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/openvinotoolkit/anomalib/blob/main/notebooks/100_datamodules/103_folder.ipynb) | ## 2. Models -| Notebook | GitHub | Colab | -| -------- | --------------------------------------------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | -| Model | [201_fastflow](200_models/201_fastflow.ipynb) | [![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/openvinotoolkit/anomalib/blob/development/notebooks/200_models/201_fastflow.ipynb) | +| Notebook | GitHub | Colab | +| -------- | --------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | +| Model | [201_fastflow](200_models/201_fastflow.ipynb) | [![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/openvinotoolkit/anomalib/blob/main/notebooks/200_models/201_fastflow.ipynb) | ## 3. Benchmarking and Hyperparameter Optimization -| Notebook | GitHub | Colab | -| ------------ | ----------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ | -| Benchmarking | [301_benchmarking](300_benchmarking/301_benchmarking.ipynb) | [![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/openvinotoolkit/anomalib/blob/development/notebooks/300_benchmarking/301_benchmarking.ipynb) | +| Notebook | GitHub | Colab | +| ------------ | ----------------------------------------------------------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | +| Benchmarking | [301_benchmarking](300_benchmarking/301_benchmarking.ipynb) | [![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/openvinotoolkit/anomalib/blob/main/notebooks/300_benchmarking/301_benchmarking.ipynb) | From 65975b1953d2722c90fdab38f8857ed05cd440eb Mon Sep 17 00:00:00 2001 From: Ashwin Vaidya Date: Wed, 3 Aug 2022 10:48:13 +0200 Subject: [PATCH 2/3] fix formatting --- .github/workflows/publish.yml | 26 +++++++++++++------------- 1 file changed, 13 insertions(+), 13 deletions(-) diff --git a/.github/workflows/publish.yml b/.github/workflows/publish.yml index 73e35b7dff..5573b7e39d 100644 --- a/.github/workflows/publish.yml +++ b/.github/workflows/publish.yml @@ -6,16 +6,16 @@ jobs: deploy: runs-on: ubuntu-20.04 steps: - - uses: actions/checkout@v2 - - uses: actions/setup-python@v2 - - name: Install dependencies - run: | - python -m pip install --upgrade pip - pip install setuptools wheel twine - - name: Build and publish - env: - TWINE_USERNAME: __token__ - TWINE_PASSWORD: ${{ secrets.PYPI_PASSWORD }} - run: | - python setup.py sdist bdist_wheel - twine upload dist/* + - uses: actions/checkout@v2 + - uses: actions/setup-python@v2 + - name: Install dependencies + run: | + python -m pip install --upgrade pip + pip install setuptools wheel twine + - name: Build and publish + env: + TWINE_USERNAME: __token__ + TWINE_PASSWORD: ${{ secrets.PYPI_PASSWORD }} + run: | + python setup.py sdist bdist_wheel + twine upload dist/* From d50090b7349f818e08196351e3096e18e9e41945 Mon Sep 17 00:00:00 2001 From: Ashwin Vaidya Date: Wed, 3 Aug 2022 13:12:37 +0200 Subject: [PATCH 3/3] Limit pytorch lightning version + fix references --- README.md | 8 ++++---- requirements/base.txt | 2 +- 2 files changed, 5 insertions(+), 5 deletions(-) diff --git a/README.md b/README.md index 8e539a38f5..114b72095c 100644 --- a/README.md +++ b/README.md @@ -9,7 +9,7 @@ [Key Features](#key-features) • [Getting Started](#getting-started) • [Docs](https://openvinotoolkit.github.io/anomalib) • -[License](https://github.com/openvinotoolkit/anomalib/blob/development/LICENSE) +[License](https://github.com/openvinotoolkit/anomalib/blob/main/LICENSE) [![python](https://img.shields.io/badge/python-3.7%2B-green)]() [![pytorch](https://img.shields.io/badge/pytorch-1.8.1%2B-orange)]() @@ -76,7 +76,7 @@ pip install -e . ## ⚠️ Anomalib < v.0.4.0 -By default [`python tools/train.py`](https://gitlab-icv.inn.intel.com/algo_rnd_team/anomaly/-/blob/development/train.py) +By default [`python tools/train.py`](https://gitlab-icv.inn.intel.com/algo_rnd_team/anomaly/-/blob/main/train.py) runs [PADIM](https://arxiv.org/abs/2011.08785) model on `leather` category from the [MVTec AD](https://www.mvtec.com/company/research/datasets/mvtec-ad) [(CC BY-NC-SA 4.0)](https://creativecommons.org/licenses/by-nc-sa/4.0/) dataset. ```bash @@ -84,7 +84,7 @@ python tools/train.py # Train PADIM on MVTec AD leather ``` Training a model on a specific dataset and category requires further configuration. Each model has its own configuration -file, [`config.yaml`](https://gitlab-icv.inn.intel.com/algo_rnd_team/anomaly/-/blob/development/padim/anomalib/models/padim/config.yaml) +file, [`config.yaml`](https://gitlab-icv.inn.intel.com/algo_rnd_team/anomaly/-/blob/main/padim/anomalib/models/padim/config.yaml) , which contains data, model and training configurable parameters. To train a specific model on a specific dataset and category, the config file is to be provided: @@ -156,7 +156,7 @@ We introduce a new CLI approach that uses [PyTorch Lightning CLI](https://pytorc anomalib fit --config ``` -For instance, to train a [PatchCore](https://github.com/openvinotoolkit/anomalib/tree/development/anomalib/models/patchcore) model, the following command would be run: +For instance, to train a [PatchCore](https://github.com/openvinotoolkit/anomalib/tree/main/anomalib/models/patchcore) model, the following command would be run: ```bash anomalib fit --config ./configs/model/patchcore.yaml diff --git a/requirements/base.txt b/requirements/base.txt index 877a51f84d..e7de2fbbbc 100644 --- a/requirements/base.txt +++ b/requirements/base.txt @@ -8,7 +8,7 @@ matplotlib>=3.4.3 omegaconf>=2.1.1 opencv-python>=4.5.3.56 pandas>=1.1.0 -pytorch-lightning>=1.6.0 +pytorch-lightning>=1.6.0,<1.7.0 timm==0.5.4 torchmetrics>=0.9.0,<=0.9.1 torchvision>=0.9.1,<=0.12.0