Merge branch 'main' of github.com:OpenAstronomy/packaging-guide

.gitignore

@@ -120,6 +120,9 @@ venv.bak/
 .spyderproject
 .spyproject
 
+# Pycharm project settings
+.idea
+
 # Rope project settings
 .ropeproject
 

cookiecutter.json

@@ -4,14 +4,14 @@
     "short_description": "",
     "author_name": "",
     "author_email": "",
+    "project_url": "",
     "license": [
         "BSD 3-Clause",
         "GNU GPL v3+",
         "Apache Software License 2.0",
         "BSD 2-Clause",
         "Other"
     ],
-    "project_url": "",
     "minimum_python_version": [
         "3.9",
         "3.10",
@@ -21,11 +21,13 @@
     "use_compiled_extensions": "n",
     "enable_dynamic_dev_versions": "n",
     "include_example_code": "n",
+    "include_cruft_update_github_workflow": "n",
     "_sphinx_theme": "alabaster",
     "_parent_project": "",
     "_install_requires": "",
     "_copy_without_render": [
         "docs/_templates",
-        "docs/_static"
+        "docs/_static",
+        ".github/workflows/sub_package_update.yml"
     ]
 }

docs/ci.rst

@@ -1,25 +1,29 @@
 .. _ci:
+
 ======================
 Continuous Integration
 ======================
 
-Continuous Integration (CI) is the method by which software is tested and built before deployment to users or clients.
+Continuous Integration (CI) is the method by which software is tested and built before deployment to users.
 A set of 'jobs' are defined in a ``.yml`` file, roughly taking the flow build - test - deploy.
-Each run is built from a clean environment that ideally supplies the basic requirements to perform the task.
-Workflows can be set to begin on triggers such as a ``git push`` or the completion of another workflow.
+Each run is built from a clean environment.
+Workflows can be set to begin on triggers for example, a ``git push`` or a new tag.
 
-There are an array solutions for running CI, Open Astronomy recommends `GitHub Actions <https://docs.github.com/en/actions/>`__.
-GitHub Actions workflows are defined in ``.github/workflows/`` at the root of the repo.
-Open Astronomy supplies a `set of tooling <https://github.com/OpenAstronomy/github-actions-workflow>`__ to make configuring GitHub Actions easier.
+There are an array solutions for running CI, Open Astronomy recommends `GitHub Actions <https://docs.github.com/en/actions/>`__ for projects using GitHub.
+GitHub Actions workflows are defined in the ``.github/workflows/`` folder at the root of the repo.
+Open Astronomy maintains a `set of tools <https://github.com/OpenAstronomy/github-actions-workflow>`__ to make configuring GitHub Actions easier.
 
 Examples
 ++++++++
 Testing
 -------
-In order GitHub Actions to run your workflow, it requires; an event to trigger the workflow, one or more jobs to complete on virtual machines and all steps must either run a script or trigger an action.
+In order GitHub Actions to run your workflow, it requires; an event to trigger the workflow, one or more jobs to complete and all steps must either run a script or trigger an action.
 Looking at this in context:
 
-.. code-block:: yml
+.. code-block:: yaml
+
+    name: Run template tests
+
     on:
       push:
       pull_request:
@@ -30,42 +34,48 @@ Looking at this in context:
         uses: OpenAstronomy/github-actions-workflows/.github/workflows/tox.yml@v1
         with:
           envs: |
-            - linux: py311
+            - linux: py311-test
 
-In this case the workflow is triggered on push to branch, a PR and a manual trigger of the workflow on GitHub actions.
-The second line of the job defines the name, in this case ``test`` and uses instructs the virtual machine to use Open Astronomy's pre-defined test with tox.
-The ``envs`` list, defines which environments the test set is going to be ran on.
-Therefore the package will be built using tox, will then trigger the testing in the selected environment/s.
+In this case the workflow is triggered on a push to a branch, a PR, or a manual trigger of the workflow (`workflow_dispatch`).
+The second line of the job defines the name of the job, in this case ``test``, and uses Open Astronomy's pre-defined workflow to run the tests with tox.
+The ``envs`` list, defines which tox environments will be run for that job.
 
-Publising to PyPI
------------------
+Publishing to PyPI
+------------------
 
 No Compiled Extensions
 ######################
 
-Python packages should be published on `pypi <https://pypi.org/>`__, in which case GitHub actions can facilitate this.
-The ``on`` command will need to be more selective in this case.
-Publishing to pypi would only be desirable on merges to master, here the trigger for the flow will be on push to the main branch specifically.
-We also need to pass a pypi key, associated with your pypi account.
-`Instructions on creating up your key here <https://pypi.org/help/#apitoken>`__.
-The secret can be stored at organisation or repo level in GitHub settings, and the secret defined earlier in the workflow.
+Python packages should be published on `PyPI <https://pypi.org/>`__, which can be automated on CI.
+This can improve security (as fewer people need access to publish on PyPI) and make it less effort for maintainers to publish a release.
+When we are building and publishing releases to PyPI we only want this to happen on a `git tag <https://git-scm.com/book/en/v2/Git-Basics-Tagging>`__, as opposed to on every commit.
+However, if we only run the build job on tags, we never have a way to test that the job works before we tag a release of our package.
+The OpenAstronomy publish workflows will (by default) only publish to PyPI on a tag which starts with `v` (`see here <https://github-actions-workflows.openastronomy.org/en/stable/publish.html#upload-to-pypi>`__).
+Therefore, we recommend running the workflow on both push to your default branch (`main`), on tags and on manual runs.
 
+.. code-block:: yaml
 
-.. code-block:: yml
     on:
       push:
+        branches:
+          - 'main'
       tag:
+      workflow_dispatch:
 
    jobs:
      publish:
        uses: OpenAstronomy/github-actions-workflows/.github/workflows/publish_pure_python.yml@v1
        with:
          test_extras: test
-         test_command: pytest --pyargs test_package
+         test_command: pytest --pyargs <package name>
      secrets:
        pypi_token: ${{ secrets.pypi_token }}
 
-Replace references to test_package with the package to be published.
+Replace references to `<package_name>` with the package to be published.
+
+To publish to PyPI we need a PyPI token, associated with your PyPI account.
+`Instructions on creating up your key here <https://pypi.org/help/#apitoken>`__.
+The secret can be stored at `organisation <https://docs.github.com/en/actions/security-guides/using-secrets-in-github-actions#creating-secrets-for-an-organization>`__ or `repo level <https://docs.github.com/en/actions/security-guides/using-secrets-in-github-actions#creating-secrets-for-a-repository>`__ in GitHub settings.
 
 With Compiled Extensions
 ########################
@@ -77,7 +87,8 @@ In this case, in addition to the running the tests, the ``with`` block also incl
 'Targets' are the distributions which the binary will be built for, so in this case it would be linux and MacOS 64 bit.
 The ``publish`` method from the Open Astronomy GitHub actions packages the module with the dependencies for the specific targets listed
 
-.. code-block:: yml
+.. code-block:: yaml
+
     jobs:
       publish:
         uses: OpenAstronomy/github-actions-workflows/.github/workflows/publish.yml@v1
@@ -99,7 +110,8 @@ Putting it all together
 
 Combining the above steps reveals a total workflow, build, testing and publishing
 
-.. code-block:: yml
+.. code-block:: yaml
+
     name: package_deployment
 
     on:

docs/docs.rst

@@ -119,10 +119,11 @@ mentioned in ``dependencies``):
 .. code-block:: toml
 
     [project.optional-dependencies]
-    docs =
-        sphinx
-        sphinx-automodapi
-        numpydoc
+    docs = [
+        "sphinx",
+        "sphinx-automodapi",
+        "numpydoc",
+    ]
 
 This will then allow contributors to type::
 

docs/index.rst

@@ -22,7 +22,22 @@ Using the Template
 
 With this guide is a `cookiecutter <https://cookiecutter.readthedocs.io/>`__ template which allows you to get started quickly with a package as described in this guide.
 
-To get started run:
+To create a new package based on the template run:
+
+.. code-block:: console
+
+   $ pip install cookiecutter cruft
+   $ cruft create https://github.com/OpenAstronomy/packaging-guide
+
+and go through the steps offered in the cli naming your package and filling in your details.
+Cruft is built on cookiecutter, and enables the updating of the template from the source.
+This takes the form of pull requests to the repository that the new package is pushed to.
+If a package already has a cookiecutter template, it can be linked to the parent repository using ``cruft link url-to-template``.
+
+To manually check whether the current environment matches with the template then ``cruft check`` will tell you what the current status is.
+``cruft update`` will manually trigger an updating of the package to the template.
+
+If you would like to stick to simply the cookiecutter approach, the template still supports that functionality thusly:
 
 .. code-block:: console
 

hooks/post_gen_project.py

@@ -36,10 +36,17 @@ def process_version(enable_dynamic_dev_versions):
         remove_dir(os.path.join(PROJECT_DIRECTORY, '{{ cookiecutter.module_name }}', '_dev'))
         remove_file(os.path.join(PROJECT_DIRECTORY, '{{ cookiecutter.module_name }}', 'version.py'))
 
+
+def process_github_workflow(include_cruft_update_github_workflow):
+    if include_cruft_update_github_workflow != "y":
+        remove_dir(os.path.join(PROJECT_DIRECTORY,  '.github'))
+
+
 if __name__ == '__main__':
 
     process_license('{{ cookiecutter.license }}')
     process_version('{{ cookiecutter.enable_dynamic_dev_versions }}')
+    process_github_workflow('{{ cookiecutter.include_cruft_update_github_workflow }}')
     include_examples = '{{ cookiecutter.include_example_code }}' == 'y'
     use_compiled = '{{ cookiecutter.use_compiled_extensions }}' == 'y'
 

{{ cookiecutter.package_name }}/.github/workflows/sub_package_update.yml

@@ -0,0 +1,80 @@
+"""
+This template is taken from the cruft example code, for further information please see:
+https://cruft.github.io/cruft/#automating-updates-with-github-actions
+"""
+name: Automatic Update from package template
+permissions:
+    contents: write
+    pull-requests: write
+
+on:
+  pull_request:
+    branches:
+      main
+jobs:
+  update:
+    runs-on: ubuntu-latest
+    strategy:
+      fail-fast: true
+      matrix:
+        include:
+          - add-paths: .
+            body: Use this to merge the changes to the repo
+            branch: cruft/update
+            commit-message: "Automate package template update"
+            title: Incoming updates from package template
+          - add-paths: .cruft.json
+            body: Use this to reject changes in the repo
+            branch: cruft/reject
+            commit-message: "Chore: reject this cruft update"
+            title: Reject new updates from package template
+
+    steps:
+      - uses: actions/checkout@v3
+
+      - uses: actions/setup-python@v4
+        with:
+          python-version: "3.10"
+
+      - name: Install Cruft
+        run: pip3 install cruft
+
+      - name: Check if update is available
+        continue-on-error: false
+        id: check
+        run: |
+          CHANGES=0
+          if [ -f .cruft.json ]; then
+            if ! cruft check; then 
+              CHANGES=1
+            fi
+          else
+            echo "No .cruft.json file"
+          fi
+          
+          echo "has_changes=$CHANGES" >> "$GITHUB_OUTPUT"
+
+      - name: Run update if available
+        if: steps.check.outputs.has_changes == '1'
+        run: |
+          git config --global user.email "[email protected]"
+          git config --global user.name "Gromit"
+          
+          cruft update --skip-apply-ask --refresh-private-variables
+          git restore --staged
+
+      - name: Create pull request
+        if: steps.check.output.has_changes == '1'
+        uses: peter-evans/create-pull-request@v4
+        with:
+          token: ${{ secrets.GITHUB_TOKEN }}
+          add-paths: ${{ matrix.add-paths }}
+          commit-message: ${{ matrix.commit-message }}
+          branch: ${{ matrix.branch }}
+          delete-branch: true
+          branch-suffix: timestamp
+          title: ${{ matrix.title }}
+          body: |
+            This is an autogenerated PR. ${{ matrix.body }}
+              [Cruft](https://cruft.github.io/cruft/) has detected updates from the Package Template 
+

{{ cookiecutter.package_name }}/.gitignore

@@ -25,6 +25,11 @@ share/python-wheels/
 .installed.cfg
 *.egg
 MANIFEST
+{%- if cookiecutter.enable_dynamic_dev_versions == 'y' %}
+{{ cookiecutter.module_name }}/_version.py
+{% else %}
+{{ cookiecutter.module_name }}/version.py
+{% endif -%}
 
 # PyInstaller
 #  Usually these files are written by a python script from a template

{{ cookiecutter.package_name }}/MANIFEST.in

@@ -5,9 +5,9 @@ prune build
 prune docs/_build
 prune docs/api
 global-exclude *.pyc *.o
-{% if cookiecutter.enable_dynamic_dev_versions == 'y' %}
+{%- if cookiecutter.enable_dynamic_dev_versions == 'y' %}
 
 # This subpackage is only used in development checkouts
 # and should not be included in built tarballs
 prune {{ cookiecutter.module_name }}/_dev
-{% endif %}
+{% endif -%}