Merge branch 'master' into enable_spelling_check

* master:
  refactor setup.py to setup.cfg (SciTools#4168)
  update docs pypi release (SciTools#4173)
  Update CI environment lockfiles (SciTools#4137)
  update CONTRIBUTING.md (SciTools#4165)
  RTD support link update (SciTools#4166)
  drop py36 support (SciTools#4163)
  github issues contact link for discussions (SciTools#4164)
  Bump black version (SciTools#4162)
  Stop CI from clobbering commits on lockfile updates (SciTools#4157)
  [pre-commit.ci] pre-commit autoupdate (SciTools#4161)
  Add a method to return a CubeList from CubeList.copy() (SciTools#4094)
  Update black et al (SciTools#4155)

.cirrus.yml

@@ -137,8 +137,6 @@ test_minimal_task:
   only_if: ${SKIP_TEST_MINIMAL_TASK} == "" && ${SKIP_ALL_TEST_TASKS} == ""
   << : *CREDITS_TEMPLATE
   matrix:
-    env:
-      PY_VER: 3.6
     env:
       PY_VER: 3.7
     env:
@@ -159,8 +157,6 @@ test_full_task:
   only_if: ${SKIP_TEST_FULL_TASK} == "" && ${SKIP_ALL_TEST_TASKS} == ""
   << : *CREDITS_TEMPLATE
   matrix:
-    env:
-      PY_VER: 3.6
     env:
       PY_VER: 3.7
     env:

.github/CONTRIBUTING.md

@@ -1 +1,13 @@
-See the [Developers Guide](https://scitools-iris.readthedocs.io/en/latest/index.html#development-index)
+# Contributing to Iris
+
+Want to contribute to Iris? But don't know where to start 🤔
+
+We recommend that you first checkout our advice to [First Time Contributors](https://github.com/SciTools/iris/issues/4133),
+which has some solid suggestions on where to begin.
+
+Otherwise, head over to our [Developers Guide on Getting Involved](https://scitools-iris.readthedocs.io/en/stable/developers_guide/contributing_getting_involved.html)
+for all the information you need to start on your Iris development journey.
+
+Still need help or advice?
+
+Then join us over on our [Iris GitHub Discussions](https://github.com/SciTools/iris/discussions). We'd love to hear from you!

.github/ISSUE_TEMPLATE/config.yml

@@ -1,2 +1,6 @@
 # reference: https://docs.github.com/en/github/building-a-strong-community/configuring-issue-templates-for-your-repository#configuring-the-template-chooser
 blank_issues_enabled: false
+contact_links:
+  -  name: 💬 Iris GitHub Discussions
+     url: https://github.com/SciTools/iris/discussions
+     about: Engage with the Iris community to discuss your issue

.github/workflows/refresh-lockfiles.yml

@@ -14,24 +14,62 @@ name: Refresh Lockfiles
 
 on:
   workflow_dispatch:
+    inputs:
+      clobber:
+        description: |
+          Force the workflow to run, potentially clobbering any commits already made to the branch. 
+          Enter "yes" or "true" to run.
+        default: "no"
   schedule:
     # Run once a week on a Saturday night 
     - cron: 1 0 * * 6
 
 
 jobs:
 
+  no_clobber:
+    runs-on: ubuntu-latest
+    steps:
+      # check if the auto-update-lockfiles branch exists.  If it does, and someone other than
+      # the lockfile bot has made the head commit, abort the workflow.
+      # This job can be manually overridden by running directly from the github actions panel
+      # (known as a "workflow_dispatch") and setting the `clobber` input to "yes".
+      - uses: actions/script@v4
+        with:
+          github-token: ${{ secrets.GITHUB_TOKEN }}
+          script: |
+            if (context.eventName == "workflow_dispatch") {
+              const clobber = context.payload.inputs.clobber || "no";
+              if (["yes", "true", "y"].includes(clobber.trim().toLowerCase())) {
+                core.info("Manual override, continuing workflow, potentially overwriting previous commits to auto-update-lockfiles");
+                return
+              }
+            }
+            github.repos.getBranch({...context.repo, branch: "auto-update-lockfiles"}).then(res => {
+              const committer = res.data.commit.commit.committer;
+              if (committer && committer.name === "Lockfile bot") {
+                core.info("Lockfile bot was the last to push to auto-update-lockfiles. Continue.");
+              } else {
+                core.setFailed("New commits to auto-update-lockfiles since bot last ran. Abort!");
+              }
+            }).catch(err => {
+              if (err.status === 404) {
+                  core.info("auto-update-lockfiles branch not found, continue");
+              }
+            })
+
   gen_lockfiles:
     # this is a matrix job: it splits to create new lockfiles for each
     # of the CI test python versions.
     # this list below should be changed when covering more python versions
     # TODO: generate this matrix automatically from the list of available py**.yml files
     #       ref: https://tomasvotruba.com/blog/2020/11/16/how-to-make-dynamic-matrix-in-github-actions/
     runs-on: ubuntu-latest
+    needs: no_clobber
 
     strategy:
       matrix:
-        python: ['36', '37', '38']
+        python: ['37', '38']
 
     steps:
       - uses: actions/checkout@v2
@@ -62,14 +100,16 @@ jobs:
           path: artifacts
 
       - name: Update lock files in repo
-        run: | 
+        run: |
           cp artifacts/artifact/*.lock requirements/ci/nox.lock
           rm -r artifacts
         
       - name: Create Pull Request
         uses: peter-evans/create-pull-request@052fc72b4198ba9fbc81b818c6e1859f747d49a8
         with:
           commit-message: Updated environment lockfiles
+          committer: "Lockfile bot <[email protected]>"
+          author: "Lockfile bot <[email protected]>"
           delete-branch: true
           branch: auto-update-lockfiles
           title: Update CI environment lockfiles

.pre-commit-config.yaml

@@ -2,7 +2,7 @@
 # See https://pre-commit.com/hooks.html for more hooks
 repos:
 -   repo: https://github.com/pre-commit/pre-commit-hooks
-    rev: 'v3.4.0'
+    rev: 'v4.0.1'
     hooks:
         # Prevent giant files from being committed.
     -   id: check-added-large-files
@@ -17,14 +17,14 @@ repos:
         # Don't commit to master branch.
     -   id: no-commit-to-branch
 -   repo: https://github.com/psf/black
-    rev: '20.8b1'
+    rev: '21.5b2'
     hooks:
     -   id: black
         # Force black to run on whole repo, using settings from pyproject.toml
         pass_filenames: false
         args: [--config=./pyproject.toml, .]
 -   repo: https://github.com/PyCQA/flake8
-    rev: '3.9.0'
+    rev: '3.9.2'
     hooks:
         # Run flake8.
     -   id: flake8

MANIFEST.in

@@ -3,19 +3,22 @@ include CHANGES COPYING COPYING.LESSER
 
 # Files from setup.py package_data that are not automatically added to source distributions
 recursive-include lib/iris/tests/results *.cml *.cdl *.txt *.xml *.json
+recursive-exclude lib/iris/fileformats/_pyke_rules/compiled_krb *
 recursive-include lib/iris/etc *
-include lib/iris/fileformats/_pyke_rules/*.k?b
-include lib/iris/tests/stock*.npz
+include lib/iris/fileformats/_pyke_rules/*.krb
 
-include requirements/*.txt
+recursive-include requirements *
 
 # File required to build docs
 recursive-include docs Makefile *.js *.png *.py *.rst
-prune docs/build
+prune docs/src/_build
+prune docs/src/generated
+prune docs/gallery_tests
 
 # Files required to build std_names module
 include tools/generate_std_names.py
 include etc/cf-standard-name-table.xml
 
 global-exclude *.pyc
 global-exclude __pycache__
+global-exclude iris_image_test_output

docs/src/common_links.inc

@@ -11,9 +11,9 @@
 .. _core developers: https://github.com/SciTools/scitools.org.uk/blob/master/contributors.json
 .. _generating sss keys for GitHub: https://docs.github.com/en/github/authenticating-to-github/adding-a-new-ssh-key-to-your-github-account
 .. _GitHub Help Documentation: https://docs.github.com/en/github
+.. _Iris GitHub Discussions: https://github.com/SciTools/iris/discussions
 .. _Iris: https://github.com/SciTools/iris
 .. _Iris GitHub: https://github.com/SciTools/iris
-.. _iris mailing list: https://groups.google.com/forum/#!forum/scitools-iris
 .. _iris-sample-data: https://github.com/SciTools/iris-sample-data
 .. _iris-test-data: https://github.com/SciTools/iris-test-data
 .. _issue: https://github.com/SciTools/iris/issues

docs/src/conf.py

@@ -19,9 +19,15 @@
 
 # ----------------------------------------------------------------------------
 
+import datetime
 import ntpath
 import os
+from pathlib import Path
+import re
 import sys
+import warnings
+
+import iris
 
 
 # function to write  useful output to stdout, prefixing the source.
@@ -53,9 +59,6 @@ def autolog(message):
 # add these directories to sys.path here. If the directory is relative to the
 # documentation root, use os.path.abspath to make it absolute, like shown here.
 
-import datetime
-import warnings
-
 # custom sphinx extensions
 sys.path.append(os.path.abspath("sphinxext"))
 
@@ -80,8 +83,6 @@ def autolog(message):
 # |version| and |release|, also used in various other places throughout the
 # built documents.
 
-import iris
-
 # The short X.Y version.
 if iris.__version__ == "dev":
     version = "dev"
@@ -101,9 +102,35 @@ def autolog(message):
 
 build_python_version = ".".join([str(i) for i in sys.version_info[:3]])
 
+
+def _dotv(version):
+    result = version
+    match = re.match(r"^py(\d+)$", version)
+    if match:
+        digits = match.group(1)
+        if len(digits) > 1:
+            result = f"{digits[0]}.{digits[1:]}"
+    return result
+
+
+# Automate the discovery of the python versions tested with CI.
+python_support = sorted(
+    [fname.stem for fname in Path(".").glob("../../requirements/ci/py*.yml")]
+)
+
+if not python_support:
+    python_support = "unknown Python versions"
+elif len(python_support) == 1:
+    python_support = f"Python {_dotv(python_support[0])}"
+else:
+    rest = ", ".join([_dotv(v) for v in python_support[:-1]])
+    last = _dotv(python_support[-1])
+    python_support = f"Python {rest} and {last}"
+
 rst_epilog = f"""
 .. |copyright_years| replace:: {copyright_years}
 .. |python_version| replace:: {build_python_version}
+.. |python_support| replace:: {python_support}
 .. |iris_version| replace:: v{version}
 .. |build_date| replace:: ({datetime.datetime.now().strftime('%d %b %Y')})
 """
@@ -243,12 +270,8 @@ def autolog(message):
             "https://github.com/SciTools/iris",
         ),
         (
-            '<i class="fa fa-comments fa-fw"></i> Users Google Group',
-            "https://groups.google.com/forum/#!forum/scitools-iris",
-        ),
-        (
-            '<i class="fa fa-comments fa-fw"></i> Developers Google Group',
-            "https://groups.google.com/forum/#!forum/scitools-iris-dev",
+            '<i class="fa fa-comments fa-fw"></i> GitHub Discussions',
+            "https://github.com/SciTools/iris/discussions",
         ),
         (
             '<i class="fa fa-question fa-fw"></i> StackOverflow for "How Do I?"',

docs/src/developers_guide/contributing_ci_tests.rst

@@ -66,6 +66,16 @@ and add the changed lockfiles to your pull request.
 
 New lockfiles are generated automatically each week to ensure that Iris continues to be
 tested against the latest available version of its dependencies.
+Each week the yaml files in ``requirements/ci`` are resolved by a GitHub Action.
+If the resolved environment has changed, a pull request is created with the new lock files.
+The CI test suite will run on this pull request and fixes for failed tests can be pushed to
+the ``auto-update-lockfiles`` branch to be included in the PR. 
+Once a developer has pushed to this branch, the auto-update process will not run again until
+the PR is merged, to prevent overwriting developer commits.
+The auto-updater can still be invoked manually in this situation by going to the `GitHub Actions`_
+page for the workflow, and manually running using the "Run Workflow" button.  
+By default, this will also not override developer commits.  To force an update, you must 
+confirm "yes" in the "Run Worflow" prompt.
 
 
 .. _skipping Cirrus-CI tasks:
@@ -137,3 +147,4 @@ See the `pre-commit.ci dashboard`_ for details of recent past and active Iris jo
 .. _Cirrus-CI Documentation: https://cirrus-ci.org/guide/writing-tasks/
 .. _.pre-commit-config.yaml: https://github.com/SciTools/iris/blob/master/.pre-commit-config.yaml
 .. _pre-commit.ci dashboard: https://results.pre-commit.ci/repo/github/5312648
+.. _GitHub Actions: https://github.com/SciTools/iris/actions/workflows/refresh-lockfiles.yml

docs/src/developers_guide/gitwash/development_workflow.rst

@@ -25,7 +25,7 @@ In what follows we'll refer to the upstream iris ``master`` branch, as
 * If you can possibly avoid it, avoid merging trunk or any other branches into
   your feature branch while you are working.
 * If you do find yourself merging from trunk, consider :ref:`rebase-on-trunk`
-* Ask on the `iris mailing list`_ if you get stuck.
+* Ask on the `Iris GitHub Discussions`_ if you get stuck.
 * Ask for code review!
 
 This way of working helps to keep work well organized, with readable history.

docs/src/developers_guide/release.rst

@@ -107,10 +107,11 @@ source distribution (``sdist``) and pure Python wheel (``bdist_wheel``)::
     > conda create -n iris-pypi -c conda-forge --yes pip pyke python setuptools twine wheel
     > . activate iris-pypi
 
-Checkout the latest Iris ``<release>`` tag::
+Checkout the appropriate Iris ``<release>`` tag from the appropriate ``<repo>``.
+For example, to checkout tag ``v1.0`` from ``upstream``::
 
-    > git fetch --tags
-    > git checkout <release>
+    > git fetch upstream --tags
+    > git checkout v1.0 
 
 Build the source distribution and wheel from the Iris root directory::
 
@@ -131,7 +132,7 @@ To list and check the contents of the binary wheel::
 If all seems well, sufficient maintainer privileges will be required to
 upload these artifacts to `scitools-iris`_ on PyPI::
 
-    > python -m twine upload --repository-url https://upload.pypi.org/legecy/ dist/*
+    > python -m twine upload --repository-url https://upload.pypi.org/legacy/ dist/*
 
 Ensure that the artifacts are successfully uploaded and available on
 `scitools-iris`_ before creating a conda test environment to install Iris

docs/src/installing.rst

@@ -16,8 +16,7 @@ any WSL_ distributions.
 
 .. _WSL: https://docs.microsoft.com/en-us/windows/wsl/install-win10
 
-.. note:: Iris is currently supported and tested against Python ``3.6``,
-          ``3.7``, and ``3.8``.
+.. note:: Iris is currently supported and tested against |python_support|.
 
 .. note:: This documentation was built using Python |python_version|.