Merge branch 'dev' into s3fs

.coveragerc

@@ -5,6 +5,7 @@ omit =
     src/pynwb/_version.py
     src/pynwb/_due.py
     src/pynwb/testing/*
+    src/pynwb/legacy/*
 
 [report]
 exclude_lines =

.github/workflows/run_coverage.yml

@@ -0,0 +1,82 @@
+name: Run code coverage
+on:
+  push:
+    branches:
+      - dev
+    tags-ignore:  # exclude tags created by "ci_addons publish_github_release"
+      - 'latest'
+      - 'latest-tmp'
+  pull_request:
+  workflow_dispatch:
+
+jobs:
+  run-coverage:
+    name: ${{ matrix.os }}
+    runs-on: ${{ matrix.os }}
+    # TODO
+    # run pipeline on either a push event or a PR event on a fork
+    # if: github.event_name != 'pull_request' || github.event.pull_request.head.repo.full_name != github.event.pull_request.base.repo.full_name
+    defaults:
+      run:
+        shell: bash
+    strategy:
+      matrix:
+        os: [ubuntu-latest, macos-latest, windows-latest]
+    env:
+      OS: ${{ matrix.os }}
+      PYTHON: '3.10'
+    steps:
+      - name: Cancel any previous incomplete runs
+        uses: styfle/[email protected]
+        with:
+          all_but_latest: true
+          access_token: ${{ github.token }}
+
+      - uses: actions/checkout@v3
+        with:
+          submodules: 'recursive'
+
+      - name: Set up Python
+        uses: actions/setup-python@v3
+        with:
+          python-version: '3.9'
+
+      - name: Install dependencies
+        run: |
+          python -m pip install --upgrade pip
+          python -m pip install -r requirements-dev.txt -r requirements.txt
+
+      - name: Install optional dependencies
+        if: ${{ matrix.opt_req }}
+        run: python -m pip install -r requirements-opt.txt
+
+      - name: Install package
+        run: |
+          python -m pip install -e .  # must install in editable mode for coverage to find sources
+          python -m pip list
+
+      - name: Run unit tests and generate coverage report
+        run: |
+          python -m coverage run test.py --pynwb
+          python -m coverage xml  # codecov uploader requires xml format
+          python -m coverage report -m
+
+      - name: Upload coverage to Codecov
+        uses: codecov/codecov-action@v3
+        with:
+          flags: unit
+          fail_ci_if_error: true
+
+      - name: Run integration tests and generate coverage report
+        run: |
+          python -m coverage run -p test.py --integration
+          # validation CLI tests generate separate .coverage files that need to be merged
+          python -m coverage combine
+          python -m coverage xml  # codecov uploader requires xml format
+          python -m coverage report -m
+
+      - name: Upload coverage to Codecov
+        uses: codecov/codecov-action@v3
+        with:
+          flags: integration
+          fail_ci_if_error: true

CHANGELOG.md

@@ -2,11 +2,18 @@
 
 ## PyNWB 2.1.1 (Upcoming)
 
-### Enhancements and minor changes:
-- Enable passing an S3File created through s3fs, which provides a method for reading an NWB file directly from s3 that is an alternative to ros3. This required relaxing of `NWBHDF5IO` input validation. The `path` arg is not needed if `file` is provided. `mode` now has a default value of "r". @bendichter ([1499](https://github.com/NeurodataWithoutBorders/pynwb/pull/1499))
-
 ### Documentation and tutorial enhancements:
-- Add example code for s3fs in the streaming tutorial.
+- Support explicit ordering of sphinx gallery tutorials in the docs. @oruebel (#1504), @bdichter (#1495)
+- Add developer guide on how to create a new tutorial. @oruebel (#1504) 
+- Add images tutorial. @weiglszonja (#1470)
+- Add example code for s3fs in the streaming tutorial. @bdichter (#1499)
+
+### Enhancements and minor changes
+- Update coverage workflow, report separate unit vs integration coverage. @rly (#1509)
+- Enable passing an S3File created through s3fs, which provides a method for reading an NWB file directly 
+  from s3 that is an alternative to ros3. This required relaxing of `NWBHDF5IO` input validation. The `path`
+  arg is not needed if `file` is provided. `mode` now has a default value of "r". 
+  @bendichter (#1499)
 
 ## PyNWB 2.1.0 (July 6, 2022)
 

README.rst

@@ -1,7 +1,7 @@
 .. image:: docs/source/figures/logo_pynwb.png
     :width: 200px
 
-PyNWB is a Python package for working with NWB files. The PyNWB 
+PyNWB is a Python package for working with NWB files. The PyNWB
 documentation can be found at https://pynwb.readthedocs.io
 
 Latest Release
@@ -34,15 +34,30 @@ Build Status
      :target: https://circleci.com/gh/conda-forge/pynwb-feedstocks
 
 
-Overall Health
+Code Coverage
 ==============
 
 .. image:: https://github.com/NeurodataWithoutBorders/pynwb/workflows/Run%20coverage/badge.svg
     :target: https://github.com/NeurodataWithoutBorders/pynwb/actions?query=workflow%3A%22Run+coverage%22
 
+Overall test coverage
+
 .. image:: https://codecov.io/gh/NeurodataWithoutBorders/pynwb/branch/dev/graph/badge.svg
     :target: https://codecov.io/gh/NeurodataWithoutBorders/pynwb
 
+Unit test coverage
+
+.. image:: https://codecov.io/gh/NeurodataWithoutBorders/pynwb/branch/dev/graph/badge.svg?flag=unit
+    :target: https://codecov.io/gh/NeurodataWithoutBorders/pynwb
+
+Integration test coverage
+
+.. image:: https://codecov.io/gh/NeurodataWithoutBorders/pynwb/branch/dev/graph/badge.svg?flag=integration
+    :target: https://codecov.io/gh/NeurodataWithoutBorders/pynwb
+
+Overall Health
+==============
+
 .. image:: https://requires.io/github/NeurodataWithoutBorders/pynwb/requirements.svg?branch=dev
      :target: https://requires.io/github/NeurodataWithoutBorders/pynwb/requirements/?branch=dev
      :alt: Requirements Status

docs/CONTRIBUTING.rst

@@ -18,30 +18,30 @@ Did you find a bug? or Do you intend to add a new feature or change an existing
 
 * **Identify the appropriate repository** for the change you are suggesting:
 
-   * Use `nwb-schema <https://github.com/NeurodataWithoutBorders/nwb-schema/>`_ for any changes to the NWB:N format schema, schema language, storage, and other NWB:N related documents
+   * Use `nwb-schema <https://github.com/NeurodataWithoutBorders/nwb-schema/>`_ for any changes to the NWB format schema, schema language, storage, and other NWB related documents
    * Use `PyNWB <https://github.com/NeurodataWithoutBorders/pynwb>`_  for any changes regarding the PyNWB API and the corresponding documentation
-   * Use `MatNWB <https://github.com/NeurodataWithoutBorders/matnwb>`_  for any changes regarding the PyNWB API and the corresponding documentation
+   * Use `MatNWB <https://github.com/NeurodataWithoutBorders/matnwb>`_  for any changes regarding the MatNWB API and the corresponding documentation
 
 * **Ensure the feature or change was not already reported** by searching on GitHub under `PyNWB Issues <https://github.com/NeurodataWithoutBorders/pynwb/issues>`_ and `nwb-schema issues <https://github.com/NeurodataWithoutBorders/nwb-schema/issues>`_, respectively .
 
 * If you are unable to find an open issue addressing the problem then open a new issue on the respective repository. Be sure to include:
 
     * **brief and descriptive title**
-    * **clear description of the problem you are trying to solve***. Describing the use case is often more important than proposing a specific solution. By describing the use case and problem you are trying to solve gives the development team and ultimately the NWB:N community a better understanding for the reasons of changes and enables others to suggest solutions.
+    * **clear description of the problem you are trying to solve***. Describing the use case is often more important than proposing a specific solution. By describing the use case and problem you are trying to solve gives the development team and ultimately the NWB community a better understanding for the reasons of changes and enables others to suggest solutions.
     * **context** providing as much relevant information as possible and if available a **code sample** or an **executable test case** demonstrating the expected behavior and/or problem.
 
 * Be sure to select the appropriate labels (see :ref:`sec-issue-labels`) for your tickets so that they can be processed accordingly.
 
-* NWB:N is currently being developed primarily by staff at scientific research institutions and industry, most of which work on many different research projects. Please be patient, if our development team is not able to respond immediately to your issues. In particular issues that belong to later project milestones may not be reviewed or processed until work on that milestone begins.
+* NWB is currently being developed primarily by staff at scientific research institutions and industry, most of which work on many different research projects. Please be patient, if our development team is not able to respond immediately to your issues. In particular issues that belong to later project milestones may not be reviewed or processed until work on that milestone begins.
 
 Did you write a patch that fixes a bug or implements a new feature?
 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
 See the ``Contributing Patches and Changes`` section below for details.
 
-Do you have questions about NWB:N?
+Do you have questions about NWB?
 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
 
-Ask questions on our `Slack workspace <https://nwb-users.slack.com>`_ or sign up for our `NWB:N mailing list <http://visitor.r20.constantcontact.com/manage/optin?v=001nQUq2GTjwCjZxK_V2-6RLElLJO1HMVtoNLJ-wGyDCukZQZxu2AFJmNh6NS0_lGMsWc2w9hZpeNn74HuWdv5RtLX9qX0o0Hy1P0hOgMrkm2NoGAX3VoY25wx8HAtIZwredcCuM0nCUGodpvoaue3SzQ%3D%3D>`_ for updates.
+Ask questions on our `Slack workspace <https://nwb-users.slack.com>`_ or sign up for our `NWB mailing list <http://visitor.r20.constantcontact.com/manage/optin?v=001nQUq2GTjwCjZxK_V2-6RLElLJO1HMVtoNLJ-wGyDCukZQZxu2AFJmNh6NS0_lGMsWc2w9hZpeNn74HuWdv5RtLX9qX0o0Hy1P0hOgMrkm2NoGAX3VoY25wx8HAtIZwredcCuM0nCUGodpvoaue3SzQ%3D%3D>`_ for updates.
 
 Informal discussions between developers and users?
 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

docs/gallery/advanced_io/iterative_write.py

@@ -160,7 +160,7 @@ def iter_sin(chunk_length=10, max_chunks=100):
     """
     x = 0
     num_chunks = 0
-    while(x < 0.5 and num_chunks < max_chunks):
+    while (x < 0.5 and num_chunks < max_chunks):
         val = np.asarray([sin(random() * 2 * pi) for i in range(chunk_length)])
         x = random()
         num_chunks += 1