🎉 Source GitHub: PullRequestStats and Reviews streams now respects start date property

[jlourenc]

What

Closes #7866

How

PullRequestStats and Reviews streams are now defined as PullRequest substreams, by inheriting from HttpSubStream via PullRequestSubstream classes.

This ensures that the start_date property, part of the configuration of the PullRequest stream, is considered. Consequently, the start_date also applies to the substreams, since their stream_slices are derived from their parent stream.

Recommended reading order

1. streams.py
2. source.py

🚨 User Impact 🚨

If start_date is defined, PullRequestStats and Reviews streams only refresh from that date, instead of refreshing all before.

Pre-merge Checklist

Community member or Airbyter

• Grant edit access to maintainers (instructions)
• Secrets in the connector's spec are annotated with airbyte_secret
• Unit & integration tests added and passing. Community members, please provide proof of success locally e.g: screenshot or copy-paste unit, integration, and acceptance test output. To run acceptance tests for a Python connector, follow instructions in the README. For java connectors run ./gradlew :airbyte-integrations:connectors:<name>:integrationTest.
• Code reviews completed
• Documentation updated
  • Connector's README.md
  • Connector's bootstrap.md. See description and examples
  • Changelog updated in docs/integrations/<source or destination>/<name>.md including changelog. See changelog example
• PR name follows PR naming conventions

Airbyter

If this is a community PR, the Airbyte engineer reviewing this PR is responsible for the below items.

• Create a non-forked branch based on this PR and test the below items on it
• Build is successful
• Credentials added to Github CI. Instructions.
• /test connector=connectors/<name> command is passing.
• New Connector version released on Dockerhub by running the /publish command described here
• After the new connector version is published, connector version bumped in the seed directory as described here
• Seed specs have been re-generated by building the platform and committing the changes to the seed spec files, as described here

[CLAassistant]

All committers have signed the CLA.

[jlourenc]

/test connector=connectors/source-github

[jlourenc]

❯ python -m pytest unit_tests
Test session starts (platform: darwin, Python 3.8.9, pytest 6.2.5, pytest-sugar 0.9.4)
cachedir: .pytest_cache
rootdir: /Users/***/***/airbyte, configfile: pytest.ini
plugins: sugar-0.9.4, timeout-1.4.2
collecting ...
 airbyte-integrations/connectors/source-github/unit_tests/unit_test.py::test_single_token ✓                                                                                                                     50% █████
 airbyte-integrations/connectors/source-github/unit_tests/unit_test.py::test_multiple_tokens ✓                                                                                                                 100% ██████████
====================================================================================================== warnings summary ======================================================================================================
airbyte-integrations/connectors/source-github/unit_tests/unit_test.py::test_single_token
airbyte-integrations/connectors/source-github/unit_tests/unit_test.py::test_single_token
airbyte-integrations/connectors/source-github/unit_tests/unit_test.py::test_single_token
airbyte-integrations/connectors/source-github/unit_tests/unit_test.py::test_multiple_tokens
  /Users/***/***/airbyte/airbyte-integrations/connectors/source-github/source_github/source.py:75: DeprecationWarning: Call to deprecated class MultipleTokenAuthenticator. (Use airbyte_cdk.sources.streams.http.requests_native_auth.MultipleTokenAuthenticator instead) -- Deprecated since version 0.1.20.
    return MultipleTokenAuthenticator(tokens=tokens, auth_method="token")

airbyte-integrations/connectors/source-github/unit_tests/unit_test.py::test_single_token
airbyte-integrations/connectors/source-github/unit_tests/unit_test.py::test_single_token
airbyte-integrations/connectors/source-github/unit_tests/unit_test.py::test_single_token
airbyte-integrations/connectors/source-github/unit_tests/unit_test.py::test_multiple_tokens
  /Users/***/***/airbyte/.venv/lib/python3.8/site-packages/deprecated/classic.py:173: DeprecationWarning: Call to deprecated class HttpAuthenticator. (Use requests.auth.AuthBase instead) -- Deprecated since version 0.1.20.
    return old_new1(cls, *args, **kwargs)

-- Docs: https://docs.pytest.org/en/stable/warnings.html

Results (0.22s):
       2 passed

[harshithmullapudi]

@jlourenc thanks for your contribution. Can you also update the version of connector. You can follow this https://docs.airbyte.io/connector-development#4.-publish-the-connector

[jlourenc]

@harshithmullapudi That's done. Let me know wether or not this is what you were asking for.

Also, the documentation states:

Typically this will be handled as part of code review by an Airbyter.

Hence, I was expecting this to be done last, by an Airbyter, once the review is almost complete, in case another PR for the GitHub connector cause a versioning conflict.

[yevhenii-ldv]

Looks great in general, can make a couple of cosmetic fixes and we can prepare it for merge.

[jlourenc]

Hey @yevhenii-ldv, thanks for the review. I've applied your suggestions, fixed the merged conflicts and bumped the connector version to 0.2.6.

[yevhenii-ldv]

Hi @jlourenc, we have just merged into master branch your changes and released new version for Github connector.
Thank you very much for your contribution.

Merged as #8233