Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[FEATURE] Experimental Python 3.12 support #8862

Merged
merged 64 commits into from
Jun 17, 2024
Merged

[FEATURE] Experimental Python 3.12 support #8862

merged 64 commits into from
Jun 17, 2024

Conversation

Kilo59
Copy link
Contributor

@Kilo59 Kilo59 commented Oct 24, 2023

Update to make great_expectations and the test CI compatible with Python 3.12.

To install great_expectations on Python 3.12 a GX_PYTHON_EXPERIMENTAL environment variable must be set.

Changes

  1. Update versioneer usage of configparser https://docs.python.org/3.12/whatsnew/3.12.html#configparser
  2. Prevent snapshottest running in CI for 3.12
  3. Filter deprecated datetime usage warnings.
    • update DTZ ruff rule to include test usage
  4. Update moto to deal with ssl changes. - https://docs.getmoto.org/en/latest/docs/releases.html
  5. Set python_requirest = ">=3.8,<3.12" unless GX_PYTHON_EXPERIMENTAL env var is found.

Notes

Waiting on pandas support for 3.12 before resuming this work.

CI currently installs pact-python for every test job (even if it isn't being used).

@Kilo59 Kilo59 self-assigned this Oct 24, 2023
@netlify
Copy link

netlify bot commented Oct 24, 2023

Deploy Preview for niobium-lead-7998 canceled.

Name Link
🔨 Latest commit c257a8f
🔍 Latest deploy log https://app.netlify.com/sites/niobium-lead-7998/deploys/6670705f7506a20008b369b1

@github-actions github-actions bot added the core label Oct 24, 2023
@ghost
Copy link

ghost commented Oct 24, 2023

👇 Click on the image for a new way to code review

Review these changes using an interactive CodeSee Map

Legend

CodeSee Map legend

Comment on lines +342 to +344
parser = configparser.ConfigParser()
with open(setup_cfg) as f:
parser.readfp(f)
parser.read_file(f)
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

only install for cloud tests
@Kilo59 Kilo59 marked this pull request as draft October 24, 2023 18:16
Copy link

codecov bot commented May 23, 2024

Codecov Report

Attention: Patch coverage is 63.63636% with 4 lines in your changes missing coverage. Please review.

Project coverage is 79.55%. Comparing base (37cc3f7) to head (c257a8f).

Files Patch % Lines
...eat_expectations/compatibility/docstring_parser.py 63.63% 4 Missing ⚠️
Additional details and impacted files
@@             Coverage Diff             @@
##           develop    #8862      +/-   ##
===========================================
- Coverage    79.55%   79.55%   -0.01%     
===========================================
  Files          456      456              
  Lines        39407    39410       +3     
===========================================
+ Hits         31351    31353       +2     
- Misses        8056     8057       +1     
Flag Coverage Δ
3.10 66.05% <63.63%> (+<0.01%) ⬆️
3.11 66.05% <63.63%> (+<0.01%) ⬆️
3.11 athena or clickhouse or openpyxl or pyarrow or project or sqlite or aws_creds ?
3.11 aws_deps ?
3.11 big ?
3.11 databricks ?
3.11 filesystem ?
3.11 mssql ?
3.11 mysql ?
3.11 postgresql ?
3.11 snowflake ?
3.11 spark ?
3.11 trino ?
3.12 64.62% <63.63%> (?)
3.12 aws_deps 45.85% <63.63%> (?)
3.12 big 54.63% <63.63%> (?)
3.12 filesystem 60.43% <63.63%> (?)
3.12 mssql 49.99% <63.63%> (?)
3.12 mysql 50.05% <63.63%> (?)
3.12 postgresql 54.20% <63.63%> (?)
3.12 spark 57.42% <63.63%> (?)
3.12 trino 52.12% <63.63%> (?)
3.8 66.10% <63.63%> (+0.01%) ⬆️
3.8 athena or clickhouse or openpyxl or pyarrow or project or sqlite or aws_creds 54.60% <63.63%> (+<0.01%) ⬆️
3.8 aws_deps 45.88% <63.63%> (+<0.01%) ⬆️
3.8 big 54.66% <63.63%> (+<0.01%) ⬆️
3.8 databricks 47.02% <63.63%> (+<0.01%) ⬆️
3.8 filesystem 60.45% <63.63%> (+<0.01%) ⬆️
3.8 mssql 49.98% <63.63%> (+<0.01%) ⬆️
3.8 mysql 50.04% <63.63%> (+<0.01%) ⬆️
3.8 postgresql 54.19% <63.63%> (+<0.01%) ⬆️
3.8 snowflake 47.79% <63.63%> (+<0.01%) ⬆️
3.8 spark 57.39% <63.63%> (+<0.01%) ⬆️
3.8 trino 52.11% <63.63%> (+<0.01%) ⬆️
3.9 66.09% <63.63%> (+<0.01%) ⬆️
cloud 0.00% <0.00%> (ø)
docs-basic 48.86% <63.63%> (+<0.01%) ⬆️
docs-creds-needed 50.00% <63.63%> (+<0.01%) ⬆️
docs-spark 48.72% <63.63%> (+<0.01%) ⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

Comment on lines 1 to 4
clickhouse-sqlalchemy>=0.2.2; python_version < "3.12"
clickhouse-sqlalchemy>=0.3.0; python_version >= "3.12" # SQLALchemy 2 support added in 0.3.0
pandas<2.2.0; python_version >= "3.9"
pandas>=2.0.0; python_version <= "3.12"
Copy link
Contributor Author

@Kilo59 Kilo59 May 28, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

SQLAlchemy 2.0 support was added
https://github.com/xzkostyan/clickhouse-sqlalchemy/blob/master/CHANGELOG.md#030---2023-11-06

I believe this is why we are restricting the pandas version. I think we can lift this restriction for all versions of python.

responses>=0.23.1 # requests mocking
snapshottest==0.6.0 # GX Cloud atomic renderer tests
setuptools>=70.0.0
Copy link
Contributor Author

@Kilo59 Kilo59 Jun 17, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

package_resources is part of setuptools, and not installed by default for Python 3.12.

See related issue in a different project.

@Kilo59 Kilo59 changed the title [FEATURE] Python 3.12 support [FEATURE] Experimental Python 3.12 support Jun 17, 2024
@Kilo59 Kilo59 added this pull request to the merge queue Jun 17, 2024
Merged via the queue into develop with commit faa77f7 Jun 17, 2024
72 checks passed
@Kilo59 Kilo59 deleted the f/_/python312 branch June 17, 2024 18:22
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants