Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix: Output JSONPath expression with match count message #2118

Merged
merged 1 commit into from
Dec 19, 2023

Conversation

mjsqu
Copy link
Contributor

@mjsqu mjsqu commented Dec 19, 2023

Adding the expression to the jsonpath logging message can be helpful in determining whether the message refers to a "next page" jsonpath parsing call, or a "records" parsing call


📚 Documentation preview 📚: https://meltano-sdk--2118.org.readthedocs.build/en/2118/

@mjsqu mjsqu changed the title feat(logs): Output jsonpath expression with match count message feat: Output jsonpath expression with match count message Dec 19, 2023
@mjsqu
Copy link
Contributor Author

mjsqu commented Dec 19, 2023

Sample output with:

poetry run pytest tests/core/rest/test_pagination.py --log-cli-level=INFO
====================================================================================================== test session starts =======================================================================================================
platform linux -- Python 3.8.16, pytest-7.4.3, pluggy-1.2.0
codspeed: 2.2.0 (callgraph: not supported)
benchmark: 4.0.0 (defaults: timer=time.perf_counter disable_gc=False min_rounds=5 min_time=0.000005 max_time=1.0 calibration_precision=10 warmup=False warmup_iterations=100000)
sqlalchemy: 2.0.23
rootdir: /home/mjohnston/repos/sdk
configfile: pyproject.toml
plugins: time-machine-2.13.0, benchmark-4.0.0, codspeed-2.2.0, requests-mock-1.11.0, xdoctest-1.1.2, snapshot-0.9.0
collected 13 items                                                                                                                                                                                                               

tests/core/rest/test_pagination.py::test_paginator_base_missing_implementation PASSED                                                                                                                                      [  7%]
tests/core/rest/test_pagination.py::test_single_page_paginator PASSED                                                                                                                                                      [ 15%]
tests/core/rest/test_pagination.py::test_paginator_page_number_missing_implementation PASSED                                                                                                                               [ 23%]
tests/core/rest/test_pagination.py::test_paginator_offset_missing_implementation PASSED                                                                                                                                    [ 30%]
tests/core/rest/test_pagination.py::test_paginator_hateoas_missing_implementation PASSED                                                                                                                                   [ 38%]
tests/core/rest/test_pagination.py::test_paginator_attributes 
--------------------------------------------------------------------------------------------------------- live log call ----------------------------------------------------------------------------------------------------------
INFO     singer_sdk.helpers.jsonpath:jsonpath.py:36 JSONPath $.nextPageToken match count: 1
PASSED                                                                                                                                                                                                                     [ 46%]
tests/core/rest/test_pagination.py::test_paginator_loop 
--------------------------------------------------------------------------------------------------------- live log call ----------------------------------------------------------------------------------------------------------
INFO     singer_sdk.helpers.jsonpath:jsonpath.py:36 JSONPath $.nextPageToken match count: 1
INFO     singer_sdk.helpers.jsonpath:jsonpath.py:36 JSONPath $.nextPageToken match count: 1
PASSED                                                                                                                                                                                                                     [ 53%]
tests/core/rest/test_pagination.py::test_paginator_page_number PASSED                                                                                                                                                      [ 61%]
tests/core/rest/test_pagination.py::test_paginator_offset 
--------------------------------------------------------------------------------------------------------- live log call ----------------------------------------------------------------------------------------------------------
INFO     singer_sdk.helpers.jsonpath:jsonpath.py:36 JSONPath $[*] match count: 2
INFO     singer_sdk.helpers.jsonpath:jsonpath.py:36 JSONPath $[*] match count: 1
INFO     singer_sdk.helpers.jsonpath:jsonpath.py:36 JSONPath $[*] match count: 0

Open to suggestions regarding the wording of the output message

Copy link

codspeed-hq bot commented Dec 19, 2023

CodSpeed Performance Report

Merging #2118 will not alter performance

Comparing mjsqu:feat/improved_jsonpath_logs (e34c62d) with main (40b6333)

Summary

✅ 6 untouched benchmarks

Copy link

codecov bot commented Dec 19, 2023

Codecov Report

All modified and coverable lines are covered by tests ✅

Comparison is base (40b6333) 87.76% compared to head (e34c62d) 87.76%.

Additional details and impacted files
@@           Coverage Diff           @@
##             main    #2118   +/-   ##
=======================================
  Coverage   87.76%   87.76%           
=======================================
  Files          60       60           
  Lines        4946     4946           
  Branches     1007     1007           
=======================================
  Hits         4341     4341           
  Misses        424      424           
  Partials      181      181           

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

Copy link
Collaborator

@edgarrmondragon edgarrmondragon left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nice, thanks @mjsqu!

@edgarrmondragon edgarrmondragon changed the title feat: Output jsonpath expression with match count message fix: Output JSONPath expression with match count message Dec 19, 2023
@edgarrmondragon edgarrmondragon added this pull request to the merge queue Dec 19, 2023
@edgarrmondragon edgarrmondragon removed this pull request from the merge queue due to a manual request Dec 19, 2023
@edgarrmondragon edgarrmondragon added this pull request to the merge queue Dec 19, 2023
Merged via the queue into meltano:main with commit 49c2a9c Dec 19, 2023
31 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants