Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Congruence values count #268

Merged
merged 4 commits into from
Sep 4, 2023
Merged

Congruence values count #268

merged 4 commits into from
Sep 4, 2023

Conversation

joein
Copy link
Member

@joein joein commented Aug 25, 2023

Methods value_counts, isempty, isnull didn't work properly in local mode due to an approach when we flatten all input values into one array

For instance if we have the following payload: {"nested": [{"numeric": 1, "numeric": 2, "numeric": 3}]} and want to filter by value count of numeric, then local mode piles up everything into one array and counts them together (values_count=3), but it should count them separately (values_count=1)

Another problem with payload like
{"nested": [{"empty": [None]}, {"empty": [None]}, {"empty": [None]}
or
{"nones": [None, None]}

Local mode considers such fields equal to None, however [None, None] != None.

This PR adjusts interface of payload_value_extractor and allows to disable flattening during an extraction of values.

All Submissions:

  • Contributions should target the dev branch. Did you create your branch from dev?
  • Have you followed the guidelines in our Contributing document?
  • Have you checked to ensure there aren't other open Pull Requests for the same update/change?

New Feature Submissions:

  1. Does your submission pass tests?
  2. Have you installed pre-commit with pip3 install pre-commit and set up hooks with pre-commit install?

Changes to Core Features:

  • Have you added an explanation of what your changes do and why you'd like us to include them?
  • Have you written new tests for your core changes, as applicable?
  • Have you successfully ran tests with your changes locally?

@joein joein added the bug Something isn't working label Aug 25, 2023
@joein joein self-assigned this Aug 25, 2023
@joein joein linked an issue Aug 25, 2023 that may be closed by this pull request
@joein joein requested review from timvisee and generall August 25, 2023 15:17
@generall generall merged commit 2b6a6b7 into dev Sep 4, 2023
generall pushed a commit that referenced this pull request Sep 7, 2023
* tests: add some values count congruence tests

* fix: fix incorrect behaviour of value_counts, isempty, isnull in local mode

* fix: update isempty behaviour

* tests: add test cases for isnull and isempty
@generall generall deleted the congruence-values-count branch May 3, 2024 10:48
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working
Projects
None yet
Development

Successfully merging this pull request may close these issues.

local client values count differs from remote client
3 participants