Dynamic filter all values not working

[Vicinius]

Bug description

Whenever I try to use dynamic filter between all values, it simply won't work. For this use case I have 29000+ values on a filter, superset only shows up the first 1000. But when I start typing, it only searches on the first 1000, even with this option enabled which should search on the whole result set. Using trino/presto version 426 as my db

How to reproduce the bug

1. Go to add/edit filters
2. On a filter that has more than 1000 results, select the option dynamically search between all filter values
3. Type a value or part of
4. Check that it's only searching the first 1000, not on the whole result set

Screenshots/recordings

Superset version

4.0.1

Python version

3.9

Node version

16

Browser

Chrome

Additional context

I'm using microsoft edge, not chrome and presto version 426

Checklist

• I have searched Superset docs and Slack and didn't find a solution to my problem.
• I have searched the GitHub issue tracker and didn't find a similar bug report.
• I have checked Superset's logs for errors and if I found a relevant Python stacktrace, I included it here as text in the "additional context" section.

[dosubot]

Olá @Vicinius! Eu sou Dosu, um bot que pode te ajudar a resolver bugs, responder perguntas e se tornar um colaborador. Estou aqui para te ajudar enquanto você aguarda a resposta de um mantenedor humano. Como posso ajudar hoje? 😊

To resolve the issue in Superset version 4.0.1 where the dynamic filter functionality only searches within the first 1000 values, even though the option to dynamically search all filter values is enabled, you need to ensure that the "Dynamically search all filter values" option is correctly set. The provided code snippets indicate that this option should be available and can be enabled.

Here is the relevant part of the code that describes the option:

export const nativeFilterTooltips = {
  searchAllFilterOptions:
    'By default, each filter loads at most 1000 choices at the initial page load. Check this box if you have more than 1000 filter values and want to enable dynamically searching that loads filter values as users type (may add stress to your database).',
  // other options...
};

export const valueNativeFilterOptions = [
  // other options...
  'Dynamically search all filter values',
  // other options...
];

To ensure that the dynamic search is enabled, you should verify that the "Dynamically search all filter values" option is checked in your filter settings. If it is already checked and the issue persists, it might be a bug or a configuration issue that needs further investigation.

If you have access to the codebase, you can also check if the intercepts and filter settings are correctly applied:

export function setFilter(filter: string, option: string) {
  interceptFiltering();

  cy.get(`[aria-label="${filter}"]`).first().click();
  cy.get(`[aria-label="${filter}"] [title="${option}"]`).click();

  cy.wait('@filtering');
}

Ensure that the setFilter function is correctly setting the filter options, including the dynamic search option.

If the problem still persists after these checks, it might be necessary to look into the backend configurations or logs to identify any potential issues with the API calls or database queries related to the filter values.

Additionally, according to the changelog for version 4.0.1, there was a specific fix related to dynamic filters:

• #28036 fix: Dynamic filter does not show all values on blur/clear events (@michael-s-molina)

This fix addresses an issue where the dynamic filter did not show all values on blur/clear events. Ensure that your instance of Superset includes this fix, as it might resolve the issue you are experiencing ^[1][2].

Also, there are known performance issues or limitations with the Trino/Presto database connector in Superset that could affect dynamic filtering. Specifically, the Trino client blocks until the query is complete, which necessitates running the query in another thread and handling the cursor asynchronously. This blocking behavior can impact the performance and responsiveness of dynamic filtering ^[3][4].

To continue talking to Dosu, mention @dosu.

[visharavana]

4.0.0 filter is working well, but I don't know about 4.0.1.