Move search.includeFrozen to be on a per-index pattern basis.

[maihde]

Describe the feature:

Frozen indices are a desirable feature for maintaining time-series data for a longer duration, but the current Kibana feature (implemented in #27297) is all or nothing.

Although the comment in #26244 states "frozen indices also have a protection that allows fast evaluation of date-ranges which makes it suitable for kibana since even if a pattern expands to a frozen index the query will be likely very fast", my experiments have shown that even if the date range doesn't match the frozen index dates the query itself the query gets progressively slower as more frozen indices match the index pattern (probably related to #32742 and other associated issues).

Even if fixes like #42510 and #39835 might resolve the performance issue for narrow cases, I believe there will be remain situations where a user wants to have frozen indices included in some cases but not others...thus, a global Kibana option is not ideal. What I'd like to propose is that includeFrozen be moved to a per index-pattern basis.

Describe a specific use case for the feature:

A cluster has two different types of data (index-dataA and index-dataB). Both use frozen indices to manage old data. Users of Kibana want to query frozen indexes from index-dataA, so the search.includeFrozen is turned on. However, this impacts users of Kibana who are querying index-dataB who don't want frozen indices included.

[elasticmachine]

Pinging @elastic/kibana-app (Team:KibanaApp)

[elasticmachine]

Pinging @elastic/kibana-app-arch (Team:AppArch)

[ppisljar]

Thank you for contributing to this issue, however, we are closing this issue due to inactivity as part of a backlog grooming effort. If you believe this feature/bug should still be considered, please reopen with a comment.