Kibana loses filters in TSVB after upgrade from to 7.3

[AlonaNadler]

Kibana version: 7.3

when upgrading to 7.3 from 7.2, in Kibana you lose filters that were in Lucene syntax on TSVB visuals, it might be related to KQL support introduced in TSVB filters in 7.3

This issue breaks visualizations in the sense users don't understand why results are off and the cause is not easy to find.

@elastic/kibana-app

[TinaHeiligers]

Reported behavior:

1. All the filters created in TSVB visualizations in 7.2 were Lucene syntax.
2. Filter loss is not consistent. While the majority are lost, some are retained.
3. When using group by filters in the Lucene syntax some of them were changed to KQL but some aren't during the upgrade. All the visuals were created the same, but some were created in different versions of Kibana.
   @troy012 Does the summary above look accurate to you? If you have anymore info on how to reproduce the bug, including the older version numbers where the TSVB visualizations were created, please add it here.
   @Bargs, @markov00 There were a few migrations added for TSVB in 7.3 that I am aware of. Are there any previous migrations that might be involved here?

[TinaHeiligers]

I am trying to reproduce the issue by creating TSVB visualizations in Kibana 6.8, saving the visualizations as saved objects and importing them into Kibana 7.3.

[alexfrancoeur]

Hi all, I ran into this behavior on our internal repo that's tracking enhancement requests. I feel like we jumped from 6.x to 7.3 but I'm not 100% sure and the person responsible for the upgrade is currently on paternity leave.

This seems like a pretty big issue, potentially affecting a large number of users with time series use cases. We have plans to upgrade other internal clusters as well that heavily utilize the filters in TSVB across numerous visualizations. When we have a fix for this, is this something we can backport? This seems like a pretty serious regression to me, especially if you don't know what those filters were previously.

Happy to share the instance internally if needed (though I've fixed the visualization).

[TinaHeiligers]

@alexfrancoeur It will help a lot if you can share the instance and the saved object exported from the 6.x version! If you don't have the saved visualizations from 6.x, hopefully you'll be able to let me know where you had to fix the visualization.

[TinaHeiligers]

I was able to reproduce the issue by creating a TSVB visualization in Kibana 6.8 the split filters in the series items. After importing the saved object into 7.3, the split filters items were not migrated.

On inspecting the saved object's visState using the Saved Objects list inspection tool, before migration the series items are:

After importing the saved object into Kibana 7.3, the series items are:

What we expect to see is:

The source of the error appears to be that the split_filters items within the visState are not being migrated.

A fix is in progress.

[TinaHeiligers]

A PR has been submitted to resolve the situation in Kibana version 7.5: #49000

[TinaHeiligers]

When migrating to Kibana 7.3, split_filters from the 'Group by' series option were not migrated to query:language objects. After an upgrade to 7.3, visualizations that had split_filters in previous versions don't interpret the filter as a string.
For example, if we have a TSVB visualization in Kibana 6.8 with the following configuration:

After an upgrade to 7.3, the filters appear to have been lost:

However, the filter strings haven't been deleted, they are not interpretable by the code because they are not {query:<string>, language: 'lucene' objects.

Inspecting the visState data using the utilities in the Management -> Saved Objects list, we see that the filter strings are still there:

Patches are not being backported to 7.3 anymore, so a fix to the original migration can't be applied.
A new migration has been added and runs for version 7.5, the oldest version we can backport to.
The migration transforms any remaining split_filters filters that are still strings. The migration will effectively restore the original filters if they have not been deleted.
migrated_filters_on_import_to_v8

Versions between 7.3 and 7.5: Manual work around

Unfortunately, the only way to 'fix' the missing filters in a version between 7.3 and 7.5 is to manually convert the filter string to an object in the visState panel shown when inspecting the saved object.
As an example, say we have the following filter as an item in the Group by series panel:
Group by: Filters
The filter is response.keyword:200
To migrate the filter and have it reapplied:

1. Go to the Management page and select Saved Objects.
2. Find the visualization with 'missing' filters
3. Click on the 'inspect' icon. This will show a series of panels with json in them.
4. Find the panel that shows the visState
5. Look for the split_filters array and convert the filter:'response.keyword:200' into the following object:
   filter: { "query": "response.keyword:200", "language": "lucene" }
6. Save the object.

You should see that the visualization's filters have been restored.

The other way to restore the filters after an upgrade to 7.3 is to inspect the saved object and manually reapply the filter as seen in the json under visState -> params -> series -> split_filters -> filter in the visualization itself.

The manual work-around has to be applied to every visualization.

[TinaHeiligers]

@troy012 @alexfrancoeur @AlonaNadler Fortunately, the filters aren't deleted! Unfortunately, we can't backport further back than 7.5.

I've submitted a PR for a backport fix to 7.5 and added manual work-arounds in the comment above. For versions between 7.3 and 7.5, we have to apply a manual work-around (described above). The manual work-around has to be applied to every visualization that uses Group by: Filters items.

Please let me know if you have any questions.

[rayafratkina]

@TinaHeiligers let's backport the fix to 7.4 - even though there is no patch planned at the moment, we may decide to do one. Let's make sure the fix is in the 7.4 branch proactively.

[VijayDoshi]

Can we also port this to 7.3? This is a "correctness" issue - showing data that is not correct really worries me. I am aware of our pattern of only back-porting security issues; however, I categorize correctness as being nearly as severe as security since it potentially impacts decisions our customers make based on data they are looking at.

[TinaHeiligers]

@VijayDoshi The request to be able to backport to 7.3.x has been made.