[Fleet] Integration input groups, phase 1

[ph]

Requirements

After the input groups support is added in fleet UI, :

• users will be able to search for AWS integration, see its details in readme and Add AWS integration - This remains the same experience as today
• users will be able to search for a specific service within AWS, say dynamodb and then only dynamodb tile will show up
• clicking dynamodb tile: they see the AWS integration readme, the Add integration button says add AWS Dynamodb
• clicking on Add AWS Dynamodb shows only the dynamodb metrics configuration is enabled and the rest of the options in metrics and logs are disabled.
• When viewing the agent policy to which dynamdb integration was added shows aws integration
• When you edit the installed AWS integration, it shows dynamodb metrics configuration only
• Support package-level vars in the policy editor
• Datastreams tab shows aws.dynamodb.metrics and the "..." menu takes you to dynamodb dashboards

[elasticmachine]

Pinging @elastic/fleet (Team:Fleet)

[sorantis]

@ph not sure what this issue is for since we're tracking related changes in the linked Meta issue

[mukeshelastic]

@ph I added the requirements for fleet UI in the description.

@mostlyjason can you take a pass at reviewing the reqs?

[mostlyjason]

When you edit the installed AWS integration, it only shows dynamodb configuration only

@mukeshelastic do you mean it shows only the dynamodb metrics configuration is enabled and the rest of the options in metrics and logs are disabled?

Datastreams tab shows aws.dynamodb.metrics and the "..." menu takes you to dynamodb dashboards

I think the datastreams dashboards are defined at the package level, so they would see a list of all the AWS dashboards, of which dynamodb is one

[mukeshelastic]

When you edit the installed AWS integration, it only shows dynamodb configuration only

@mukeshelastic do you mean it shows only the dynamodb metrics configuration is enabled and the rest of the options in metrics and logs are disabled?
That's right. Whatever user configured when they installed the integration initially is enabled.

Datastreams tab shows aws.dynamodb.metrics and the "..." menu takes you to dynamodb dashboards

I think the datastreams dashboards are defined at the package level, so they would see a list of all the AWS dashboards, of which dynamodb is one

That is right, Users will need to drill down from AWS dashboards into the dynamodb specific dashboard.

[jen-huang]

FYI Kibana work on this is blocked until work is done to package registry: elastic/package-spec#144

[sorantis]

@kaiyan-sheng is currently building the AWS package based on the new structure. Once ready it should enable all related work.

[jen-huang]

I left a question regarding package-level vars as part of Kibana phase 1 work here: elastic/integrations#767 (comment)

[jen-huang]

@sorantis Can you help clarify what AWS tiles you expect to see on "Installed integrations" tab once the AWS package installed? Just the main AWS tile or main+all integrations tiles like in "All integrations" tab? I'm thinking it might be a better UX to just show the main AWS tile?

Edit: Can you also clarify what you expect to see in the "All integrations" tab if the user has not entered any search queries?

[sorantis]

@jen-huang All views should show granular integrations, i.e. the second screenshot from your questions. Since we maintain package <-> policy_template relationship, at some point we might want to reflect this nested structure in the UI, bit for now the user should be able to see granular integrations.
Could you elaborate why the user would also see the AWS package tile?

The AWS service tiles looking good by the way :)

[jen-huang]

Could you elaborate why the user would also see the AWS package tile?

@sorantis I don't have a strong opinion on only showing granular integrations vs showing both the overall package+its integrations as tiles. This part of the issue requirement confused me as I take it to mean that the AWS package tile is present in addition to the integration tiles: users will be able to search for AWS integration, see its details in readme and Add AWS integration - This remains the same experience as today. Please let me know if that is not the case 🙂

Re: showing integration tiles on Installed integrations page - the way the code is today, all the AWS integrations will show up here even if the user "only" added DynamoDB (as an example) because we install the entire AWS package at that time. Are there any UX concerns here? (That's why I thought only showing the AWS tile on the installed tab might be better, but it is nice to have the tiles be consistent.)

Edit: I left some additional thoughts related to this and the registry work here: elastic/package-registry#685 (comment)

[ruflin]

We should perhaps change the term "installed integrations" here as it is kind of confusing. Lets assume you have AWS with DynamoDB and S3. Now "DynamoDB" and "S3" are used. Assuming "S3" is not used in any policy anymore, what will show here? It is not that "S3" was uninstalled but it is not used anymore. Same if AWS with S3 is "used" multiple times, I guess it is only shown once?

[amolnater-qasource]

Hi @EricDavisX
We have created 06 new testcases for this Feature in our Ingest Manager Test Suit at link : Integration Granularity

clicking on Add AWS Dynamodb shows only the dynamodb metrics configuration is enabled and the rest of the options in metrics and logs are disabled.

As mentioned under #93315 (comment)
DynamoDB shows only dynamodb metrics configuration enabled, however rest options are not even available on Add Integration page.

Screenshot:

Please let us know if anything else is required from our end.
Thanks
QAS

[amolnater-qasource]

Hi @EricDavisX
We have executed Test plan for Integration Granularity feature at Ingest Manager 7.14.0 New Feature Integrations Granularity Test Plan

Thanks
QAS

[jen-huang]

@amolnater-qasource Thanks for the thorough testing! This requirement was changed slightly, this is what is expected:

• clicking on Add AWS Dynamodb shows only the dynamodb metrics configuration is enabled and no other configuration options
• from Integrations > AWS Dynamodb > Policies page, or Fleet > Policies > (some agent policy), clicking on Edit integration will show all configuration options