Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[GenAI][Integrations] UI for the custom integration creation with AI #186304

Merged
merged 22 commits into from
Jun 21, 2024
Merged

[GenAI][Integrations] UI for the custom integration creation with AI #186304

merged 22 commits into from
Jun 21, 2024

Conversation

semd
Copy link
Contributor

@semd semd commented Jun 17, 2024 •
edited
Loading

Summary

This ticket is the initial implementation for the UI side for the AI-driven custom integration creation.

This PR only contains the implementation of the UI, due to the tight timing it will not include tests, everything will be tested manually for 8.15 FF. We'll implement the tests later.

Enable Feature

The new integration assistant plugin is disabled by default, to enable it:

xpack.integration_assistant.enabled: true

Complete tasks

  • New integration button on the /integrations page
  • New integration "landing" page with buttons to upload zip and assistant
  • Upload zip page to install integration
  • Integration assistant:
    • Connector selection step
    • Integration details step
    • Data stream step
    • Review and install

Follow-ups (will be implemented in separate PRs)

Demo

-4022755641775836210ai_ingestion.MP4

Files by Code Owner

elastic/docs

  • packages/kbn-doc-links/src/get_doc_links.ts
  • packages/kbn-doc-links/src/types.ts

elastic/fleet

  • x-pack/plugins/fleet/kibana.jsonc
  • x-pack/plugins/fleet/public/applications/integrations/hooks/use_breadcrumbs.tsx
  • x-pack/plugins/fleet/public/applications/integrations/layouts/default.tsx
  • x-pack/plugins/fleet/public/applications/integrations/sections/epm/index.tsx
  • x-pack/plugins/fleet/public/applications/integrations/sections/epm/screens/create/index.tsx
  • x-pack/plugins/fleet/public/components/header.tsx
  • x-pack/plugins/fleet/public/constants/page_paths.ts
  • x-pack/plugins/fleet/public/plugin.ts
  • x-pack/plugins/fleet/tsconfig.json

elastic/kibana-core

  • x-pack/plugins/fleet/kibana.jsonc
  • x-pack/plugins/integration_assistant/kibana.jsonc

elastic/kibana-operations

  • packages/kbn-optimizer/limits.yml

elastic/security-solution

  • x-pack/plugins/integration_assistant/**/*

@semd semd added release_note:feature Makes this part of the condensed release notes Team:Security Generative AI Security Generative AI v8.15.0 labels Jun 17, 2024
@semd semd self-assigned this Jun 17, 2024
@semd
Copy link
Contributor Author

semd commented Jun 17, 2024

/ci

bhapas
bhapas reviewed Jun 18, 2024
View reviewed changes
Copy link
Contributor

@bhapas bhapas left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Minor nits and questions

Should we add our plugin in https://github.com/elastic/kibana/blob/main/.eslintrc.js#L1057

@semd
Copy link
Contributor Author

semd commented Jun 18, 2024

/ci

@semd semd marked this pull request as ready for review June 18, 2024 15:31
@semd semd requested review from a team as code owners June 18, 2024 15:31
@semd
Copy link
Contributor Author

semd commented Jun 18, 2024

@elasticmachine merge upstream

@botelastic botelastic bot added the Team:Fleet Team label for Observability Data Collection Fleet team label Jun 18, 2024
@elasticmachine
Copy link
Contributor

Pinging @elastic/fleet (Team:Fleet)

@semd semd changed the title [Integrations] UI for the custom integration creation with AI [GenAI][Integrations] UI for the custom integration creation with AI Jun 18, 2024
jbudz
jbudz approved these changes Jun 18, 2024
View reviewed changes
Copy link
Member

@jbudz jbudz left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

limits.yml

@stephmilovic
Copy link
Contributor

@elasticmachine merge upstream

stephmilovic
stephmilovic approved these changes Jun 21, 2024
View reviewed changes
Copy link
Contributor

@stephmilovic stephmilovic left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM! Nice work Sergi! I discussed over Zoom an issue with the Error state. It seems to be loading still even though an error occurred:
Screenshot 2024-06-21 at 9 20 30 AM
Sergi plans to follow up to update this design:
Captura de pantalla 2024-06-21 a les 17 41 53

That was the only issue I noticed in manual testing. Code looks great and when using the new Sonnet 3.5 model, the integration was installed very quickly. Amazing work!

@kibana-ci
Copy link
Collaborator

💛 Build succeeded, but was flaky

Failed CI Steps

Metrics [docs]

Module Count

Fewer modules leads to a faster build time

id before after diff
fleet 1034 1035 +1
integrationAssistant - 505 +505
total +506

Public APIs missing comments

Total count of every public API that lacks a comment. Target amount is 0. Run node scripts/build_api_docs --plugin [yourplugin] --stats comments for more detailed information.

id before after diff
@kbn/doc-links 77 78 +1
fleet 1217 1219 +2
integrationAssistant 33 38 +5
total +8

Async chunks

Total size of all lazy-loaded chunks that will be downloaded as the user navigates the app

id before after diff
fleet 1.4MB 1.4MB +711.0B
integrationAssistant - 841.0KB ⚠️ +841.0KB
lists 142.5KB 142.7KB +112.0B
total +841.8KB

Public APIs missing exports

Total count of every type that is part of your API that should be exported but is not. This will cause broken links in the API documentation system. Target amount is 0. Run node scripts/build_api_docs --plugin [yourplugin] --stats exports for more detailed information.

id before after diff
integrationAssistant 1 2 +1

Page load bundle

Size of the bundles that are downloaded on every page load. Target size is below 100kb

id before after diff
core 415.1KB 415.3KB +112.0B
fleet 165.6KB 165.6KB +67.0B
integrationAssistant - 4.4KB +4.4KB
total +4.6KB
Unknown metric groups

API count

id before after diff
@kbn/doc-links 77 78 +1
fleet 1339 1341 +2
integrationAssistant 38 44 +6
total +9

async chunk count

id before after diff
integrationAssistant - 2 +2

ESLint disabled line counts

id before after diff
integrationAssistant 2 4 +2

miscellaneous assets size

id before after diff
integrationAssistant - 250.8KB ⚠️ +250.8KB

Total ESLint disabled count

id before after diff
integrationAssistant 5 7 +2

History

To update your PR or re-run it, just comment with:
@elasticmachine merge upstream

cc @semd

benironside
benironside approved these changes Jun 21, 2024
View reviewed changes
Copy link
Contributor

@benironside benironside left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewed the demo video. LGTM!

@semd semd merged commit 9b3775e into elastic:main Jun 21, 2024
24 checks passed
@kibanamachine kibanamachine added the backport:skip This commit does not require backporting label Jun 21, 2024
bhapas pushed a commit to bhapas/kibana that referenced this pull request Jun 24, 2024
@semd @kibanamachine @bhapas
[GenAI][Integrations] UI for the custom integration creation with AI (e…
e265ead 
…lastic#186304)

## Summary 

This ticket is the initial implementation for the UI side for the
AI-driven custom integration creation.

This PR only contains the implementation of the UI, due to the tight
timing it will not include tests, everything will be tested manually for
8.15 FF. We'll implement the tests later.

#### Enable Feature

The new integration assistant plugin is disabled by default, to enable
it:

```
xpack.integration_assistant.enabled: true
```


#### Complete tasks

- [x] New integration button on the /integrations page
- [x] New integration "landing" page with buttons to upload zip and
assistant
- [x] Upload zip page to install integration
- [x] Integration assistant:
  - [x] Connector selection step
  - [x] Integration details step
  - [x] Data stream step
  - [x] Review and install

#### Follow-ups (will be implemented in separate PRs)

- [ ] Add RBAC
- [ ] Add telemetry
- [ ] Documentation
- [ ] Add license/productType controls
- [ ] Add links to the create integration page
- [ ] Improve package name retrieval:
elastic#185932
- [ ] Add time estimation on the generation stage
- [ ] Add support for multi-valuated "input type"
- [ ] Enable Langsmith tracing using AI assistant settings

#### Demo


https://github.com/elastic/kibana/assets/17747913/b04c21c6-09cf-49bb-be8f-bf4b9d3feb8e


## Files by Code Owner

### elastic/docs

* packages/kbn-doc-links/src/get_doc_links.ts
* packages/kbn-doc-links/src/types.ts

### elastic/fleet

* x-pack/plugins/fleet/kibana.jsonc
*
x-pack/plugins/fleet/public/applications/integrations/hooks/use_breadcrumbs.tsx
*
x-pack/plugins/fleet/public/applications/integrations/layouts/default.tsx
*
x-pack/plugins/fleet/public/applications/integrations/sections/epm/index.tsx
*
x-pack/plugins/fleet/public/applications/integrations/sections/epm/screens/create/index.tsx
* x-pack/plugins/fleet/public/components/header.tsx
* x-pack/plugins/fleet/public/constants/page_paths.ts
* x-pack/plugins/fleet/public/plugin.ts
* x-pack/plugins/fleet/tsconfig.json

### elastic/kibana-core

* x-pack/plugins/fleet/kibana.jsonc
* x-pack/plugins/integration_assistant/kibana.jsonc

### elastic/kibana-operations

* packages/kbn-optimizer/limits.yml

### elastic/security-solution

* x-pack/plugins/integration_assistant/**/*

---------

Co-authored-by: Kibana Machine <[email protected]>
@semd semd mentioned this pull request Jun 29, 2024
Merged
semd added a commit that referenced this pull request Jul 1, 2024
@semd @kibanamachine
[GenAi] Improve Integration assistant generation errors (#187204)
71a47c0 
## Summary

Follow-up of
#186304 (review)

Improved the error messages and `Retry` button added.

### Screenshots

Before


![before](https://github.com/elastic/kibana/assets/17747913/b412449f-1f3b-4d98-b069-916f881c1cad)

After


![after](https://github.com/elastic/kibana/assets/17747913/07847ad5-8d15-485f-82d0-3c9ac9ac7671)

---------

Co-authored-by: kibanamachine <[email protected]>
This was referenced Jul 15, 2024
Merged
Merged
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@pgayvallet pgayvallet pgayvallet approved these changes

@jbudz jbudz jbudz approved these changes

@stephmilovic stephmilovic stephmilovic approved these changes

@criamico criamico criamico approved these changes

@benironside benironside benironside approved these changes

@bhapas bhapas bhapas approved these changes

Assignees

@semd semd

Labels
backport:skip This commit does not require backporting release_note:feature Makes this part of the condensed release notes Team:Fleet Team label for Observability Data Collection Fleet team Team:Security Generative AI Security Generative AI v8.15.0
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
10 participants
@semd @elasticmachine @stephmilovic @kibana-ci @pgayvallet @jbudz @criamico @benironside @bhapas @kibanamachine