[ResponseOps][Stack Connectors] Opsgenie UI phase 2 #143480

jonathan-buttner · 2022-10-17T22:03:54Z

This PR implements most of the Phase 2 section of the above issue. We decided not to implement the ability to dynamically adding the fields. Instead we are going to provide a JSON editor that users can easily define those complex fields (e.g. responders, visibleTo).

Notable details:

If you switch between the json editor and form view the fields will be preserved (as long as the JSON editor is valid)
Validation is done using io-ts on the frontend and kbn/config-schema on the backend
- After discussing using kbn/config-schema on the frontend I was told that Joi is not supported by all browsers and would increase the page load size by a lot
- The frontend validation is mainly for the json editor, the reason we need that is if a user sets a field to an invalid value it could cause the form fields to crash when switching views
The Opsgenie icon is now added
The action selector for creating or closing an alert is only shown when you're in the testing tab, it is defaulted when you're in the rule action form

Testing

Refer to this PR's testing section for details on how to create an Opsgenie environment: #142164

Retrieving an alert after it is created/closed

If you'd like to retrieve an alert via Opsgenie's API you can use curl or Postman and use the following requests

GET https://api.opsgenie.com/v2/alerts/<identifier>?identifierType=<tiny|alias|id>

Using the Tiny ID

Grab the tiny value (shown with the # sign under the alert message header in Opsgenie, i.e. 22 in the image below)

GET https://api.opsgenie.com/v2/alerts/22?identifierType=tiny

Alias

Use the alias specified in the request. The docs state that you can't retrieve a closed alert using the alias though.

ID

This is used by default, it is the id field in the alert body.

Notes

I'm not able to get the responders and visibleTo fields to cause any noticeable changes within the alert when it is created. I believe this is because the plan you are give when trying Opsgenie for free is the Essential plan.

https://support.atlassian.com/opsgenie/docs/create-a-default-api-integration/

The Essential plan only allows you to add integrations to individual teams. The Opsgenie docs say if the API key used to create an alert is tied to a team, then responders will be overridden to the team and you can't change it. I assume this is also affecting the visibleTo field.

If the API Key belongs to a team integration, this field will be overwritten with the owner team.

https://docs.opsgenie.com/docs/alert-api#create-alert

Valid JSON examples

JSON examples

{
    "message": "an alert message"
}

{
    "message": "an alert message",
    "alias": "unique_alias"
}

{
    "message": "an alert message",
    "alias": "unique_alias",
    "visibleTo": [
        {
            "id":"4513b7ea-3b91-438f-b7e4-e3e54af9147c",
            "type":"team"
        },
        {
            "name":"rocket_team",
            "type":"team"
        }
    ]
}

More examples can be found here: https://docs.opsgenie.com/docs/alert-api#create-alert

Examples

Opsgenie Icon

Create alert rule action form

JSON editor

Close alert rule action form

…e-ui

Co-authored-by: nastasha-solomon <[email protected]>

…ana into opsgenie-ui-phase-2

...ck/plugins/stack_connectors/public/connector_types/stack/opsgenie/create_alert/schem.test.ts

ymao1 · 2022-10-31T14:51:15Z

...ck/plugins/stack_connectors/public/connector_types/stack/opsgenie/create_alert/schem.test.ts

+    expect(() =>
+      decodeCreateAlert({
+        message: 'hi',
+        responders: [{ name: 'sam', type: 'team', invalidField: 'scott' }],


Wondering if it would be useful to show these additional fields somehow if they are populated in the JSON in the non-JSON editor view

Yeah ideally we'd have that. We didn't think we'd be able to create those components in time so we opted for the json editor create the complex arrays. We could include them in the future though.

x-pack/plugins/stack_connectors/public/connector_types/stack/opsgenie/params.tsx

ymao1 · 2022-10-31T14:58:29Z

x-pack/plugins/stack_connectors/public/connector_types/stack/opsgenie/params.tsx

-  TextFieldWithMessageVariables,
-} from '@kbn/triggers-actions-ui-plugin/public';
-import { EuiFormRow, EuiSelect, RecursivePartial } from '@elastic/eui';
+import { ActionParamsProps, ActionConnectorMode } from '@kbn/triggers-actions-ui-plugin/public';


When creating a rule, if I select the OpsGenie action and just click Save without entering any values, it does not save but also does not show any validation errors on the form.

Thanks, working on a fix.

…se-2

docs/management/connectors/action-types/opsgenie.asciidoc

lcawl · 2022-10-31T20:12:54Z

docs/management/connectors/action-types/opsgenie.asciidoc

+Opsgenie connectors have the following configuration properties.
+
+Name::      The name of the connector. The name is used to identify a  connector in the management UI connector listing, or in the connector list when configuring an action.
+URL::     The Opsgenie URL: https://api.opsgenie.com. For the EU instance of Opsgenie use: https://api.eu.opsgenie.com. If you are using the <<action-settings, `xpack.actions.allowedHosts`>> setting, make sure the hostname is added to the allowed hosts.


IMO since this URL is not something we control, we could just provide these as example URLs. Something like this:

Suggested change

URL:: The Opsgenie URL: https://api.opsgenie.com. For the EU instance of Opsgenie use: https://api.eu.opsgenie.com. If you are using the <<action-settings, `xpack.actions.allowedHosts`>> setting, make sure the hostname is added to the allowed hosts.

URL:: The Opsgenie URL. For example, https://api.opsgenie.com or https://api.eu.opsgenie.com. If you are using the <<action-settings,`xpack.actions.allowedHosts`>> setting, make sure the hostname is added to the allowed hosts.

docs/management/connectors/action-types/opsgenie.asciidoc

.../plugins/stack_connectors/public/connector_types/stack/opsgenie/create_alert/translations.ts

lcawl · 2022-10-31T20:54:29Z

docs/management/connectors/action-types/opsgenie.asciidoc

+[[opsgenie-action-create-alert-configuration]]
+===== Configure the create alert action
+
+You can configure the create alert action through the form view or using a JSON editor.


Though I know this page is following the example of the example connector pages, IMO rather than listing each field and meaning here, it's more important to (1) have the appropriate API documentation up-to-date with these properties and descriptions, and (2) have tooltips in the UI for any fields that aren't obvious (e.g. "alias", "entity", "source"). If you want me to add those tooltips in this PR, @jonathan-buttner let me know. Otherwise, I'll create a follow-up PR.

Great point @lcawl . Let's do a follow up PR to address that. I'll add these fields in the docs here and we can remove them once we have the tooltips.

…se-2

peteharverson

Tested generating some alerts in Opsgenie using a variety of the 'create alert' API parameters, in test mode and by using the connector in a rule, and overall LGTM.

ymao1

LGTM!

cnasikas · 2022-11-03T11:42:40Z

x-pack/plugins/stack_connectors/public/connector_types/stack/opsgenie/close_alert.test.tsx

+import { CloseAlert } from './close_alert';
+import userEvent from '@testing-library/user-event';
+
+describe('CreateAlert', () => {


nit: close alert

cnasikas · 2022-11-03T12:13:21Z

x-pack/plugins/stack_connectors/public/connector_types/stack/opsgenie/create_alert/schema.ts

+import { isEmpty, isObject } from 'lodash';
+import * as i18n from './translations';
+
+const MessageNonEmptyString = new rt.Type<string, string, unknown>(


seems like magic to me 😂

Haha I actually grabbed it from here: https://github.com/elastic/kibana/blob/main/packages/kbn-securitysolution-io-ts-types/src/non_empty_string/index.ts

and modified it a bit :)

cnasikas · 2022-11-03T15:47:11Z

x-pack/plugins/stack_connectors/server/connector_types/stack/opsgenie/connector.ts

+    if (error.response?.data?.errors != null) {
+      const message = this.getDetailedErrorMessage(error.response?.data?.errors);
+      if (!isEmpty(message)) {
+        return `${mainMessage}: ${message}`;


nit: if message is undefined will the message be what ever in the main message: undefined?

lodash isEmpty will actually return true if message is undefined so the if block won't be executed.

Silly me, hahaha

cnasikas · 2022-11-03T16:03:40Z

x-pack/plugins/stack_connectors/server/connector_types/stack/opsgenie/schema.ts

+     *   "username": "must be a well-formed email address"
+     * }
+     */
+    errors: schema.maybe(schema.any()),


Do you think changing it to Record<string, string> will be risky? If the error schema is different it will throw an error.

Yeah that was my concern, I figured the safest route is just to JSON.stringify the result and not actually make assumptions on how it is formatted. When I looked through the Opsgenie API docs I didn't see any references to the format of the errors object. So my understanding of the format is just based on my experience with trying to cause errors.

Ok, that make sense!

cnasikas · 2022-11-03T16:21:43Z

...rs_actions_ui/public/application/sections/action_connector_form/test_connector_form.test.tsx

+      <I18nProvider>
+        <TestConnectorForm
+          connector={connector}
+          executeEnabled={true}


Should we test the case where executeEnabled=false or undefined?

I think I might be missing what you're getting at here. The executeEnabled doesn't have any affect on the field I added (executionMode) right?

Yes, sorry wrong line of code. I mean the executionMode.

Got it, yep I'll add some tests for those scenarios 👍

Actually I don't think there's much else I can test here since the TestConnectorForm is passing ActionConnectorMode.Test always right? Or am I missing something? I'll add a check to ensure the other text field shouldn't be displayed when executionMode === Test.

You are right! Ensuring that the other text field shouldn't be displayed is enough.

cnasikas · 2022-11-03T16:25:44Z

x-pack/plugins/triggers_actions_ui/public/types.ts

@@ -173,6 +177,7 @@ export interface ActionParamsProps<TParams> {
  isLoading?: boolean;
  isDisabled?: boolean;
  showEmailSubjectAndMessage?: boolean;
+  executionMode?: ActionConnectorMode;


I think it is better if the executionMode is required and the framework will pass either ActionConnectorMode.Test or ActionConnectorMode.NORMAL (or similar) to avoid confusion in the future in case we need to add more execution modes. Another option is to change the executionMode to testingMode?: boolean.

Good point, I'll switch it to being required and create a new enum for the ActionForm code path 👍

There's one place where the email connector is instantiated outside of the action framework: x-pack/plugins/synthetics/public/legacy_uptime/components/settings/default_email.tsx

After chatting with @ymao1 it seemed easiest to keep executionMode as optional that way that instance doesn't need to pass a value for no reason. I still changed the location in the action form to pass in the ActionForm enum.

cnasikas · 2022-11-03T16:35:38Z

x-pack/test/functional/services/actions/common.ts

+
+      if (isEnabled && isDisplayed) {
+        await flyOutCancelButton.click();
+      }


Should we wait for the flyout to disappear and throw if not?

cnasikas

Amazing job Jonathan! Code LGTM and tested without any issues 🚀. I noticed that you cannot update (add more keys) the details attribute. This is how OpsGenie behaves. I tested their API. Btw, I created a flaky test runner here: https://buildkite.com/elastic/kibana-flaky-test-suite-runner/builds/1512

cnasikas · 2022-11-04T08:07:14Z

x-pack/test/functional_with_es_ssl/apps/triggers_actions_ui/connectors/opsgenie.ts

+
+          await testSubjects.missingOrFail('messageInput');
+          await retry.waitFor('message input to be displayed', async () => {
+            await testSubjects.selectValue('opsgenie-subActionSelect', 'createAlert');


nit: Do we need to select the value on each retry?

Yeah I was seeing flakiness with the form changing so I think reselecting the value seemed to help.

kibana-ci · 2022-11-07T14:48:47Z

💛 Build succeeded, but was flaky

Failed CI Steps

Security Solution Tests #1

Test Failures

[job] [logs] Security Solution Tests #1 / Alerts timeline Privileges: read only "before each" hook for "should not allow user with read only privileges to attach alerts to existing cases"

Metrics [docs]

Module Count

Fewer modules leads to a faster build time

id	before	after	diff
`stackConnectors`	141	167	+26

Public APIs missing comments

Total count of every public API that lacks a comment. Target amount is 0. Run node scripts/build_api_docs --plugin [yourplugin] --stats comments for more detailed information.

id	before	after	diff
`triggersActionsUi`	491	495	+4

Async chunks

Total size of all lazy-loaded chunks that will be downloaded as the user navigates the app

id	before	after	diff
`stackConnectors`	311.9KB	378.5KB	+66.6KB
`triggersActionsUi`	660.0KB	658.6KB	-1.3KB
total			+65.2KB

Page load bundle

Size of the bundles that are downloaded on every page load. Target size is below 100kb

id	before	after	diff
`stackConnectors`	22.5KB	22.6KB	+152.0B
`triggersActionsUi`	101.2KB	101.7KB	+495.0B
total			+647.0B

Unknown metric groups

API count

id	before	after	diff
`triggersActionsUi`	520	524	+4

async chunk count

id	before	after	diff
`stackConnectors`	54	55	+1

ESLint disabled in files

id	before	after	diff
`osquery`	1	2	+1

ESLint disabled line counts

id	before	after	diff
`enterpriseSearch`	19	21	+2
`fleet`	58	64	+6
`osquery`	108	113	+5
`securitySolution`	440	446	+6
`stackConnectors`	72	74	+2
total			+21

Total ESLint disabled count

id	before	after	diff
`enterpriseSearch`	20	22	+2
`fleet`	66	72	+6
`osquery`	109	115	+6
`securitySolution`	517	523	+6
`stackConnectors`	76	78	+2
total			+22

History

💚 Build #85290 succeeded 2b1e572
💔 Build #85258 failed 272d456
💛 Build #84496 was flaky ebe4839
💚 Build #84477 succeeded f29e51e
💚 Build #83013 succeeded a71d514

To update your PR or re-run it, just comment with:
@elasticmachine merge upstream

jonathan-buttner added 25 commits September 28, 2022 17:38

Starting opsgenie backend

33bcdcc

Merge branch 'main' of github.com:elastic/kibana into opsgenie-connector

7413f2c

Adding more integration tests

dcb1fd9

Updating readme

b6950fc

Starting ui

d019785

Merge branch 'main' of github.com:elastic/kibana into opsgenie-connector

0ba5a6c

Adding hash and alias

0111502

Fixing tests

af4a94a

Switch to platinum for now

3eb263a

Adding server side translations

854492b

Merge branch 'main' of github.com:elastic/kibana into opsgenie-connector

a08975b

Fixing merge issues

9892a2d

Merge branch 'main' of github.com:elastic/kibana into opsgenie-connector

40bbcc4

Fixing file location error

c79afdc

Merge remote-tracking branch 'origin/opsgenie-connector' into opsgeni…

8495a20

…e-ui

Working ui

1822228

Default alias is working

e62858c

Almost working validation fails sometimes

34f37d9

Adding end to end tests

f029784

Merge branch 'main' of github.com:elastic/kibana into opsgenie-ui

7840529

Adding more tests

7c15ded

Adding note and description fields

5bac5de

Merge branch 'main' of github.com:elastic/kibana into opsgenie-ui

686cf97

Removing todo

69529d6

Adding in advanced sections

b01e931

jonathan-buttner added release_note:skip Skip the PR/issue when compiling release notes backport:skip This commit does not require backporting Feature:Actions Team:ResponseOps Label for the ResponseOps team (formerly the Cases and Alerting teams) v8.6.0 labels Oct 17, 2022

Update docs/management/connectors/action-types/opsgenie.asciidoc

34407d8

Co-authored-by: nastasha-solomon <[email protected]>

jonathan-buttner requested a review from a team as a code owner October 31, 2022 14:55

jonathan-buttner and others added 6 commits October 31, 2022 10:55

Update docs/management/connectors/action-types/opsgenie.asciidoc

3646cac

Co-authored-by: nastasha-solomon <[email protected]>

Update docs/management/connectors/action-types/opsgenie.asciidoc

78873c7

Co-authored-by: nastasha-solomon <[email protected]>

Update docs/management/connectors/action-types/opsgenie.asciidoc

771bf20

Co-authored-by: nastasha-solomon <[email protected]>

Update docs/management/connectors/action-types/opsgenie.asciidoc

abeaef6

Co-authored-by: nastasha-solomon <[email protected]>

Update docs/management/connectors/action-types/opsgenie.asciidoc

e671759

Co-authored-by: nastasha-solomon <[email protected]>

Merge branch 'opsgenie-ui-phase-2' of github.com:jonathan-buttner/kib…

97a6feb

…ana into opsgenie-ui-phase-2

ymao1 reviewed Oct 31, 2022

View reviewed changes

Merge branch 'main' of github.com:elastic/kibana into opsgenie-ui-pha…

df338d3

…se-2

lcawl reviewed Oct 31, 2022

View reviewed changes

jonathan-buttner added 3 commits November 1, 2022 15:37

Addressing feedback

e10d180

Merge branch 'main' of github.com:elastic/kibana into opsgenie-ui-pha…

f29e51e

…se-2

Adding new image with lowercase tags

ebe4839

jonathan-buttner mentioned this pull request Nov 2, 2022

[ResponseOps][Stack Connectors] Opsgenie UI refinements #144486

Merged

peteharverson approved these changes Nov 3, 2022

View reviewed changes

ymao1 approved these changes Nov 3, 2022

View reviewed changes

cnasikas reviewed Nov 3, 2022

View reviewed changes

Addressing feedback

0f11781

jonathan-buttner requested a review from a team November 3, 2022 18:25

jonathan-buttner and others added 2 commits November 3, 2022 14:25

Merge branch 'main' into opsgenie-ui-phase-2

272d456

Making executionMode optional

2b1e572

cnasikas approved these changes Nov 4, 2022

View reviewed changes

jonathan-buttner and others added 2 commits November 7, 2022 09:04

Merge branch 'main' into opsgenie-ui-phase-2

f9bc4b4

[CI] Auto-commit changed files from 'node scripts/generate codeowners'

b3dbba5

jonathan-buttner merged commit ae9dd59 into elastic:main Nov 7, 2022

jonathan-buttner deleted the opsgenie-ui-phase-2 branch November 7, 2022 16:22

cnasikas mentioned this pull request Nov 9, 2022

[Security Solution] Tines connector #143505

Merged

3 tasks

lcawl mentioned this pull request Nov 15, 2022

[DOCS] OpsGenie connector #145198

Closed

5 tasks

	URL:: The Opsgenie URL: https://api.opsgenie.com. For the EU instance of Opsgenie use: https://api.eu.opsgenie.com. If you are using the <<action-settings, `xpack.actions.allowedHosts`>> setting, make sure the hostname is added to the allowed hosts.
	URL:: The Opsgenie URL. For example, https://api.opsgenie.com or https://api.eu.opsgenie.com. If you are using the <<action-settings,`xpack.actions.allowedHosts`>> setting, make sure the hostname is added to the allowed hosts.

[ResponseOps][Stack Connectors] Opsgenie UI phase 2 #143480

[ResponseOps][Stack Connectors] Opsgenie UI phase 2 #143480

Conversation

jonathan-buttner commented Oct 17, 2022 • edited Loading

Testing

Retrieving an alert after it is created/closed

Using the Tiny ID

Alias

ID

Notes

Valid JSON examples

Examples

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

peteharverson left a comment

Choose a reason for hiding this comment

ymao1 left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

cnasikas Nov 3, 2022 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

jonathan-buttner Nov 3, 2022 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

cnasikas left a comment • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

kibana-ci commented Nov 7, 2022 • edited Loading

💛 Build succeeded, but was flaky

Failed CI Steps

Test Failures

Metrics [docs]

Module Count

Public APIs missing comments

Async chunks

Page load bundle

API count

async chunk count

ESLint disabled in files

ESLint disabled line counts

Total ESLint disabled count

History

jonathan-buttner commented Oct 17, 2022 •

edited

Loading

cnasikas Nov 3, 2022 •

edited

Loading

jonathan-buttner Nov 3, 2022 •

edited

Loading

cnasikas left a comment •

edited

Loading

kibana-ci commented Nov 7, 2022 •

edited

Loading