Error sending event to webhook Summary or Text is required for MSTeams

[ramakrishnateja]

Describe the bug

Here is my canary definition

apiVersion: flagger.app/v1beta1
kind: Canary
metadata:
name: helloweather
spec:

service mesh provider can be: kubernetes, istio, appmesh, nginx, gloo

provider: kubernetes

deployment reference

targetRef:
apiVersion: apps/v1
kind: Deployment
name: helloweather-deployment

the maximum time in seconds for the canary deployment

to make progress before rollback (default 600s)

progressDeadlineSeconds: 60

HPA reference (optional)

autoscalerRef:
apiVersion: autoscaling/v2beta1
kind: HorizontalPodAutoscaler
name: helloweather-hpa
service:
name: helloweather
port: 80
portDiscovery: true
analysis:
# schedule interval (default 60s)
interval: 30s
# max number of failed checks before rollback
threshold: 2
# number of checks to run before rollback
iterations: 10
# Prometheus checks based on
# http_request_duration_seconds histogram
metrics:
- name: request-success-rate
# minimum req success rate (non 5xx responses)
# percentage (0-100)
thresholdRange:
min: 99
interval: 1m
- name: request-duration
# maximum req duration P99
# milliseconds
thresholdRange:
max: 500
interval: 30s
# acceptance/load testing hooks
webhooks:
- name: "send to Teams"
type: event
url: "https://teamswebhook"
- name: smoke-test
type: pre-rollout
url: http://flagger-loadtester.linkerd/
timeout: 15s
metadata:
type: bash
cmd: "curl http://helloweather.manualbluegreen:80/WeatherForecast"
- name: gate
type: confirm-rollout
url: http://flagger-loadtester.linkerd/gate/check
- name: load-test
url: http://flagger-loadtester.linkerd/
timeout: 5s
metadata:
type: cmd
cmd: "hey -z 1m -q 10 -c 2 http://helloweather.manualbluegreen:80/WeatherForecast/"
alerts:
- name: "Error MS Teams"
severity: error
providerRef:
name: "msteams"
- name: "Warn MS Teams"
severity: warn
providerRef:
name: "msteams"
- name: "Info MS Teams"
severity: info
providerRef:
name: "msteams"

To Reproduce

When the canary is applied and image for deployment changed i am able to see the events raised by canary by doing a kubectl get events but not seeing the events sent to the MSTeams channel. Its failing with following error.

{"level":"error","ts":"2022-03-14T18:02:17.239Z","caller":"controller/events.go:69","msg":"error sending event to webhook: Summary or Text is required.","canary":"helloweather.manualbluegreen","stacktrace":"github.com/fluxcd/flagger/pkg/controller.(*Controller).sendEventToWebhook\n\t/workspace/pkg/controller/events.go:69\ngithub.jparrowsec.cn/fluxcd/flagger/pkg/controller.(*Controller).recordEventInfof\n\t/workspace/pkg/controller/events.go:35\ngithub.jparrowsec.cn/fluxcd/flagger/pkg/controller.(*Controller).runBlueGreen\n\t/workspace/pkg/controller/scheduler.go:620\ngithub.jparrowsec.cn/fluxcd/flagger/pkg/controller.(*Controller).advanceCanary\n\t/workspace/pkg/controller/scheduler.go:410\ngithub.jparrowsec.cn/fluxcd/flagger/pkg/controller.CanaryJob.Start.func1\n\t/workspace/pkg/controller/job.go:39"}

Expected behavior

Should see the events sent to MSTeams

Additional context

• Flagger version: 1.19.0
• Kubernetes version: 1.21
• Service Mesh provider: linkerd
• Ingress provider: nginx

[aryan9600]

This is the payload that Flagger sends to webhooks:

{
    "name": "podinfo",
    "namespace": "test",
    "phase": "Progressing", 
    "metadata": {
        "test":  "all",
        "token":  "16688eb5e9f289f1991c"
    }
}

This is not compatible with the webhook payload that MS teams requires.

[stefanprodan]

We do have support for MS Teams by setting type: msteams in the AlertProvider, docs here: https://docs.flagger.app/usage/alerting#canary-configuration

[claudiu-albu]

@stefanprodan

We do have support for MS Teams by setting type: msteams in the AlertProvider, docs here: https://docs.flagger.app/usage/alerting#canary-configuration

I'm trying exactly this with following setup:

Flagger AlertProvider:

apiVersion: flagger.app/v1beta1
kind: AlertProvider
metadata:
  name: msteams-notifs
  namespace: flagger
spec:
  type: msteams
  secretRef:
    name: msteams-webhook

Flagger HelmRelease:

apiVersion: helm.toolkit.fluxcd.io/v2
kind: HelmRelease
metadata:
  name: flagger
  namespace: flagger
spec:
  interval: 5m
  chart:
    spec:
      chart: flagger
      version: 1.38.0
      sourceRef:
        kind: HelmRepository
        name: flagger
        namespace: flagger
      interval: 1m
  install:
    remediation:
      retries: 3
  upgrade:
    remediation:
      remediateLastFailure: false
  values:
    meshProvider: istio
    metricsServer: http://kube-prometheus-stack-prometheus.monitoring.svc.cluster.local:9090
    clusterName: "${project}-${environment_name}"

and added this bit to Flagger Canary, in spec.analysis:

    alerts:
      - name: "MS Teams Notifications"
        severity: info
        providerRef:
          name: msteams-notifs
          namespace: flagger

I may not be entirely clear about what documentation explains,
but I can't for the life of me make it send MS Teams notifications other than:

and:

(these might as well be generated by underlying FluxCD alerting given their title "FluxCD Alerts". not entirely sure at the moment.)

this is a typical output for my canary deployment testing:

but even when I make Canary fail altogether, I still don't get anything corresponding in MS Teams, like documentation shows here:

while using setup shown above, I don't get any errors in Flagger k8s pod either while Canary deployment is running.

in Flagger HelmRelease, in spec.values, I've also tried using:

    env:
    - name: MSTEAMS_URL
      valueFrom:
       secretKeyRef:
         name: msteams-webhook
         key: address

or:

    msteams:
      url: <MSTEAMS_WEBHOOK_URL>

but this only results in making error "Error sending event to webhook Summary or Text is required for MSTeams" show up in Flagger pod logs again.

Additional context

• Flagger version: 1.38.0
• Kubernetes version: 1.28.11
• Service Mesh provider: Istio

so am I getting it wrong in terms of what Flagger MS Teams alerting does?
what can I actually make Flagger alert on in MS Teams notifications?

any input would be greatly appreciated.
thanks in advance,
Claudiu