design new external service resource

[lahabana]

Description

For #1496 we need to move external service to be namespace based. It's a good time to rethink this resource and improve it.

Rough design from a thinking session:

type: externalService
metadata:
  namespace: kuma-system
  name: my-service
spec:
  hostname: my-service.foo.bar
  kuma.io/service: my-service
  protocol: http
  port: 443
  instances:
     - address: east-1
       port: 334
       tags:
          region: east-1
       tls:
         enabled: true
     - address: west-1
       port: 335
       tags:
          region: west-1

Each instance of a resource is an envoy cluster.
Is each instance inside the resource a different envoy endpoint? If yes can we have different TLS config for example (seems like it)?

Should we disallow having an internal and an external service with the same name? --> Probably

There are a few issues around ES that we should take into consideration for this:

• External Service with kafka protocol is not working  #6180
• External Service that points to CNAME #6041
• Cannot disable SNI in external service #5474
• External Service easily misconfigured in an insecure way #4758
• Have ExternalService mTLS feature to support same hosts with multiple mTLS credentials. #4400
• ExternalService with wrong address leads to the lack of listeners in DPP #3980
• external services with different hostnames will not route to specific hosts #3097
• Option to use internal cert key pair for external services #2248
• ExternalService with kuma.io/zone tag behave differently w/ and w/o ZoneEgress #6352
• ExternalServices from different zones doesn't work on ZoneEgress #6391
• Limitation when apply TrafficPermission, FaultInjection and RateLimit on ZoneEgress kuma-website#727
• Skip hostname verification for external services #7121
• Handling wildcards in hostnames of called external services #7200
• Change of mesh in ExternalService doesn't change it's ownerReference  #8721
• ExternalService support for SRV record (if possible)
• Integrate with MeshExternalService (externalService v2) kuma-gui#2471

[lahabana]

Think about this in the context of a lambda as well

[jakubdyszkiewicz]

Take into account this issue when redesigning kumahq/kuma-gui#1529

[slonka]

Think about this in the context of a lambda as well

do you mean consuming in Kuma a lambda exposed by a AWS API gateway? if so then what specific things do we need to worry about? If I'm reading things right lambdas just get a domain like r275xc9bmd.execute-api.us-east-1.amazonaws.com so they should be just like any other ES.

Take into account this issue when redesigning kumahq/kuma-gui#1529

the problem here is that we're getting multiple externalservices pointing to the same domain or is it something else? I think I found the original issue, it's probably about using https://www.envoyproxy.io/docs/envoy/latest/configuration/http/http_filters/aws_lambda_filter

[bartsmykla]

xref: #4758

[lahabana]

Design is done #10238 and #10300 start tracking the actual implem