Run envoy as non-root

[skonto]

Changes

• Runs envoy as non-root, similar to cmd/contour, examples: Run Envoy as non-root projectcontour/contour#4084
• On OCP we use non-root for some time.
• Tested on kind 1.25 with restricted PSP:

  kubectl create ns kourier-system
  kubectl label --overwrite ns kourier-system\
    pod-security.kubernetes.io/enforce=restricted \
    pod-security.kubernetes.io/warn=restricted \
    pod-security.kubernetes.io/audit=restricted
$ oc get pods -n kourier-system
NAME                                      READY   STATUS    RESTARTS   AGE
3scale-kourier-gateway-85878884dd-4wp6n   1/1     Running   0          16m

[knative-prow]

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: skonto

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

• OWNERS [skonto]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

[codecov]

Codecov Report

Merging #934 (18bf5db) into main (ef78043) will not change coverage.
The diff coverage is n/a.

@@           Coverage Diff           @@
##             main     #934   +/-   ##
=======================================
  Coverage   80.93%   80.93%           
=======================================
  Files          18       18           
  Lines        1243     1243           
=======================================
  Hits         1006     1006           
  Misses        190      190           
  Partials       47       47           

Help us with your feedback. Take ten seconds to tell us how you rate us. Have a feature suggestion? Share it here.

[psschwei]

Does @nak3 's comment here still apply?

[nak3]

Yeah, I think runAsUser is not allowed on OCP by default (or some condition).

The doc still mentions about it:
https://docs.openshift.com/container-platform/4.11/authentication/managing-security-context-constraints.html#security-context-constraints-example_configuring-internal-oauth

A container or pod that requests a specific user ID will be accepted by OpenShift Container Platform only when a service account or a user is granted access to a SCC that allows such a user ID. The SCC can allow arbitrary IDs, an ID that falls into a range, or the exact user ID specific to the request.

With said, we (downstream) already tweak runAsNonRoot by a patch, so Stavros is going to patch the change (runAsUser) on downstream? (Or we don't care about it in the newer OCP version?)

[nak3]

/lgtm
I assume that OCP will be solved somehow.

[skonto]

@nak3 yeah probably we will patch it, it is not a problem ;) My rationale is keep moving forward with these changes.