[BUG] traefik ingressroute doesnt work out of the box with latest k3d and k3s

[bayeslearnerold]

What did you do

• How was the cluster created?

  • k3d cluster create -x A -y B

kind: Simple
apiVersion: k3d.io/v1alpha3
name: k3d-learn
servers: 3
agents: 2
kubeAPI:
  host:  bayes-learn.home
  hostIP: 0.0.0.0
  hostPort: "6443"
image: rancher/k3s:v1.22.2-k3s1
volumes:
- volume: $HOME/Data/k3dvol:/var/lib/rancher/k3s/storage
  nodeFilters:
  - all
ports:
- port: 80:80
  nodeFilters:
  - loadbalancer
- port: 443:443
  nodeFilters:
  - loadbalancer
- port: 9443:9443
  nodeFilters:
  - loadbalancer
options:
  k3d:
    wait: true
    timeout: 6m0s
    disableLoadbalancer: false
    disableImageVolume: false
    disableRollback: false
  k3s:
    extraArgs:
    - arg: --tls-san=127.0.0.1
      nodeFilters:
      - server:*
    nodeLabels: []
  kubeconfig:
    updateDefaultKubeconfig: true
    switchCurrentContext: true
  runtime:
    gpuRequest: ""
    serversMemory: ""
    agentsMemory: ""
    labels:
    - label: foo=bar
      nodeFilters:
      - server:0
      - loadbalancer
env:
- envVar: bar=baz
  nodeFilters:
  - all

• What did you do afterwards?

What did you expect to happen

Concise description of what you expected to happen after doing what you described above.

apiVersion: apps/v1
kind: Deployment
metadata:
  name: whoami
  namespace: dev
  labels:
    app: whoami
spec:
  replicas: 1
  selector:
    matchLabels:
      app: whoami
  template:
    metadata:
      labels:
        app: whoami
    spec:
      containers:
      - image: docker.io/containous/whoami:v1.5.0
        name: whoami
        ports:
        - containerPort: 80
---
apiVersion: v1
kind: Service
metadata:
  name: whoami
  namespace: dev
spec:
  ports:
  - name: whoami
    port: 80
    targetPort: 80
  selector:
    app: whoami
---
apiVersion: traefik.containo.us/v1alpha1
kind: IngressRoute
metadata:
  name: whoami
  namespace: dev
spec:
  entryPoints:
    - websecure
  routes:
  - kind: Rule
    match: Host(`whoami.192.168.8.118.nip.io`)
    services:
    - name: whoami
      port: 80

I'm getting 404 error.

[bayeslearnerold]

kubectl -n kube-system port-forward $(kubectl -n kube-system get pods --selector "app.kubernetes.io/name=traefik" --output=name) 9000:9000

takes me to inspect the routes and the needed IngressRoute is not created there.

[bayeslearnerold]

helm -n kube-system get values traefik -a

shows Ingressroute provider is enabled. Why didn't it work?

COMPUTED VALUES:
additionalArguments: []
additionalVolumeMounts: []
affinity: {}
autoscaling:
  enabled: false
deployment:
  additionalContainers: []
  additionalVolumes: []
  annotations: {}
  enabled: true
  imagePullSecrets: []
  initContainers: []
  kind: Deployment
  labels: {}
  podAnnotations: {}
  podLabels: {}
  replicas: 1
env: []
envFrom: []
experimental:
  kubernetesGateway:
    appLabelSelector: traefik
    certificates: []
    enabled: false
  plugins:
    enabled: false
global:
  systemDefaultRegistry: ""
globalArguments:
- --global.checknewversion
- --global.sendanonymoususage
hostNetwork: false
image:
  name: rancher/mirrored-library-traefik
  pullPolicy: IfNotPresent
  tag: ""
ingressClass:
  enabled: false
  fallbackApiVersion: ""
  isDefaultClass: false
ingressRoute:
  dashboard:
    annotations: {}
    enabled: true
    labels: {}
logs:
  access:
    enabled: false
    fields:
      general:
        defaultmode: keep
        names: {}
      headers:
        defaultmode: drop
        names: {}
    filters: {}
  general:
    level: ERROR
metrics:
  prometheus:
    entryPoint: metrics
nodeSelector: {}
persistence:
  accessMode: ReadWriteOnce
  annotations: {}
  enabled: false
  name: data
  path: /data
  size: 128Mi
pilot:
  enabled: false
  token: ""
podAnnotations:
  prometheus.io/port: "8082"
  prometheus.io/scrape: "true"
podDisruptionBudget:
  enabled: false
podSecurityContext:
  fsGroup: 65532
podSecurityPolicy:
  enabled: false
ports:
  metrics:
    expose: false
    exposedPort: 9100
    port: 9100
    protocol: TCP
  traefik:
    expose: false
    exposedPort: 9000
    port: 9000
    protocol: TCP
  web:
    expose: true
    exposedPort: 80
    port: 8000
    protocol: TCP
  websecure:
    expose: true
    exposedPort: 443
    port: 8443
    protocol: TCP
    tls:
      certResolver: ""
      domains: []
      enabled: true
      options: ""
priorityClassName: system-cluster-critical
providers:
  kubernetesCRD:
    enabled: true
    namespaces: []
  kubernetesIngress:
    enabled: true
    namespaces: []
    publishedService:
      enabled: true
rbac:
  enabled: true
  namespaced: false
resources: {}
rollingUpdate:
  maxSurge: 1
  maxUnavailable: 1
securityContext:
  capabilities:
    drop:
    - ALL
  readOnlyRootFilesystem: true
  runAsGroup: 65532
  runAsNonRoot: true
  runAsUser: 65532
service:
  annotations: {}
  annotationsTCP: {}
  annotationsUDP: {}
  enabled: true
  externalIPs: []
  labels: {}
  loadBalancerSourceRanges: []
  spec: {}
  type: LoadBalancer
serviceAccount:
  name: ""
serviceAccountAnnotations: {}
tlsOptions: {}
tolerations:
- key: CriticalAddonsOnly
  operator: Exists
- effect: NoSchedule
  key: node-role.kubernetes.io/control-plane
  operator: Exists
- effect: NoSchedule
  key: node-role.kubernetes.io/master
  operator: Exists
volumes: []

[iwilltry42]

Hi @bayeslearner , thanks for opening this issue! Sorry for getting back to it this late.
Can you paste your curl output here as well?
I never really used the IngressRoute resource, but from a first glance I think it should work that way if you do curl -k https://whoami.192.168.8.118.nip.io (given that that's your correct IP) 🤔

[iwilltry42]

I just gave this a try (just using 8443 instead of 443 on my machine as it's in use already):

$ curl -k https://whoami.192.168.1.97.nip.io:8443
Hostname: whoami-645d767f47-4q2bl
IP: 127.0.0.1
IP: ::1
IP: 10.42.1.3
IP: fe80::50db:5dff:fe57:fbf3
RemoteAddr: 10.42.3.3:46822
GET / HTTP/1.1
Host: whoami.192.168.1.97.nip.io:8443
User-Agent: curl/7.81.0
Accept: */*
Accept-Encoding: gzip
X-Forwarded-For: 10.42.3.1
X-Forwarded-Host: whoami.192.168.1.97.nip.io:8443
X-Forwarded-Port: 8443
X-Forwarded-Proto: https
X-Forwarded-Server: traefik-74dd4975f9-74qzn
X-Real-Ip: 10.42.3.1

I also see the Route in the Traefik dashboard: