headless svc forwarding behaviour changed for sts with single pod

[onmomo]

We noticed that kubefwd 14.x behaves differently for forwarding headless services forwarding to a single sts container

apiVersion: v1
kind: Service
metadata:
  name: kafka-headless-svc
spec:
  ports:
    - port: 9092
      name: broker
  clusterIP: None
  selector:
    app: kafka-sts

apiVersion: apps/v1beta1
kind: StatefulSet
metadata:
  name: kafka-sts
  labels:
    app: kafka-sts
spec:
  serviceName: kafka-headless-svc
  podManagementPolicy: OrderedReady
  replicas: 1
  updateStrategy:
    type: RollingUpdate
  template:
    metadata:
      labels:
        app: kafka-sts
    spec:
      containers:
      - name: kafka-sts
        image: "confluentinc/cp-kafka:5.0.1"
        imagePullPolicy: "IfNotPresent"
        ports:
        - containerPort: 9092
          name: kafka        
        env:
        - name: POD_IP
          valueFrom:
            fieldRef:
              fieldPath: status.podIP
        - name: HOST_IP
          valueFrom:
            fieldRef:
              fieldPath: status.hostIP
        - name: "KAFKA_LISTENER_SECURITY_PROTOCOL_MAP"
          value: "PLAINTEXT:PLAINTEXT,EXTERNAL:PLAINTEXT"    
        command:
        - sh
        - -exc
        - |
          export KAFKA_BROKER_ID=${HOSTNAME##*-} 
          export KAFKA_ADVERTISED_LISTENERS=PLAINTEXT://kafka-sts-${HOSTNAME##*-}.kafka-headless-svc:9092,EXTERNAL://${HOST_IP}:$((31090 + ${KAFKA_BROKER_ID})) && \
          exec /etc/confluent/docker/run          

kubefwd 1.13.1

INFO[13:29:28] Forwarding: kafka-headless-svc:9092 to pod kafka-sts-0:9092 
INFO[13:29:28] Forwarding: kafka-sts-0.kafka-headless-svc:9092 to pod kafka-sts-0:9092 

kubefwd 14.2

INFO[14:06:42] Port-Forward: kafka-headless-svc:9092 to pod kafka-sts-0:9092 
INFO[14:06:42] Port-Forward: kafka-headless-svc:9092 to pod kafka-sts-0:9092 

If the kafka-sts is scaled to 2 with 14.2, it will result in:

INFO[14:06:24] Port-Forward: kafka-headless-svc:9092 to pod kafka-sts-0:9092 
INFO[14:06:24] Port-Forward: kafka-sts-0.kafka-headless-svc:9092 to pod kafka-sts-0:9092 
INFO[14:06:24] Port-Forward: kafka-sts-1.kafka-headless-svc:9092 to pod kafka-sts-1:9092 

in 14.2, this is probably caused because of this check https://github.com/txn2/kubefwd/blob/1.14.2/cmd/kubefwd/services/services.go#L431
and this statement: https://github.com/txn2/kubefwd/blob/1.14.2/cmd/kubefwd/services/services.go#L458

Our apps are configured to use kafka-sts-0.kafka-headless-svc:9092 and therefore fail with 14.2.
We should be able to fix it while using kafka-headless-svc:9092 instead but we are wondering if this is the new way to go or an issue?

[cjimti]

This is a bug introduced in 1.14. The last few contributions have caused some issues I need to unwind. I am scheduling some time to work on this in the next two weeks.

Thank you for raising this issue.

[calmkart]

fix in #134
@onmomo

[onmomo]

@calmkart @cjimti thanks a lot and keep it up!

[cjimti]

Fixed in 1.17.2