Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[Question] Windows node networking #177

Open
tmyroadctfig opened this issue Apr 9, 2024 · 2 comments
Open

[Question] Windows node networking #177

tmyroadctfig opened this issue Apr 9, 2024 · 2 comments
Assignees
@abhilashaagarwala
Labels
question Further information is requested

Comments

@tmyroadctfig
Copy link

Describe scenario
I've set up a single machine k3s cluster with a linux and Windows worker node, and deployed the sample applications:

kubectl apply -f  https://raw.githubusercontent.com/Azure/AKS-Edge/main/samples/others/linux-sample.yaml
kubectl apply -f https://raw.githubusercontent.com/Azure/AKS-Edge/main/samples/others/win-sample.yaml

Question
I can't seem to access the Windows sample container service, either from the host machine, or from the linux pods. Is any further setup required to get the Windows worker node networking setup correctly?

PS C:\Users\luke> kubectl.exe get services -o wide
NAME               TYPE           CLUSTER-IP      EXTERNAL-IP   PORT(S)        AGE   SELECTOR
azure-vote-back    ClusterIP      10.43.43.226    <none>        6379/TCP       18h   app=azure-vote-back
azure-vote-front   LoadBalancer   10.43.225.42    <pending>     80:30010/TCP   18h   app=azure-vote-front
kubernetes         ClusterIP      10.43.0.1       <none>        443/TCP        18h   <none>
sample             NodePort       10.43.124.239   <none>        80:31230/TCP   3s    app=sample
PS C:\Users\luke> curl -UseBasicParsing -Uri http://192.168.0.3:31230
curl : Unable to connect to the remote server
At line:1 char:1
+ curl -UseBasicParsing -Uri http://192.168.0.3:31230
+ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
    + CategoryInfo          : InvalidOperation: (System.Net.HttpWebRequest:HttpWebRequest) [Invoke-WebRequest], WebExc
   eption
    + FullyQualifiedErrorId : WebCmdletWebResponseException,Microsoft.PowerShell.Commands.InvokeWebRequestCommand

PS C:\Users\luke> Get-AksEdgeNodeAddr -NodeType Windows

[04/09/2024 00:04:31] Querying IP and MAC addresses from virtual machine (luke-testing-wedge)

 - Virtual machine MAC: 00:15:5d:00:05:0b
 - Virtual machine IP : 192.168.0.3 retrieved directly from virtual machine

Name                           Value
----                           -----
IpAddress                      192.168.0.3
MacAddress                     00:15:5d:00:05:0b

FWIW, running a web request from the Windows node to the linux node is working without any issues:

PS C:\inetpub\wwwroot> Invoke-WebRequest -Uri http://192.168.0.2:30010 -UseBasicParsing


StatusCode        : 200
StatusDescription : OK
Content           : <!DOCTYPE html>
                    <html xmlns="http://www.w3.org/1999/xhtml">
                    <head>
                        <link rel="stylesheet" type="text/css" href="/static/default.css">
                        <title>Azure Voting App</title>

                        <script language="Jav...
RawContent        : HTTP/1.1 200 OK
                    Connection: keep-alive
                    Content-Length: 950
                    Content-Type: text/html; charset=utf-8
                    Date: Mon, 08 Apr 2024 23:55:30 GMT
                    Server: nginx/1.13.7

                    <!DOCTYPE html>
                    <html xmlns="http://w...
Forms             :
Headers           : {[Connection, keep-alive], [Content-Length, 950], [Content-Type, text/html; charset=utf-8], [Date, Mon, 
                    08 Apr 2024 23:55:30 GMT]...}
Images            : {}
InputFields       : {}
Links             : {}
ParsedHtml        :
RawContentLength  : 950
@tmyroadctfig tmyroadctfig added the question Further information is requested label Apr 9, 2024
@haodeon
Copy link

haodeon commented Sep 12, 2024

I am encountering the same issue if I restart the Windows node.

I am not able to connect on the Windows nodeport but the nodeport on Linux works. Looking into it further there appears to be missing HNS Endpoints after the node is restarted. Only the Outbound NAT endpoint for routing to the Linux node pod network appears which explains how it's able to connect on the Linux nodeport.

@haodeon
Copy link

haodeon commented Sep 12, 2024

Tested k8s with calico.

At first windows pod networking didn't work at all. Applied the registry fix from microsoft/Windows-Containers#516

Then tested restarting the windows node. HNS Endpoints missing. Found projectcalico/calico#5164. Deleted the service and deployment, reapplied the manifest and endpoints came back.

Retested k3s+flannel with the registry fix. Networking still broken after node restart. Doesn't matter if resources are deleted and reapplied.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@abhilashaagarwala abhilashaagarwala

Labels
question Further information is requested
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
3 participants
@tmyroadctfig @haodeon @abhilashaagarwala