Skip to content

Commit a37866c

Browse files
committed
ci: setup cilium+hubble nightly run
deps: bump github.com/spf13/viper from 1.16.0 to 1.18.2 (#2495) Bumps [github.com/spf13/viper](https://github.com/spf13/viper) from 1.16.0 to 1.18.2. - [Release notes](https://github.com/spf13/viper/releases) - [Commits](spf13/viper@v1.16.0...v1.18.2) --- updated-dependencies: - dependency-name: github.com/spf13/viper dependency-type: direct:production update-type: version-update:semver-minor ... Signed-off-by: dependabot[bot] <[email protected]> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> change config to be uniform ci: increase dependabot PR limit to 10 (#2497) ci: constrain cilium to linux on Release Test Pipeline (#2489) ci: constrain cilium to linux ci: Improve CNI|NPM integration test (#2498) * ci: increase timeout for CNI|NPM intergration test * add: retry to generate NPM logs deps: bump golang.org/x/sys from 0.15.0 to 0.16.0 (#2500) Bumps [golang.org/x/sys](https://github.com/golang/sys) from 0.15.0 to 0.16.0. - [Commits](golang/sys@v0.15.0...v0.16.0) --- updated-dependencies: - dependency-name: golang.org/x/sys dependency-type: direct:production update-type: version-update:semver-minor ... Signed-off-by: dependabot[bot] <[email protected]> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> deps: bump github.com/prometheus/client_golang from 1.16.0 to 1.18.0 (#2501) Bumps [github.com/prometheus/client_golang](https://github.com/prometheus/client_golang) from 1.16.0 to 1.18.0. - [Release notes](https://github.com/prometheus/client_golang/releases) - [Changelog](https://github.com/prometheus/client_golang/blob/main/CHANGELOG.md) - [Commits](prometheus/client_golang@v1.16.0...v1.18.0) --- updated-dependencies: - dependency-name: github.com/prometheus/client_golang dependency-type: direct:production update-type: version-update:semver-minor ... Signed-off-by: dependabot[bot] <[email protected]> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> deps: bump google.golang.org/protobuf from 1.31.0 to 1.32.0 (#2502) Bumps google.golang.org/protobuf from 1.31.0 to 1.32.0. --- updated-dependencies: - dependency-name: google.golang.org/protobuf dependency-type: direct:production update-type: version-update:semver-minor ... Signed-off-by: dependabot[bot] <[email protected]> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> deps: bump golang.org/x/crypto from 0.17.0 to 0.18.0 (#2492) Bumps [golang.org/x/crypto](https://github.com/golang/crypto) from 0.17.0 to 0.18.0. - [Commits](golang/crypto@v0.17.0...v0.18.0) --- updated-dependencies: - dependency-name: golang.org/x/crypto dependency-type: direct:production update-type: version-update:semver-minor ... Signed-off-by: dependabot[bot] <[email protected]> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> deps: bump k8s.io/klog/v2 from 2.110.1 to 2.120.0 (#2503) Bumps [k8s.io/klog/v2](https://github.com/kubernetes/klog) from 2.110.1 to 2.120.0. - [Release notes](https://github.com/kubernetes/klog/releases) - [Changelog](https://github.com/kubernetes/klog/blob/main/RELEASE.md) - [Commits](kubernetes/klog@v2.110.1...v2.120.0) --- updated-dependencies: - dependency-name: k8s.io/klog/v2 dependency-type: direct:production update-type: version-update:semver-minor ... Signed-off-by: dependabot[bot] <[email protected]> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> ci: verbose output for make test-load (#2481) ci: make test-load verbose output fix review comments deps: bump google.golang.org/grpc from 1.59.0 to 1.60.1 (#2494) Bumps [google.golang.org/grpc](https://github.com/grpc/grpc-go) from 1.59.0 to 1.60.1. - [Release notes](https://github.com/grpc/grpc-go/releases) - [Commits](grpc/grpc-go@v1.59.0...v1.60.1) --- updated-dependencies: - dependency-name: google.golang.org/grpc dependency-type: direct:production update-type: version-update:semver-minor ... Signed-off-by: dependabot[bot] <[email protected]> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> ci: add OS scale timeout (#2490) ci: add kubeproxy restart before windows k8s conformance testing (#2496) * ci: ensure kubeproxy is running on windows * ci: add set -e Signed-off-by: John Payne <[email protected]> --------- Signed-off-by: John Payne <[email protected]> ci: Improve TestValidateState calls from load package (#2484) * ci: improve TestValidateState * chore: address comments fix: joining multiple dns server lists (#2506) * fix: joining multiple dns server lists * address comments * add 0 check fix: downgrade k8s components to 1.28 (#2505) Signed-off-by: GitHub <[email protected]>
1 parent 8bfb906 commit a37866c

25 files changed

+1669
-457
lines changed

.github/dependabot.yaml

+7
Original file line numberDiff line numberDiff line change
@@ -10,6 +10,7 @@ updates:
1010
commit-message:
1111
prefix: "ci"
1212
labels: [ "ci", "dependencies" ]
13+
open-pull-requests-limit: 10
1314
- package-ecosystem: "github-actions"
1415
directory: "/"
1516
schedule:
@@ -19,6 +20,7 @@ updates:
1920
commit-message:
2021
prefix: "ci"
2122
labels: [ "ci", "dependencies" ]
23+
open-pull-requests-limit: 10
2224
- package-ecosystem: "gomod"
2325
directory: "/"
2426
schedule:
@@ -27,6 +29,7 @@ updates:
2729
- "azure/azure-sdn-members"
2830
commit-message:
2931
prefix: "deps"
32+
open-pull-requests-limit: 10
3033
- package-ecosystem: "gomod"
3134
directory: "/azure-ipam"
3235
schedule:
@@ -35,6 +38,7 @@ updates:
3538
- "azure/azure-sdn-members"
3639
commit-message:
3740
prefix: "deps"
41+
open-pull-requests-limit: 10
3842
- package-ecosystem: "gomod"
3943
directory: "/build/tools"
4044
schedule:
@@ -43,15 +47,18 @@ updates:
4347
- "azure/azure-sdn-members"
4448
commit-message:
4549
prefix: "deps"
50+
open-pull-requests-limit: 10
4651
- package-ecosystem: "gomod"
4752
directory: "/dropgz"
4853
schedule:
4954
interval: "daily"
5055
commit-message:
5156
prefix: "deps"
57+
open-pull-requests-limit: 10
5258
- package-ecosystem: "gomod"
5359
directory: "/zapai"
5460
schedule:
5561
interval: "daily"
5662
commit-message:
5763
prefix: "deps"
64+
open-pull-requests-limit: 10

.pipelines/cni/cilium/cilium-overlay-load-test-template.yaml

+1-1
Original file line numberDiff line numberDiff line change
@@ -14,7 +14,7 @@ parameters:
1414
# CONTROL_CNI either contains 'cniv1' or 'all'. It is not case sensitive
1515
stages:
1616
- stage: create_${{ parameters.name }}
17-
condition: and( succeeded(), or( contains(variables.CONTROL_CNI, 'cilium') , contains(variables.CONTROL_CNI, 'all') ) )
17+
condition: and( succeeded(), and( or( contains(variables.CONTROL_CNI, 'cilium') , contains(variables.CONTROL_CNI, 'all') ), or( contains(variables.CONTROL_OS, 'linux'), contains(variables.CONTROL_OS, 'all') ) ) )
1818
variables:
1919
commitID: $[ stagedependencies.setup.env.outputs['SetEnvVars.commitID'] ]
2020
dependsOn:

.pipelines/cni/k8s-e2e/k8s-e2e-job-template.yaml

+15
Original file line numberDiff line numberDiff line change
@@ -34,6 +34,21 @@ jobs:
3434
tar -xvzf kubernetes-test-linux-amd64.tar.gz --strip-components=3 kubernetes/test/bin/ginkgo kubernetes/test/bin/e2e.test
3535
3636
displayName: "Setup Environment"
37+
- ${{ if contains(parameters.os, 'windows') }}:
38+
- script: |
39+
set -e
40+
kubectl apply -f test/integration/manifests/load/privileged-daemonset-windows.yaml
41+
kubectl rollout status -n kube-system ds privileged-daemonset
42+
43+
kubectl get pod -n kube-system -l app=privileged-daemonset,os=windows -owide
44+
pods=`kubectl get pod -n kube-system -l app=privileged-daemonset,os=windows --no-headers | awk '{print $1}'`
45+
for pod in $pods; do
46+
kubectl exec -i -n kube-system $pod -- powershell "Restart-Service kubeproxy"
47+
kubectl exec -i -n kube-system $pod -- powershell "Get-Service kubeproxy"
48+
done
49+
name: kubeproxy
50+
displayName: Restart Kubeproxy on Windows nodes
51+
retryCountOnTaskFailure: 3
3752
- ${{ if eq(parameters.datapath, true) }}:
3853
- template: ../k8s-e2e/k8s-e2e-step-template.yaml
3954
parameters:
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,145 @@
1+
pr: none
2+
trigger: none
3+
4+
schedules:
5+
- cron: '0 0 * * *'
6+
displayName: Daily midnight Cilium with Hubble
7+
branches:
8+
include:
9+
- master
10+
11+
variables:
12+
clustername: ciliumhubble-$(Build.SourceBranchName)-$(Build.BuildId)
13+
cilium_version: v1.14.4
14+
15+
stages:
16+
- stage: createCluster
17+
pool:
18+
name: $(BUILD_POOL_NAME_DEFAULT)
19+
displayName: "create cluster"
20+
jobs:
21+
- template: ../templates/create-cluster.yaml
22+
parameters:
23+
name: cilium_overlay_hubble
24+
displayName: Cilium on AKS Overlay with Hubble Enabled
25+
clusterType: overlay-byocni-nokubeproxy-up
26+
clusterName: $(clustername)
27+
vmSize: Standard_B2ms
28+
k8sVersion: ""
29+
region: $(REGION_AKS_CLUSTER_TEST)
30+
- stage: setupCluster
31+
pool:
32+
name: $(BUILD_POOL_NAME_DEFAULT)
33+
jobs:
34+
- job: "ciliuminstall"
35+
steps:
36+
- bash: |
37+
go version
38+
go env
39+
mkdir -p '$(GOBIN)'
40+
mkdir -p '$(GOPATH)/pkg'
41+
mkdir -p '$(modulePath)'
42+
echo '##vso[task.prependpath]$(GOBIN)'
43+
echo '##vso[task.prependpath]$(GOROOT)/bin'
44+
name: "GoEnv"
45+
displayName: "Set up the Go environment"
46+
47+
- task: KubectlInstaller@0
48+
inputs:
49+
kubectlVersion: latest
50+
51+
- task: AzureCLI@1
52+
inputs:
53+
azureSubscription: $(BUILD_VALIDATIONS_SERVICE_CONNECTION)
54+
scriptLocation: "inlineScript"
55+
scriptType: "bash"
56+
addSpnToEnvironment: true
57+
inlineScript: |
58+
set -e
59+
make -C ./hack/aks set-kubeconf AZCLI=az CLUSTER=$(clustername)
60+
name: "setupkubeconf"
61+
displayName: "Set up kubeconfig"
62+
63+
- script: |
64+
CILIUM_CLI_VERSION=$(curl -s https://raw.githubusercontent.com/cilium/cilium-cli/main/stable.txt)
65+
CLI_ARCH=amd64
66+
if [ "$(uname -m)" = "aarch64" ]; then CLI_ARCH=arm64; fi
67+
curl -L --fail --remote-name-all https://github.com/cilium/cilium-cli/releases/download/${CILIUM_CLI_VERSION}/cilium-linux-${CLI_ARCH}.tar.gz{,.sha256sum}
68+
sha256sum --check cilium-linux-${CLI_ARCH}.tar.gz.sha256sum
69+
sudo tar xzvfC cilium-linux-${CLI_ARCH}.tar.gz /usr/local/bin
70+
rm cilium-linux-${CLI_ARCH}.tar.gz{,.sha256sum}
71+
name: "installCiliumCLI"
72+
displayName: "Install Cilium CLI"
73+
74+
- task: AzureCLI@1
75+
inputs:
76+
azureSubscription: $(BUILD_VALIDATIONS_SERVICE_CONNECTION)
77+
scriptLocation: "inlineScript"
78+
scriptType: "bash"
79+
addSpnToEnvironment: true
80+
inlineScript: |
81+
kubectl apply -f test/integration/manifests/cilium/$(cilium_version)/cilium-config/cilium-config-hubble.yaml
82+
kubectl apply -f test/integration/manifests/cilium/$(cilium_version)/cilium-agent/files
83+
kubectl apply -f test/integration/manifests/cilium/$(cilium_version)/cilium-operator/files
84+
envsubst '${CILIUM_VERSION_TAG},${CILIUM_IMAGE_REGISTRY}' < test/integration/manifests/cilium/$(cilium_version)/cilium-agent/templates/daemonset.tpl | kubectl apply -f -
85+
envsubst '${CILIUM_VERSION_TAG},${CILIUM_IMAGE_REGISTRY}' < test/integration/manifests/cilium/$(cilium_version)/cilium-operator/templates/deployment.tpl | kubectl apply -f -
86+
# Use different file directories for nightly and current cilium version
87+
name: "installCilium"
88+
displayName: "Install Cilium on AKS Overlay"
89+
90+
- script: |
91+
echo "Start Azilium E2E Tests on Overlay Cluster"
92+
if [ "$CILIUM_VERSION_TAG" = "cilium-nightly-pipeline" ]
93+
then
94+
CNS=$(CNS_VERSION) IPAM=$(AZURE_IPAM_VERSION) && echo "Running nightly"
95+
else
96+
CNS=$(make cns-version) IPAM=$(make azure-ipam-version)
97+
fi
98+
sudo -E env "PATH=$PATH" make test-integration AZURE_IPAM_VERSION=${IPAM} CNS_VERSION=${CNS} INSTALL_CNS=true INSTALL_OVERLAY=true
99+
retryCountOnTaskFailure: 3
100+
name: "aziliumTest"
101+
displayName: "Install Azure-CNS and Run Azilium E2E on AKS Overlay"
102+
enabled: true
103+
104+
- script: |
105+
cilium status --wait --wait-duration 5m
106+
name: waitforhealthy
107+
displayName: "Wait for healthy cilium pods"
108+
109+
- script: |
110+
kubectl wait --for=condition=Ready pods --all --all-namespaces --timeout=5m
111+
name: waitforallpodsrunning
112+
displayName: "Wait for all pods to be running"
113+
retryCountOnTaskFailure: 3
114+
115+
- script: |
116+
echo "Run Cilium Connectivity Tests"
117+
cilium status
118+
cilium connectivity test --connect-timeout 4s --request-timeout 30s --test '!pod-to-pod-encryption,!node-to-node-encryption'
119+
retryCountOnTaskFailure: 3
120+
name: "ciliumConnectivityTests"
121+
displayName: "Run Cilium Connectivity Tests"
122+
enabled: true
123+
124+
- script: |
125+
kubectl apply -f test/integration/manifests/cilium/$(cilium_version)/hubble/hubble-peer-svc.yaml
126+
kubectl get pods -Aowide
127+
echo "verify Hubble metrics endpoint is usable"
128+
go test ./test/integration/networkobservability -tags=networkobservability
129+
retryCountOnTaskFailure: 3
130+
name: "HubbleConnectivityTests"
131+
displayName: "Run Hubble Connectivity Tests"
132+
133+
- stage: deleteCluster
134+
condition: always()
135+
dependsOn:
136+
- createCluster
137+
- setupCluster
138+
jobs:
139+
- job: delete
140+
steps:
141+
- template: ../templates/delete-cluster.yaml
142+
parameters:
143+
name: cilium_overlay_e2e
144+
clusterName: $(clustername)
145+
region: $(REGION_AKS_CLUSTER_TEST)

.pipelines/npm/npm-cni-integration-test.yaml

+11-7
Original file line numberDiff line numberDiff line change
@@ -16,6 +16,7 @@ jobs:
1616
displayName: "NPM k8s E2E"
1717
dependsOn: ${{ parameters.dependsOn }}
1818
condition: and( and( not(canceled()), not(failed()) ), ${{ or(contains(parameters.os_version, '2022'), eq(parameters.os, 'linux') ) }} , or( contains(variables.CONTROL_SCENARIO, 'npm') , contains(variables.CONTROL_SCENARIO, 'all') ) )
19+
timeoutInMinutes: 180 # This is for testing windows, due to the 3m between the 14 tests -> results in 42m of wasted time
1920
pool:
2021
name: $(BUILD_POOL_NAME_DEFAULT)
2122
demands:
@@ -120,15 +121,17 @@ jobs:
120121
--ginkgo.focus="$focus" \
121122
--ginkgo.skip="NetworkPolicyLegacy|SCTP" \
122123
--kubeconfig=$HOME/.kube/config
123-
124-
# Untaint Linux (system) nodes once testing is complete
125-
if ${{ lower(eq(parameters.os, 'windows')) }}
126-
then
127-
kubectl taint nodes -l kubernetes.azure.com/mode=system node-role.kubernetes.io/control-plane:NoSchedule-
128-
fi
129124
displayName: "Run Kubernetes e2e.test"
130125
continueOnError: ${{ parameters.continueOnError }}
131126
127+
- ${{ if eq(parameters.os, 'windows') }}:
128+
- bash: |
129+
# Untaint Linux (system) nodes once testing is complete
130+
kubectl taint nodes -l kubernetes.azure.com/mode=system node-role.kubernetes.io/control-plane:NoSchedule-
131+
132+
displayName: Untaint Linux Nodes
133+
condition: always()
134+
132135
- bash: |
133136
npmLogs=$(System.DefaultWorkingDirectory)/${{ parameters.clusterName }}_npmLogs_Attempt_#$(System.StageAttempt)
134137
mkdir -p $npmLogs
@@ -137,9 +140,10 @@ jobs:
137140
npmPodList=`kubectl get pods -n kube-system | grep npm | awk '{print $1}'`
138141
# capture all logs
139142
for npmPod in $npmPodList; do
140-
kubectl logs -n kube-system $npmPod > $npmLogs/$npmPod-logs.txt
143+
kubectl logs -n kube-system $npmPod > $npmLogs/$npmPod-logs.txt
141144
done
142145
displayName: Generate NPM pod logs
146+
retryCountOnTaskFailure: 3
143147
condition: always()
144148
145149
- publish: $(System.DefaultWorkingDirectory)/${{ parameters.clusterName }}_npmLogs_Attempt_#$(System.StageAttempt)

.pipelines/singletenancy/cilium-overlay/cilium-overlay-e2e-step-template.yaml

+1-1
Original file line numberDiff line numberDiff line change
@@ -133,7 +133,7 @@ steps:
133133
kubectl rollout status ds cilium -n kube-system --timeout=3m
134134
kubectl get pods -Aowide
135135
echo "verify Hubble metrics endpoint is usable"
136-
go test ./test/integration/networkobservability -count=1 -v -tags=networkobservability
136+
go test ./test/integration/networkobservability -tags=networkobservability
137137
retryCountOnTaskFailure: 3
138138
name: "HubbleConnectivityTests"
139139
displayName: "Run Hubble Connectivity Tests"

Makefile

+2-2
Original file line numberDiff line numberDiff line change
@@ -785,9 +785,9 @@ test-integration: ## run all integration tests.
785785

786786
test-load: ## run all load tests
787787
AZURE_IPAM_VERSION=$(AZURE_IPAM_VERSION) \
788-
CNI_VERSION=$(CNI_VERSION)
788+
CNI_VERSION=$(CNI_VERSION)
789789
CNS_VERSION=$(CNS_VERSION) \
790-
go test -timeout 30m -race -tags=load ./test/integration/load...
790+
go test -timeout 30m -race -tags=load ./test/integration/load... -v
791791

792792
test-validate-state:
793793
cd test/integration/load && go test -mod=readonly -count=1 -timeout 30m -tags load -run ^TestValidateState

0 commit comments

Comments
 (0)