Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

chore: add Tiltfile for easier local development and testing #220

Merged
merged 1 commit into from
Jan 21, 2025
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 4 additions & 0 deletions .dockerignore
Original file line number Diff line number Diff line change
@@ -0,0 +1,4 @@
**/*
!pkg
!go.*
!cmd
6 changes: 5 additions & 1 deletion .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -5,4 +5,8 @@ flagger-k6-webhook
.vscode

# CI
.drone/temp.yml
.drone/temp.yml

# Tilt setup
.cache
dev-workload.yml
2 changes: 1 addition & 1 deletion Dockerfile
Original file line number Diff line number Diff line change
Expand Up @@ -3,7 +3,7 @@ FROM golang:1.23.4-alpine AS build
RUN mkdir /app
WORKDIR /app
COPY . /app/
RUN CGO_ENABLED=0 GOOS=linux go build -ldflags '-extldflags "-static"' -o /app/flagger-k6-webhook cmd/main.go
RUN --mount=type=cache,target=/go/pkg/mod CGO_ENABLED=0 GOOS=linux go build -ldflags '-extldflags "-static"' -o /app/flagger-k6-webhook cmd/main.go

FROM alpine:3.21

Expand Down
25 changes: 25 additions & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -100,3 +100,28 @@ By default, that limit is set to 1,000 *parallel* k6 processes which can be conf
If a new test request is received while the limit is reached, the request will be rejected with a HTTP 429 status.
The response also includes a `Retry-After` header that should be respected by the client.

## Local development

For local development we also ship a [Tiltfile](https://tilt.dev) so that, combined with [kind][] and [ctrptl][], you can test many scenarios locally:

```
# Prepare a demo host
sudo echo "127.0.0.1 demo.localhost" >> /etc/hosts

# Copy the provided dummy workload to the setup
cp dev-workload.dist.yml dev-workload.yml

# Create a kind cluster
ctlptl create cluster kind --registry=ctlptl-registry

# Start up tilt
tilt up
```

This will set up Traefik as gateway and export the port 8080.
After a short while you should see a standard nginx start page when curling `http://demo.local:8080`.

The easiest way to now test a canary is to update the nginx version in the dev workload.

[kind]: https://kind.sigs.k8s.io/
[ctlptl]: https://github.com/tilt-dev/ctlptl
92 changes: 92 additions & 0 deletions Tiltfile
Original file line number Diff line number Diff line change
@@ -0,0 +1,92 @@
if k8s_context() != 'kind-kind':
fail("failing early to avoid overwriting prod")


def name(res):
return res['metadata']['name']


def download_to_cache(url, filename):
full_path = os.path.join('.cache', filename)
if not os.path.exists(full_path):
local('curl --create-dirs --location --output %s %s' % (full_path, url))
return full_path


load('ext://secret', 'secret_from_dict')
load('ext://helm_resource', 'helm_resource', 'helm_repo')

helm_repo('helm-traefik', 'https://traefik.github.io/charts')
helm_repo('helm-flagger', 'https://flagger.app')

# Download the standard gateway CRD set
gateway_api = read_file(
download_to_cache(
'https://github.com/kubernetes-sigs/gateway-api/releases/download/v1.2.1/standard-install.yaml',
'gateway-crds.yaml'
)
)
k8s_yaml(gateway_api)

# Now let let's see all the CRDs defined here and bundle them up
crds = [res for res in decode_yaml_stream(gateway_api) if res['kind'] == 'CustomResourceDefinition']
k8s_resource(new_name='gateway-crds', objects=[name(crd) for crd in crds], labels='gateway')

local_resource(
'gateway-crds-ready',
cmd=' && '.join([('kubectl --context kind-kind wait --for=condition=Established crd %s' % name(c)) for c in crds]),
resource_deps=['gateway-crds'], labels='gateway')

# Once the CRDs are ready, we can also load rbac
gateway_rbac = read_file(
download_to_cache(
'https://raw.githubusercontent.com/traefik/traefik/v3.2/docs/content/reference/dynamic-configuration/kubernetes-gateway-rbac.yml',
'gateway-rbac.yml',
)
)
k8s_yaml(gateway_rbac)
k8s_resource(new_name='gateway-rbac', objects=[
name(res) for res in decode_yaml_stream(gateway_rbac)
], resource_deps=['gateway-crds-ready'], labels='gateway')

# Use traefik as service mesh and ingress
helm_resource('traefik', 'helm-traefik/traefik', flags=[
'--set=image.tag=v3.2.3',
'--set=providers.kubernetesGateway.enabled=true',
'--set=gateway.enabled=true',
'--set=gateway.listeners.web.namespacePolicy=null',
], resource_deps=['gateway-rbac'], port_forwards=['8000:8000'], labels='gateway')

helm_resource('flagger', 'helm-flagger/flagger', flags=[
'--set=prometheus.install=false',
'--set=meshProvider=gatewayapi:v1',
], resource_deps=['traefik'], labels='flagger')

# Now let's wait until the Canary CRD is ready
local_resource(
'flagger-crds-ready',
cmd='kubectl --context kind-kind wait --for=condition=Established crd canaries.flagger.app',
resource_deps=['flagger'], labels='flagger')


# Install local k6-loadtester
docker_build('ghcr.io/grafana/flagger-k6-webhook:development', '.')
k8s_yaml(secret_from_dict('k6-loadtester', inputs={
'KUBERNETES_CLIENT': 'in-cluster',
'K6_LOG_FORMAT': 'json',
'LOG_LEVEL': 'debug'
}))
yaml = helm('./charts/k6-loadtester', set=[
'webhook.vars.KUBERNETES_CLIENT=in-cluster',
'webhook.vars.K6_LOG_FORMAT=json',
'image.tag=development',
'logLevel=debug',
])
k8s_yaml(yaml)
k8s_resource('chart-k6-loadtester', labels='flagger')

# Now start a dev workload and let flagger create a route for it:
if os.path.exists('dev-workload.yml'):
k8s_yaml('dev-workload.yml')
k8s_resource('my-app', new_name='workload', objects=['my-app:Canary:default'], resource_deps=['flagger-crds-ready'], labels='workload')

4 changes: 2 additions & 2 deletions charts/k6-loadtester/templates/deployment.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -26,7 +26,7 @@ spec:
securityContext:
{{- toYaml .Values.podSecurityContext | nindent 8 }}
volumes:
{{- toYaml .Values.volumes | nindent 8 }}
{{- toYaml (concat .Values.volumes (list (dict "name" "tempdir" "emptyDir" (dict)))) | nindent 8 }}
initContainers:
{{ toYaml .Values.initContainers | nindent 8 }}
containers:
Expand Down Expand Up @@ -54,7 +54,7 @@ spec:
key: {{ $k | quote }}
{{- end }}
volumeMounts:
{{- toYaml .Values.volumeMounts | nindent 12 }}
{{- toYaml (concat .Values.volumeMounts (list (dict "mountPath" "/tmp" "name" "tempdir"))) | nindent 12 }}
{{- with .Values.nodeSelector }}
nodeSelector:
{{- toYaml . | nindent 8 }}
Expand Down
54 changes: 54 additions & 0 deletions dev-workload.dist.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,54 @@
---
apiVersion: apps/v1
kind: Deployment
metadata:
name: my-app
spec:
selector:
matchLabels:
app: my-app
template:
metadata:
labels:
app: my-app
spec:
containers:
- name: main
image: "nginx:1.27.1-alpine"
ports:
- containerPort: 80
---
apiVersion: flagger.app/v1beta1
kind: Canary
metadata:
name: my-app
spec:
analysis:
alerts: []
interval: 30s
iterations: 1
threshold: 2
webhooks:
- metadata:
notification_context: 'Cluster: `dev-cluster`'
script: |
import http from 'k6/http';
export default function () {
http.get('http://my-app-canary.default:80/');
}
name: k6-load-test
timeout: 5m
type: pre-rollout
url: http://chart-k6-loadtester.default:8000/launch-test
service:
name: my-app-svc
port: 80
portDiscovery: true
targetPort: 80
gatewayRefs:
- name: traefik-gateway
namespace: default
targetRef:
apiVersion: apps/v1
kind: Deployment
name: my-app
Loading