Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

troubles with previewReplicaCount in BlueGreen rollout strategy #301

Closed
vranystepan opened this issue Nov 20, 2019 · 3 comments
Closed

troubles with previewReplicaCount in BlueGreen rollout strategy #301

vranystepan opened this issue Nov 20, 2019 · 3 comments
Labels
blue-green Blue-Green related issue bug Something isn't working

Comments

@vranystepan
Copy link

Hi folks,

When I'm experimenting with previewReplicaCount, I'm getting to the situation when it's creating and killing replicas in the preview RS:

⟳ rollout-bluegreen                            Rollout     ॥ Paused       35s  
├──# revision:2                                                                
│  └──⧉ rollout-bluegreen-5f49884f5c           ReplicaSet  ◌ Progressing  8s   preview
│     ├──□ rollout-bluegreen-5f49884f5c-qpvxn  Pod         ✔ Running      8s   ready:1/1
│     ├──□ rollout-bluegreen-5f49884f5c-z2snd  Pod         ✔ Running      8s   ready:1/1
│     ├──□ rollout-bluegreen-5f49884f5c-4b6vt  Pod         ◌ Terminating  0s   ready:0/1
│     ├──□ rollout-bluegreen-5f49884f5c-69zzs  Pod         ◌ Terminating  0s   ready:0/1
│     ├──□ rollout-bluegreen-5f49884f5c-6k69q  Pod         ◌ Terminating  0s   ready:0/1
│     ├──□ rollout-bluegreen-5f49884f5c-cdhmz  Pod         ◌ Terminating  0s   ready:0/1
│     ├──□ rollout-bluegreen-5f49884f5c-dk2kk  Pod         ◌ Terminating  0s   ready:0/1
│     ├──□ rollout-bluegreen-5f49884f5c-rxt9k  Pod         ◌ Terminating  0s   ready:0/1
│     ├──□ rollout-bluegreen-5f49884f5c-wvgqr  Pod         ◌ Terminating  0s   ready:0/1
│     └──□ rollout-bluegreen-5f49884f5c-x256k  Pod         ◌ Terminating  0s   ready:0/1
└──# revision:1                                                                
   └──⧉ rollout-bluegreen-7d6b6cb796           ReplicaSet  ✔ Healthy      34s  active
      ├──□ rollout-bluegreen-7d6b6cb796-2wkjr  Pod         ✔ Running      34s  ready:1/1
      ├──□ rollout-bluegreen-7d6b6cb796-5svlv  Pod         ✔ Running      34s  ready:1/1
      ├──□ rollout-bluegreen-7d6b6cb796-7lmcx  Pod         ✔ Running      34s  ready:1/1
      ├──□ rollout-bluegreen-7d6b6cb796-hpqj5  Pod         ✔ Running      34s  ready:1/1
      ├──□ rollout-bluegreen-7d6b6cb796-llm7r  Pod         ✔ Running      34s  ready:1/1
      ├──□ rollout-bluegreen-7d6b6cb796-mnzk4  Pod         ✔ Running      34s  ready:1/1
      ├──□ rollout-bluegreen-7d6b6cb796-pss79  Pod         ✔ Running      34s  ready:1/1
      ├──□ rollout-bluegreen-7d6b6cb796-qjjjf  Pod         ✔ Running      34s  ready:1/1
      ├──□ rollout-bluegreen-7d6b6cb796-rbwf9  Pod         ✔ Running      34s  ready:1/1
      └──□ rollout-bluegreen-7d6b6cb796-w2s99  Pod         ✔ Running      34s  ready:1/1

This is the manifest I'm using:

apiVersion: argoproj.io/v1alpha1
kind: Rollout
metadata:
  name: rollout-bluegreen
spec:
  replicas: 10
  revisionHistoryLimit: 2
  selector:
    matchLabels:
      app: rollout-bluegreen
  template:
    metadata:
      labels:
        app: rollout-bluegreen
    spec:
      containers:
      - name: rollouts-demo
        image: argoproj/rollouts-demo:blue
        imagePullPolicy: Always
        ports:
        - containerPort: 8080
  strategy:
    blueGreen: 
      activeService: rollout-bluegreen-active
      previewService: rollout-bluegreen-preview
      autoPromotionEnabled: false
      previewReplicaCount: 2

---
kind: Service
apiVersion: v1
metadata:
  name: rollout-bluegreen-active
spec:
  selector:
    app: rollout-bluegreen
  ports:
  - protocol: TCP
    port: 80
    targetPort: 8080

---
kind: Service
apiVersion: v1
metadata:
  name: rollout-bluegreen-preview
spec:
  selector:
    app: rollout-bluegreen
  ports:
  - protocol: TCP
    port: 80
    targetPort: 8080

Rollouts version:

image: argoproj/argo-rollouts:v0.6.0

Please, do you have any idea what's happening there? Am I doing something wrong?
Many thanks in advance!

@dthomson25
Copy link
Member

Hi @vranystepan, it looks like the controller is getting stuck in a loop of scaling the preview ReplicaSet up and then down instead of just honoring the previewReplicaCount value. I can reproduce the issue locally and I will start working on a fix. Thanks for pointing this out!

@dthomson25 dthomson25 added bug Something isn't working blue-green Blue-Green related issue labels Nov 26, 2019
@vranystepan
Copy link
Author

Great! Thank you!

@dthomson25
Copy link
Member

Fixed by #308

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
blue-green Blue-Green related issue bug Something isn't working
Projects
None yet
Development

No branches or pull requests

3 participants