Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Rotation may happen on new instances #153

Closed
eytan-avisror opened this issue Dec 3, 2020 · 1 comment · Fixed by #155
Closed

Rotation may happen on new instances #153

eytan-avisror opened this issue Dec 3, 2020 · 1 comment · Fixed by #155
Labels
bug Something isn't working

Comments

@eytan-avisror
Copy link
Contributor

I noticed that when I rotate ASG nodes that were using launch configuration, to new nodes using launch templates, after all the nodes are rotated to new ones, few of the new ones also get rotated.
This seems to be related to some caching issue, because after few new nodes get rotated (needlessly), it finally gets marked as completed.

In the logs, the reason for rotation seems to be

launch template version differs	{"rollingupgrade": "rolling-upgrade-nodes-20201201234551-2"}

In this case, the instance was rotated from version 2 of the launch template to version 2.

controllers.RollingUpgrade	selected instances for rotation: [{
  AvailabilityZone: "us-west-2a",
  HealthStatus: "Healthy",
  InstanceId: "i-06c75c47f7166fcb1",
  InstanceType: "m5.large",
  LaunchTemplate: {
    LaunchTemplateId: "lt-093c21b6d6395defd",
    LaunchTemplateName: "eibissror-arktika-instance-manager-nodes-20201201234551",
    Version: "2"
  },
  LifecycleState: "InService",
  ProtectedFromScaleIn: false,
  WeightedCapacity: "1"
}]
@eytan-avisror eytan-avisror added the bug Something isn't working label Dec 3, 2020
@eytan-avisror
Copy link
Contributor Author

After adding some debug logs, it seems we are not resolving the latest template hence comparing a digit to $Latest

launch template version differs	{"rollingupgrade": "rolling-upgrade-nodes-20201207212022-1", "instanceVersion": "1", "templateVersion": "$Latest"}

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working
Projects
None yet
Development

Successfully merging a pull request may close this issue.

1 participant