Instance override issue when launching a worker group launch template per az

[alexganwd]

Launch templates with mixed policies fail when re-using instance type .

I'm submitting a

• bug report
• feature request
• support request
• kudos, thank you, warm fuzzy

What is the current behavior?

I'm trying to create two worker_groups with launch templates one per availability zone. The groups are created and I can see them in the console, however this TF module is reporting an issue when overriding:

• aws_autoscaling_group.workers_launch_template.1: Error creating AutoScaling Group: ValidationError: Cannot add same instance type override more than once. Remove these duplicates from the request and try again: [t3.large]
  status code: 400, request id: xxxx

• module.eks.module.eks.aws_autoscaling_group.workers_launch_template[0]: 1 error(s) occurred:

• aws_autoscaling_group.workers_launch_template.0: Error creating AutoScaling Group: ValidationError: Cannot add same instance type override more than once. Remove these duplicates from the request and try again: [t3.large]
  status code: 400, request id: yyy

If this is a bug, how to reproduce? Please include a code sample if relevant.

I'm using the following configuration to test spot fleet creation:
worker_group_launch_template_count = "2"
worker_groups_launch_template = [
{
spot_instance_pools = 3
asg_desired_capacity = 2
on_demand_base_capacity = 0
on_demand_percentage_above_base_capacity = 0
autoscaling_enabled = 1
key_name = "${var.key_name}"
instance_type = "t3.large"
name = "t3large-1a"
kubelet_extra_args = "--node-labels=spot=true --node-labels=role=t3large-1,env=k8s-env,stack=k8s-env,az=a,node-role.kubernetes.io/t3large-1=true,node-role.kubernetes.io/t3large-1a=true"
subnets = "${data.aws_subnet.private_subnet_a.id}"
},
{
spot_instance_pools = 3
asg_desired_capacity = 2
on_demand_base_capacity = 0
on_demand_percentage_above_base_capacity = 0
autoscaling_enabled = 1
key_name = "${var.key_name}"
instance_type = "t3.large"
name = "t3large-1b"
kubelet_extra_args = "--node-labels=spot=true --node-labels=role=t3large-1,env=k8s-env,stack=k8s-env,az=b,node-role.kubernetes.io/t3large-1=true,node-role.kubernetes.io/t3large-1b=true" subnets = "${data.aws_subnet.private_subnet_b.id}"
}
]

What's the expected behavior?

Have two templates with same instance size, one for az=a and one for az=b

Are you able to fix this problem and submit a PR? Link here if you have already.

Environment details

• Affected module version: 3.0.0
• OS:
• Terraform version: v0.11.13

Any other relevant info

[max-rocket-internet]

The problem is here:
https://github.com/terraform-aws-modules/terraform-aws-eks/blob/master/local.tf#L55
https://github.com/terraform-aws-modules/terraform-aws-eks/blob/master/workers_launch_template.tf#L33

Currently the code applies 2 instance overrides to the ASG, the second of which defaults to t3.large.

So for you, you would need to choose a second instance type as instance_type and you could end up with a mix of both instance types in the ASG. There's no other way unfortunately. You can't set either to "" as terraform will apply on every run: hashicorp/terraform-provider-aws#8475

[max-rocket-internet]

I think the changes we should make in this module are:

1. Remove the overrides and just use instance type from LT.
2. Provide a second worker_group option with 5 instance type overrides for people who are using spot instances.

But would be nice to hear other people's opinions 🙂

[alexganwd]

I got it running as I wanted, I only had to remove instance_type override from my config and that defaults to t3.large.

However, I kind of see this as a bug. That second override in the code should not happen if instance_type override matches local.

[max-rocket-internet]

However, I kind of see this as a bug. That second override in the code should not happen if instance_type override matches local.

I don't think this is possible in Terraform currently because it requires conditionally setting an override block on the ASG.

[max-rocket-internet]

Should all be resolved in latest release 🙂

[github-actions]

I'm going to lock this issue because it has been closed for 30 days ⏳. This helps our maintainers find and focus on the active issues. If you have found a problem that seems similar to this, please open a new issue and complete the issue template so we can capture all the details necessary to investigate further.