Add flag to control CNI policy attachment

[azalenski-castai]

Some users prefer to assign the AmazonEKS_CNI_Policy on the aws-node DaemonSet level using IRSA, rather than assigning those permissions to the worker node IAM Role. More details about this approach can be found in the AWS doc: https://docs.aws.amazon.com/eks/latest/userguide/cni-iam-role.html

Therefore to allow users to have such possibility, I have added a flag attach_worker_cni_policy, that controls the assignment of this policy. Since the policies attachment is done via a for_each loop, I have moved the policies list to locals and added a conditional to add/remove the policy based on the flag value.

Below are some basic test results when using this new setup:

1. New cluster creation and flag not set. The AmazonEKS_CNI_Policy is being assigned:

Module config:
module "castai-eks-iam-role" {
  source = "../../../terraform-castai-eks-role-iam/"

  aws_account_id     = data.aws_caller_identity.current.account_id
  aws_cluster_vpc_id = module.vpc.vpc_id
  aws_cluster_region = local.region
  aws_cluster_name   = local.name

  castai_user_arn = castai_eks_user_arn.castai_user_arn.arn
}

Plan results:
  # module.castai-eks-iam-role.aws_iam_role_policy_attachment.castai_instance_profile_policy["arn:aws:iam::aws:policy/AmazonEKS_CNI_Policy"] will be created
  + resource "aws_iam_role_policy_attachment" "castai_instance_profile_policy" {
      + id         = (known after apply)
      + policy_arn = "arn:aws:iam::aws:policy/AmazonEKS_CNI_Policy"
      + role       = "castai-eks-instance-eks-2611-az"
    }
Plan: 93 to add, 0 to change, 0 to destroy.

2. New cluster creation and flag set to false:

Module config:
module "castai-eks-iam-role" {
  source = "../../../terraform-castai-eks-role-iam/"

  aws_account_id     = data.aws_caller_identity.current.account_id
  aws_cluster_vpc_id = module.vpc.vpc_id
  aws_cluster_region = local.region
  aws_cluster_name   = local.name

  castai_user_arn = castai_eks_user_arn.castai_user_arn.arn
  attach_worker_cni_policy = false
}

Plan results:
AmazonEKS_CNI_Policy policy attachment not present
Plan: 92 to add, 0 to change, 0 to destroy.

3. Existing cluster and flag not set:

Module config:
module "castai-eks-iam-role" {
  source = "../../../terraform-castai-eks-role-iam/"

  aws_account_id     = data.aws_caller_identity.current.account_id
  aws_cluster_vpc_id = module.vpc.vpc_id
  aws_cluster_region = local.region
  aws_cluster_name   = local.name

  castai_user_arn = castai_eks_user_arn.castai_user_arn.arn
}

Plan results:
No changes. Your infrastructure matches the configuration.

Terraform has compared your real infrastructure against your configuration and found no differences, so no changes are needed.

Apply complete! Resources: 0 added, 0 changed, 0 destroyed.

4. Existing cluster and flag set to false:

Module config:
module "castai-eks-iam-role" {
  source = "../../../terraform-castai-eks-role-iam/"

  aws_account_id     = data.aws_caller_identity.current.account_id
  aws_cluster_vpc_id = module.vpc.vpc_id
  aws_cluster_region = local.region
  aws_cluster_name   = local.name

  castai_user_arn = castai_eks_user_arn.castai_user_arn.arn
  attach_worker_cni_policy = false
}

Plan results:
Terraform used the selected providers to generate the following execution plan. Resource actions are indicated with the following symbols:
  - destroy

Terraform will perform the following actions:

  # module.castai-eks-iam-role.aws_iam_role_policy_attachment.castai_instance_profile_policy["arn:aws:iam::aws:policy/AmazonEKS_CNI_Policy"] will be destroyed
  # (because key ["arn:aws:iam::aws:policy/AmazonEKS_CNI_Policy"] is not in for_each map)
  - resource "aws_iam_role_policy_attachment" "castai_instance_profile_policy" {
      - id         = "castai-eks-instance-eks-2611-az-20241127112108838500000009" -> null
      - policy_arn = "arn:aws:iam::aws:policy/AmazonEKS_CNI_Policy" -> null
      - role       = "castai-eks-instance-eks-2611-az" -> null
    }

Plan: 0 to add, 0 to change, 1 to destroy.

I have also adjusted the Readme example module name to match the actual module name we use in all of our code and examples.