Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Adding support for specifying storageClass type #906

Merged
merged 14 commits into from
Feb 29, 2024

Conversation

nrajb
Copy link
Contributor

@nrajb nrajb commented Jan 11, 2024

Issue #, if available:
#896

Description of changes:
Adds support for specifying storageClass and deploys required resources as part of post cluster steps

By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.

@nrajb
Copy link
Contributor Author

nrajb commented Jan 11, 2024

output of lint, build and test:

npm i                                                                                                                                   [11/01/24| 1:16PM]
make build
make lint
make run-test
npx cdk list

up to date, audited 684 packages in 1s

81 packages are looking for funding
  run `npm fund` for details

found 0 vulnerabilities
rm -rf dist 
node node_modules/.bin/tsc
node node_modules/.bin/copyfiles "lib/**/*.yaml" "lib/**/*.ytpl" "dist/" -u 1 -V -E
unglobed path: lib/addons/adot/otel-permissions.yaml
unglobed path: lib/addons/xray/xray-ds.yaml
copy from: lib/addons/adot/otel-permissions.yaml
copy to: dist/addons/adot/otel-permissions.yaml
unglobed path: lib/addons/amp/collector-config-amp-daemonset.ytpl
unglobed path: lib/addons/amp/collector-config-amp.ytpl
unglobed path: lib/addons/cloudwatch-adot-addon/collector-config-cloudwatch.ytpl
unglobed path: lib/addons/vpc-cni/eniConfig.ytpl
unglobed path: lib/addons/xray-adot-addon/collector-config-xray.ytpl
unglobed path: lib/teams/aws-batch/aws-batch-rbac-config.ytpl
unglobed path: lib/teams/emr-containers-rbac-config.ytpl
copy from: lib/addons/xray/xray-ds.yaml
copy to: dist/addons/xray/xray-ds.yaml
copy from: lib/addons/amp/collector-config-amp-daemonset.ytpl
copy to: dist/addons/amp/collector-config-amp-daemonset.ytpl
copy from: lib/addons/amp/collector-config-amp.ytpl
copy to: dist/addons/amp/collector-config-amp.ytpl
copy from: lib/addons/cloudwatch-adot-addon/collector-config-cloudwatch.ytpl
copy to: dist/addons/cloudwatch-adot-addon/collector-config-cloudwatch.ytpl
copy from: lib/addons/vpc-cni/eniConfig.ytpl
copy to: dist/addons/vpc-cni/eniConfig.ytpl
copy from: lib/addons/xray-adot-addon/collector-config-xray.ytpl
copy to: dist/addons/xray-adot-addon/collector-config-xray.ytpl
copy from: lib/teams/aws-batch/aws-batch-rbac-config.ytpl
copy to: dist/teams/aws-batch/aws-batch-rbac-config.ytpl
copy from: lib/teams/emr-containers-rbac-config.ytpl
copy to: dist/teams/emr-containers-rbac-config.ytpl
node node_modules/.bin/eslint . --ext .js,.jsx,.ts,.tsx

/Users/abcd/Documents/LnD/cdk_contrib/cdk-eks-blueprints/lib/addons/ebs-csi-driver/index.ts
   69:13  warning  'patchSC' is assigned a value but never used            @typescript-eslint/no-unused-vars
  117:13  warning  'updateScConstruct' is assigned a value but never used  @typescript-eslint/no-unused-vars

✖ 2 problems (0 errors, 2 warnings)

npm test

> @aws-quickstart/[email protected] test
> jest --verbose false --detectOpenHandles

 PASS  test/stacks.test.ts (10.917 s)
  ● Console

    console.debug
      Blueprints tracking (qs-1s1r465hk)

      at Object.<anonymous> (test/stacks.test.ts:14:17)

    console.debug
      My awesome description Blueprints tracking (qs-1s1r465hk)

      at Object.<anonymous> (test/stacks.test.ts:23:17)

    console.debug
      Blueprints tracking (qs-1s1r465k6)

      at Object.<anonymous> (test/stacks.test.ts:139:17)

    console.debug
      Blueprints tracking (qs-1s1r465f2)

      at Object.<anonymous> (test/stacks.test.ts:189:17)

    console.debug
      Blueprints tracking (qs-1s1r465k6)

      at Object.<anonymous> (test/stacks.test.ts:245:17)

    console.log
      Blueprints tracking (qs-1s1r465k6)

      at Object.<anonymous> (test/stacks.test.ts:356:17)

    console.log
      Unable to get ECR repository for AWS Loadbalancer Controller for region undefined

      at lookupImage (lib/addons/aws-loadbalancer-controller/index.ts:72:17)

 PASS  test/clusterprovider.test.ts
 PASS  test/karpenter.test.ts
 PASS  test/resource-providers/resource-proxy.test.ts
  ● Console

    console.log
      proxy is true

      at Object.<anonymous> (test/resource-providers/resource-proxy.test.ts:20:17)

 PASS  test/knative-addon-validation.test.ts
 PASS  test/resource-providers/efs.test.ts
 PASS  test/resource-providers/kms-key.test.ts
 PASS  test/cluster-autoscaler.test.ts
 PASS  test/jupyterhub.test.ts
 PASS  test/backstage.test.ts
 PASS  test/resource-providers/s3.test.ts
 PASS  test/amp.test.ts
  ● Console

    console.log
      DEBUG Core add-on adot is at version v0.88.0-eksbuild.2

      at Object.transportFormatted (node_modules/tslog/dist/cjs/runtime/nodejs/index.js:116:13)

    console.log
      Used Adot Addon Version is Valid

      at AdotCollectorAddOn.deploy (lib/addons/adot/index.ts:46:21)

    console.log
      DEBUG Core add-on adot is at version v0.88.0-eksbuild.2

      at Object.transportFormatted (node_modules/tslog/dist/cjs/runtime/nodejs/index.js:116:13)

    console.log
      Used Adot Addon Version is Valid

      at AdotCollectorAddOn.deploy (lib/addons/adot/index.ts:46:21)

 PASS  test/constraints-validation.test.ts
  ● Console

    console.log
      DEBUG Core add-on vpc-cni is at version v1.14.1-eksbuild.1

      at Object.transportFormatted (node_modules/tslog/dist/cjs/runtime/nodejs/index.js:116:13)

 PASS  test/emr-eks.test.ts
  ● Console

    console.log
      DEBUG Core add-on vpc-cni is at version v1.14.1-eksbuild.1

      at Object.transportFormatted (node_modules/tslog/dist/cjs/runtime/nodejs/index.js:116:13)

    console.log
      DEBUG Core add-on coredns is at version v1.10.1-eksbuild.4

      at Object.transportFormatted (node_modules/tslog/dist/cjs/runtime/nodejs/index.js:116:13)

    console.log
      DEBUG Core add-on aws-ebs-csi-driver is at version v1.23.0-eksbuild.1

      at Object.transportFormatted (node_modules/tslog/dist/cjs/runtime/nodejs/index.js:116:13)

    console.log
      DEBUG Core add-on kube-proxy is at version auto

      at Object.transportFormatted (node_modules/tslog/dist/cjs/runtime/nodejs/index.js:116:13)

    console.log
      Unable to get ECR repository for AWS Loadbalancer Controller for region ${Token[AWS.Region.12]}

      at lookupImage (lib/addons/aws-loadbalancer-controller/index.ts:72:17)

    console.log
      DEBUG Core add-on kube-proxy has autoselected version v1.28.2-eksbuild.2

      at Object.transportFormatted (node_modules/tslog/dist/cjs/runtime/nodejs/index.js:116:13)


Test Suites: 14 passed, 14 total
Tests:       68 passed, 68 total
Snapshots:   0 total
Time:        28.767 s
Ran all test suites.
INFO Chart aws-load-balancer-controller-1.6.2 is at the latest version.
INFO Chart appmesh-controller-1.12.7 is at the latest version.
INFO Chart cert-manager-1.13.3 is at the latest version.
WARN Upgrade is needed for chart kube-state-metrics-5.15.2: latest version is 5.15.3.
WARN Upgrade is needed for chart prometheus-node-exporter-4.24.0: latest version is 4.25.0.
DEBUG Core add-on adot is at version v0.88.0-eksbuild.2
WARN Upgrade is needed for chart base-1.20.1: latest version is 1.20.2.
WARN Upgrade is needed for chart istiod-1.20.1: latest version is 1.20.2.
WARN Upgrade is needed for chart tigera-operator-v3.26.4: latest version is v3.27.0.
INFO Chart metrics-server-3.11.0 is at the latest version.
WARN Upgrade is needed for chart argo-cd-5.51.6: latest version is 5.52.1.
WARN Upgrade is needed for chart nginx-ingress-1.0.2: latest version is 1.1.0.
WARN Upgrade is needed for chart velero-3.2.0: latest version is 5.2.0.
DEBUG Core add-on vpc-cni is at version v1.14.1-eksbuild.1
DEBUG Core add-on coredns is at version v1.10.1-eksbuild.4
DEBUG Core add-on kube-proxy is at version auto
INFO Chart gatekeeper-3.14.0 is at the latest version.
No versions are found for karpenter in repository oci://public.ecr.aws/karpenter/karpenter
INFO Chart aws-node-termination-handler-0.21.0 is at the latest version.
INFO Chart kubevious-1.2.1 is at the latest version.
DEBUG Core add-on aws-ebs-csi-driver is at version v1.23.0-eksbuild.1
WARN Upgrade is needed for chart aws-efs-csi-driver-2.5.2: latest version is 2.5.3.
INFO Chart keda-2.12.1 is at the latest version.
WARN Upgrade is needed for chart aws-privateca-issuer-1.2.6: latest version is v1.2.7.
WARN Upgrade is needed for chart flux2-2.12.1: latest version is 2.12.2.
INFO Chart gpu-operator-v23.9.1 is at the latest version.
No versions are found for oci://ghcr.io/grafana/helm-charts/grafana-operator in repository undefined
INFO Chart aws-for-fluent-bit-0.1.32 is at the latest version.
INFO Chart airflow-1.11.0 is at the latest version.
WARN Upgrade is needed for chart external-secrets-0.9.9: latest version is 0.9.11.
DEBUG Core add-on eks-pod-identity-agent is at version v1.0.0-eksbuild.1
WARN Upgrade is needed for chart external-dns-1.13.1: latest version is 1.14.0.
Used Adot Addon Version is Valid
INFO Chart secrets-store-csi-driver-1.4.0 is at the latest version.
DEBUG Core add-on kube-proxy has autoselected version v1.28.2-eksbuild.2
blueprint-construct-dev

@elamaran11 elamaran11 linked an issue Jan 11, 2024 that may be closed by this pull request
1 task
@elamaran11
Copy link
Collaborator

@nrajb I see some GH Action warnings, can you fix those? Also can you update the documentation to emphasize on GP3 support.

Copy link
Collaborator

@shapirov103 shapirov103 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@nrajb thank you for the contribution, I believe the functionality is valuable.

The way the implementation is done, there a bunch of things that are incorrect atm. I added some minor feedback but let's address the design first.

Core requirements:

  1. The core ebs-csi addon must be deployed. We don't want the default csi driver to be there at all. Atm, I believe it is just the default csi driver that will be running.
  2. The return value of the addon must be the addon itself or a construct that depends on the addon.

Explanation:

  1. You never call super.deploy so the actual ebs csi addon does not get a chance to deploy. I believe your patches are applied to the default storage classes supplied with the cluster creation.
  2. You return arbitrary things from the deploy method. What we return from this method is used for dependencies. So if I have another dependent on your EBS CSI driver, the expectation is that the CfnAddOn is done, before another addon can install. In your case you return the patch or noop construct.
  3. The patch and the storage class creation manifest do not depend on anything. they can fire arbitrarily in any order. Expectation: the patch and SC creation happens AFTER the ebs-csi addon is installed. I expect to see this dependency.

These are the major points, happy to answer any questions.

lib/addons/ebs-csi-driver/index.ts Outdated Show resolved Hide resolved
lib/addons/ebs-csi-driver/index.ts Outdated Show resolved Hide resolved
lib/addons/ebs-csi-driver/index.ts Outdated Show resolved Hide resolved
Copy link
Collaborator

@shapirov103 shapirov103 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@nrajb looks way better, I added some tactical feedback mostly minor, one functional issue was identified, please address.

lib/addons/ebs-csi-driver/index.ts Outdated Show resolved Hide resolved
lib/addons/ebs-csi-driver/index.ts Show resolved Hide resolved
lib/addons/ebs-csi-driver/index.ts Outdated Show resolved Hide resolved
examples/blueprint-construct/index.ts Outdated Show resolved Hide resolved
@shapirov103
Copy link
Collaborator

@nrajb let's address the remaining feedback, it would be helpful to include this in 1.14 release.

Copy link
Collaborator

@shapirov103 shapirov103 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@nrajb looks great, however please check my comments, you probably inadvertently removed the existing versionMap functionality that we rely on.

Comment on lines +37 to +38
saName: "ebs-csi-controller-sa",
storageClass: "gp3", // Set the default StorageClass to gp3
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@nrajb you removed versionMap from the default props, it has impact on how this addon will behave during build time.
please restore, it will also take care of the warning above stating the versionMap is not used.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

corrected

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@nrjab please don't resolve conversation, it should be done by the reviewer, otherwise the comments are hidden.

super({
addOnName: defaultProps.addOnName,
version: options?.version ?? defaultProps.version,
saName: defaultProps.saName,
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

same here: versionMap is removed.
@elamaran11 FYI

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Corrected!

updateSc.node.addDependency(patchSc);

return updateSc;
} else
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

minor: else clause is redundant, since you are returning updateSc.

Copy link
Contributor Author

@nrajb nrajb Feb 23, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

So with the new implementation of async deploy(clusterInfo: ClusterInfo): Promise<Construct> in this proposed changes, if I don't handle return type without else I get following error for async deploy line:

Function lacks ending return statement and return type does not include 'undefined'.ts(2366)

Please suggest @shapirov103

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

weird, it thinks that base deployment can be undefined. nm, as i said it is minor.

@elamaran11
Copy link
Collaborator

@nrajb Can you make progress with this PR so we can move towards release. Also we see VersionMap is removed, can you merge from main.

@nrajb
Copy link
Contributor Author

nrajb commented Feb 23, 2024

Check Markdown links / markdown-link-check (pull_request) Failing after 1m

Unrelated doc errors:

 FILE: docs/addons/kubecost.md
  [✖] https://docs.kubecost.com/install-and-configure/install/custom-prom

  ERROR: 2 dead links found!
  [✖] https://docs.kubecost.com/kubecost-cloud/receiving-kubecost-cloud-support

Copy link
Collaborator

@shapirov103 shapirov103 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM pending e2e validation.

updateSc.node.addDependency(patchSc);

return updateSc;
} else
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

weird, it thinks that base deployment can be undefined. nm, as i said it is minor.

@shapirov103
Copy link
Collaborator

/do-e2e-tests

Copy link

@aws-ia-ci aws-ia-ci left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

end to end tests failed. A maintainer can provide more details.

@shapirov103
Copy link
Collaborator

/do-e2e-tests

Copy link

@aws-ia-ci aws-ia-ci left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

end to end tests passed

Copy link
Collaborator

@shapirov103 shapirov103 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@shapirov103 shapirov103 merged commit 1113f99 into aws-quickstart:main Feb 29, 2024
2 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

EBS-CSI addon should default to gp3
4 participants