Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

aws-stepfunctions-tasks: Emr cluster is not terminated automatically when manually abort step function, or time out #28697

Open
1 of 2 tasks
ziruiLiu-g opened this issue Jan 12, 2024 · 1 comment
Open
1 of 2 tasks

aws-stepfunctions-tasks: Emr cluster is not terminated automatically when manually abort step function, or time out #28697

ziruiLiu-g opened this issue Jan 12, 2024 · 1 comment
Labels
@aws-cdk/aws-stepfunctions-tasks effort/medium Medium work item – several days of effort feature-request A feature should be added or improved. p3

Comments

@ziruiLiu-g
Copy link

Describe the feature

Support KeepJobFlowAliveWhenNoSteps Or Auto-termination (after idle) in stepfunction creates EMR cluster.

Use Case

Our team is using Stepfunction EMR (EmrCreateCluster function) to build our ETL pipeline.

While the emr cluster are working very well on creating, add steps and the cluster is terminated properly when stepfunction is finished or failed, it remains "waiting" status when we manually abort the step function or the step function timeout.

Wondering if we can export the KeepJobFlowAliveWhenNoSteps as an optional parameter when create the Cluster, or have an option to configure auto-termination after idle time.

Proposed Solution

No response

Other Information

No response

Acknowledgements

  • I may be able to implement this feature request
  • This feature might incur a breaking change

CDK version used

2.14.0

Environment details (OS name and version, etc.)

N/A
@ziruiLiu-g ziruiLiu-g added feature-request A feature should be added or improved. needs-triage This issue or PR still needs to be triaged. labels Jan 12, 2024
@pahud
Copy link
Contributor

pahud commented Jan 16, 2024

Thank you for your PR.

@pahud pahud added p2 effort/medium Medium work item – several days of effort and removed needs-triage This issue or PR still needs to be triaged. labels Jan 16, 2024
@github-actions github-actions bot mentioned this issue Feb 1, 2024
Closed
@pahud pahud added p3 and removed p2 labels Jun 11, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
@aws-cdk/aws-stepfunctions-tasks effort/medium Medium work item – several days of effort feature-request A feature should be added or improved. p3
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
2 participants
@pahud @ziruiLiu-g