Skip to content
This repository has been archived by the owner on Aug 25, 2021. It is now read-only.

Apply license until success #348

Merged
merged 1 commit into from
Mar 2, 2020
Merged

Apply license until success #348

merged 1 commit into from
Mar 2, 2020

Conversation

lkysow
Copy link
Member

@lkysow lkysow commented Feb 8, 2020

Previously, if the leader wasn't elected, the consul license command
would be run once in each execution of the job and fail.
On clusters where the servers took a long time to come up,
the job would reach its maximum backoff limit and the license would
never be applied.

This change runs the license put command in a loop until it's
successful or it times out after 20 minutes.

Logs look like:

Applying license...
Error putting license: Put http://consul-server:8500/v1/operator/license: dial tcp: lookup consul-server on 10.35.240.10:53: no such host
Retrying in 2s...
Applying license...
Error putting license: Put http://consul-server:8500/v1/operator/license: dial tcp: lookup consul-server on 10.35.240.10:53: no such host
Retrying in 2s...
Applying license...
Retrying in 2s...
Error putting license: Put http://consul-server:8500/v1/operator/license: dial tcp: lookup consul-server on 10.35.240.10:53: no such host
Applying license...
Error putting license: Unexpected response code: 500 (No cluster leader)
Retrying in 2s...
Applying license...
License is valid
License ID: *************
Customer ID: *****************
Expires At: 2021-01-02 05:59:59.999 +0000 UTC
Terminates At: 2021-01-02 05:59:59.999 +0000 UTC
Datacenter: *
Package: premium
Licensed Features:
	Automated Backups
	Automated Upgrades
	Enhanced Read Scalability
	Network Segments
	Redundancy Zone
	Advanced Network Federation
	Namespaces

@lkysow lkysow added the bug Something isn't working label Feb 8, 2020
@lkysow lkysow requested a review from a team February 8, 2020 00:15
@lkysow lkysow force-pushed the ent-license-job-wait branch from 7444b1b to 0db9e52 Compare February 10, 2020 04:19
Copy link
Contributor

@ishustava ishustava left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@lkysow Thank you for fixing this 💯 I added a couple of thoughts inline.

I think this works as a short-term solution, but I thought we were talking about adding a command to consul-k8s to wait for servers to be ready and then re-using it here and for acl-init. I might be misremembering though.

templates/enterprise-license-job.yaml Outdated Show resolved Hide resolved
templates/enterprise-license-job.yaml Outdated Show resolved Hide resolved
Previously, if the leader wasn't elected, the consul license command
would be run once in each execution of the job and fail.
On clusters where the servers took a long time to come up,
the job would reach its maximum backoff limit and the license would
never be applied.

This change runs the license put command in a loop until it's
successful or it times out after 20 minutes.
@lkysow lkysow force-pushed the ent-license-job-wait branch from 0db9e52 to 83acf1e Compare February 17, 2020 17:16
@lkysow
Copy link
Member Author

lkysow commented Feb 17, 2020

@ishustava good call on just running the command in a loop. That's way better. I've modified my changes accordingly and am using the timeout command to ensure we time out after 20 minutes. In my testing, AKS disks could take up to 15 minutes to install.

@lkysow lkysow changed the title Wait until leader before applying license Apply license until success Feb 17, 2020
@lkysow lkysow requested a review from ishustava February 17, 2020 17:19
Copy link
Contributor

@ishustava ishustava left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks great! Thanks for refactoring it!

@lkysow lkysow merged commit 24c7a71 into master Mar 2, 2020
@lkysow lkysow deleted the ent-license-job-wait branch March 2, 2020 22:11
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
bug Something isn't working
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants