Allow Operator Generated bootstrap token #14437

apollo13 · 2022-09-01T11:10:49Z

Description

Added support to provide an initial token via the bootstrap HTTP API, similar to hashicorp/nomad#12520

PR Checklist

updated test coverage
external facing docs updated
not a security concern

boruszak

I reviewed 14437.txt and bootstrap.mdx on behalf of the Docs team.

Great work - I made suggestions for style and additional clarity. Please let me know if you have any questions or need any additional assistance.

Approving on behalf of consul-docs.

.changelog/14437.txt

website/content/commands/acl/bootstrap.mdx

jkirschner-hashicorp · 2022-09-01T15:25:04Z

Hello @apollo13,

Thank you for submitting this PR for our consideration! To help with that consideration, can you provide more context on:

The need that this PR is intended to address (since there is no linked Github issue)?
The ways in which this PR addresses said need in a way that is more suitable for your use case than the existing Consul server agent configuration option acl.tokens.initial_management?
- As an aside, if you weren't previously aware of the acl.tokens.initial_management option, that suggests an opportunity for us to improve the ACL bootstrapping docs

apollo13 · 2022-09-01T15:53:31Z

Hi @jkirschner-hashicorp,

Thank you for submitting this PR for our consideration! To help with that consideration, can you provide more context on:

Sure, thank you for answering.

The need that this PR is intended to address (since there is no linked Github issue)?

I want to be able to programmatically bootstrap a Consul cluster. If nothing else, the hashicorp products should preferably behave the same way and this makes it nicely match nomad.

The ways in which this PR addresses said need in a way that is more suitable for your use case than the existing Consul server agent configuration option acl.tokens.initial_management?

I am aware of the initial management token but I rather not write (even if it is just an initial token) secrets like that onto the harddisk of a server if not needed. I understand that the raft/storage-layer will write the token anyways, but that storage layer is ecxpected to hold sensitive data (connect key material among others and in the worst case kv data is sensitive as well).

To make matters worse I am trying to precreate usable containers/vm images. The less key material I have to roll out the better. Initial server bootstrap works nicely via mdns cloud-join (got that working for nomad & consul). I understand that I will probably have to put other tokens into the config as well, but the fallout of those other tokens would be far less than the root key…

I hope this helps; I realize I am probably not making the best arguments for this feature -- it is not an absolute must-have (I understand that), but nevertheless it would make things easier for a few people out here.

jkirschner-hashicorp · 2022-09-01T16:05:32Z

Thank you for the context! I'll share this with the team as a next step before engineering review.

github-actions · 2022-11-01T01:36:51Z

This pull request has been automatically flagged for inactivity because it has not been acted upon in the last 60 days. It will be closed if no new activity occurs in the next 30 days. Please feel free to re-open to resurrect the change if you feel this has happened by mistake. Thank you for your contributions.

apollo13 · 2022-11-01T06:41:01Z

//Unstable

…

On Tue, Nov 1, 2022, at 02:37, github-actions[bot] wrote: This pull request has been automatically flagged for inactivity because it has not been acted upon in the last 60 days. It will be closed if no new activity occurs in the next 30 days. Please feel free to re-open to resurrect the change if you feel this has happened by mistake. Thank you for your contributions. — Reply to this email directly, view it on GitHub <#14437 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AAAT5C5ODTPHX2XSUO63VCDWGBX35ANCNFSM6AAAAAAQCHZ7ZM>. You are receiving this because you were mentioned.Message ID: ***@***.***>

rboyer

Great job on this! I have a variety of small comments and suggestions I'm adding below but none of them should be surprising or tricky to clean up before merging.

One of the longer comments is not actionable and is pointing out a msgpack non-issue to a future PR archaeologist.

agent/acl_endpoint.go

rboyer · 2022-11-09T20:26:37Z

agent/consul/acl_endpoint.go

+		if err != nil {
+			return err
+		}
+		ok, err := a.srv.checkTokenUUID(secret)


This deserves a comment. On FIRST bootstrap this doesn't matter because there are no tokens. On rebootstrap however there may already be tokens and you don't want to collide with them. If we didn't allow re-bootstrap this function call would be unnecessary.

checkTokenUUID also checks that the token does not start with ACLReservedPrefix, I'd like to at least keep that check. Should I switch to using ACLIDReserved(secret)?

It's fine to leave it as is because it is important to check against the existing database during the reset procedure: https://developer.hashicorp.com/consul/tutorials/security/access-control-troubleshoot#reset-the-acl-system

agent/consul/acl_endpoint.go

website/content/api-docs/acl/index.mdx

rboyer · 2022-11-09T21:35:36Z

agent/acl_endpoint.go

@@ -34,9 +34,20 @@ func (s *HTTPHandlers) ACLBootstrap(resp http.ResponseWriter, req *http.Request)
 		return nil, aclDisabled
 	}

-	args := structs.DCSpecificRequest{
+	args := structs.ACLInitialTokenBootstrapRequest{


For the sake of anyone else reading this PR, note that here we are switching the request type completely from one struct to another, and that is being encoded using msgpack and sent over RPC.

The new struct type ACLInitialTokenBootstrapRequest only contains a subset of the fields that exist on DCSpecificRequest so a surface read would say that this is backwards incompatible by msgpack encoding rules and would be guaranteed to lose data if you were using a different version of consul on both sides of this RPC (as you would during windows of time during an upgrade).

A deeper read shows that actually this generic request struct is used in many places, but is largely unused within the acl bootstrap codepath and all of the relevant fields were already copied to your new type with the same name+types, so it would be effectively backwards compatible.

Is this a line of code that a future dev might look at and go "wait, this looks like it might be wrong..."? If so, we might want an in-source comment. I defer to @rboyer

I think in-source might not be correct because it only matters for version-straddling. If it would go anywhere I'd say it would make sense in the body of the git commit so it's historically annotating the change as "yep, this was fine from a msgpack perspective at the time we did it".

api/acl.go

rboyer · 2022-11-09T21:53:54Z

agent/structs/acl.go

@@ -1350,10 +1350,20 @@ type ACLTokenBatchDeleteRequest struct {
 	TokenIDs []string // Tokens to delete
 }

+type ACLInitialTokenBootstrapRequest struct {
+	BootstrapSecret string


How do you feel about calling it InitialToken or InitialManagementToken instead? The config file approach for bootstrap refers to the same concept as initial_management so this would align them.

(whatever happens here it should also be mirrored to the matching api struct)

I don't have strong feelings; I mirrored nomad: https://github.com/hashicorp/nomad/blob/fbe9f590489e291b3abcfb0fb1548a9f435a5e2d/nomad/structs/structs.go#L12169 -- I'd be fine with changing it if you think another name is better.

command/acl/bootstrap/bootstrap.go

command/acl/bootstrap/bootstrap_test.go

apollo13 · 2022-11-11T09:30:40Z

Thank you for the reviews. I will need a few days to get this updated.

apollo13 · 2023-01-04T12:15:58Z

@kisunji Any chance of another review?

kisunji

LGTM; I think @rboyer's remaining comments were non-blocking but will ping for final review

Add support to provide an initial token via the bootstrap HTTP API, similar to hashicorp/nomad#12520

apollo13 requested a review from a team as a code owner September 1, 2022 11:10

github-actions bot added theme/api Relating to the HTTP API interface theme/cli Flags and documentation for the CLI interface type/docs Documentation needs to be created/updated/clarified labels Sep 1, 2022

apollo13 force-pushed the bootstrap-token branch from 26de4d3 to 6925845 Compare September 1, 2022 11:11

boruszak approved these changes Sep 1, 2022

View reviewed changes

.changelog/14437.txt Outdated Show resolved Hide resolved

website/content/commands/acl/bootstrap.mdx Outdated Show resolved Hide resolved

website/content/commands/acl/bootstrap.mdx Outdated Show resolved Hide resolved

website/content/commands/acl/bootstrap.mdx Outdated Show resolved Hide resolved

apollo13 force-pushed the bootstrap-token branch from 4217013 to 022f47f Compare September 1, 2022 15:10

jkirschner-hashicorp added theme/acls ACL and token generation theme/operator-usability Replaces UX. Anything related to making things easier for the practitioner labels Sep 1, 2022

jkirschner-hashicorp requested a review from a team September 1, 2022 16:13

github-actions bot added the meta/stale Automatically flagged for inactivity by stalebot label Nov 1, 2022

jkirschner-hashicorp added meta/staleproof Exempt from stalebot automation and removed meta/stale Automatically flagged for inactivity by stalebot labels Nov 1, 2022

rboyer requested changes Nov 9, 2022

View reviewed changes

rboyer added the waiting-reply Waiting on response from Original Poster or another individual in the thread label Nov 9, 2022

kisunji reviewed Nov 10, 2022

View reviewed changes

command/acl/bootstrap/bootstrap_test.go Outdated Show resolved Hide resolved

kisunji reviewed Nov 10, 2022

View reviewed changes

command/acl/bootstrap/bootstrap_test.go Outdated Show resolved Hide resolved

kisunji reviewed Nov 10, 2022

View reviewed changes

command/acl/bootstrap/bootstrap_test.go Outdated Show resolved Hide resolved

github-actions bot removed the waiting-reply Waiting on response from Original Poster or another individual in the thread label Nov 11, 2022

apollo13 requested a review from rboyer November 12, 2022 19:41

apollo13 force-pushed the bootstrap-token branch 2 times, most recently from 3366b91 to 00bfc9d Compare November 12, 2022 19:56

kisunji approved these changes Jan 4, 2023

View reviewed changes

rboyer approved these changes Jan 4, 2023

View reviewed changes

apollo13 added 2 commits January 4, 2023 15:05

Allow Operator Generated bootstrap token

5194939

Review comments.

9ec20e5

kisunji force-pushed the bootstrap-token branch from 00bfc9d to 9ec20e5 Compare January 4, 2023 20:05

kisunji enabled auto-merge (squash) January 4, 2023 20:05

kisunji merged commit 077b0a4 into hashicorp:main Jan 4, 2023

kisunji pushed a commit that referenced this pull request Jan 5, 2023

Allow Operator Generated bootstrap token (#14437)

9545eec

Add support to provide an initial token via the bootstrap HTTP API, similar to hashicorp/nomad#12520

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Allow Operator Generated bootstrap token #14437

Allow Operator Generated bootstrap token #14437

apollo13 commented Sep 1, 2022

boruszak left a comment

jkirschner-hashicorp commented Sep 1, 2022

apollo13 commented Sep 1, 2022

jkirschner-hashicorp commented Sep 1, 2022 •

edited

Loading

github-actions bot commented Nov 1, 2022

apollo13 commented Nov 1, 2022 via email

rboyer left a comment

rboyer Nov 9, 2022

apollo13 Nov 12, 2022 •

edited

Loading

rboyer Jan 4, 2023

rboyer Nov 9, 2022

jkirschner-hashicorp Nov 9, 2022

rboyer Nov 9, 2022

rboyer Nov 9, 2022

apollo13 Nov 12, 2022

apollo13 commented Nov 11, 2022

apollo13 commented Jan 4, 2023

kisunji left a comment

Allow Operator Generated bootstrap token #14437

Allow Operator Generated bootstrap token #14437

Conversation

apollo13 commented Sep 1, 2022

Description

PR Checklist

boruszak left a comment

Choose a reason for hiding this comment

jkirschner-hashicorp commented Sep 1, 2022

apollo13 commented Sep 1, 2022

jkirschner-hashicorp commented Sep 1, 2022 • edited Loading

github-actions bot commented Nov 1, 2022

apollo13 commented Nov 1, 2022 via email

rboyer left a comment

Choose a reason for hiding this comment

rboyer Nov 9, 2022

Choose a reason for hiding this comment

apollo13 Nov 12, 2022 • edited Loading

Choose a reason for hiding this comment

rboyer Jan 4, 2023

Choose a reason for hiding this comment

rboyer Nov 9, 2022

Choose a reason for hiding this comment

jkirschner-hashicorp Nov 9, 2022

Choose a reason for hiding this comment

rboyer Nov 9, 2022

Choose a reason for hiding this comment

rboyer Nov 9, 2022

Choose a reason for hiding this comment

apollo13 Nov 12, 2022

Choose a reason for hiding this comment

apollo13 commented Nov 11, 2022

apollo13 commented Jan 4, 2023

kisunji left a comment

Choose a reason for hiding this comment

jkirschner-hashicorp commented Sep 1, 2022 •

edited

Loading

apollo13 Nov 12, 2022 •

edited

Loading