Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Support environment configuration #1849

Open
wants to merge 3 commits into
base: master
Choose a base branch
from
Open

Conversation

cretz
Copy link
Member

@cretz cretz commented Feb 25, 2025

What was changed

  • Added go.temporal.io/sdk/contrib/envconfig module that supports loading
    • Supports conversion to/from toml
    • Supports file loading and env var overrides
    • Supports loading client options

See temporalio/features#441, this is mostly based on that proposal with some slight changes.

This SDK may support a few more shortcuts than other SDKs due to its use in CLI. Documentation about file format and which env vars do what will be made available on docs.temporal.io before this launches GA.

Checklist

  1. Closes External client configuration #1815

//
// WARNING: Environment configuration is currently experimental.
type ClientConfig struct {
Profiles map[string]*ClientConfigProfile
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Worth saying what the key is here.

// ClientConfigCodec is codec configuration for a client.
//
// WARNING: Environment configuration is currently experimental.
type ClientConfigCodec struct {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Maybe call this ClientConfigRemoteCodec? What, if anything, could ever be configured here for local codec? And, even then, maybe it'd make sense to have them be different sub-configs

Copy link
Member Author

@cretz cretz Feb 25, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It is the codec field in config and TEMPORAL_CODEC_ prefixed env vars and Codec field name in the structure in addition to this type name. So I want it to match those. If we want though, I could change the config field name, env var prefix, struct field name, and this type name.

// See [ClientConfig.FromTOML] for details on format.
//
// WARNING: Environment configuration is currently experimental.
func LoadClientConfig(options LoadClientConfigOptions) (ClientConfig, error) {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This and LoadClientOptions don't have distinct enough names, IMO.

Maybe LoadClientOptionsFromFile ?

Copy link
Member Author

@cretz cretz Feb 25, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think it's distinct enough given its return type and such, but I can change and change the param structure to LoadClientConfigFromFileOptions. Would like to wait on other opinions if anyone has any. I am concerned there may also be too-similar naming concerns in other SDKs.

if env == nil {
env = EnvLookupOS
}
if s, ok := env.LookupEnv("TEMPORAL_ADDRESS"); ok {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think it probably makes sense to make these all public constants. It helps with both documentation and, since people can technically implement their own lookup, they might want to programmatically use them.

Copy link
Member Author

@cretz cretz Feb 25, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't expect this to be where such things are documented. I am expecting docs.temporal.io to house the documentation with these environment variables. I don't expect anyone to ever reference such a constant programmatically either like one might expect with an exported constant.

}

// ToClientOptionsOptions are options for [ClientConfig.ToClientOptions] and [ClientConfigProfile.ToClientOptions].
type ToClientOptionsOptions struct {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

OptionsOptions feels a little awkward for a name, but can't think of a great alternative..

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Agreed, I had brought this up internally but no good solution was provided. I don't expect users to use this part of the API much.

if err != nil {
return ClientConfigProfile{}, err
}
// Find profile or fail. The one situation where we don't fail here is if the profile is not user-set
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This comment took me a sec to understand, I think something like this could be easier to understand

Suggested change
// Find profile or fail. The one situation where we don't fail here is if the profile is not user-set
// Find user-set profile or use default. Fail if user uses invalid profile.

@tomwheeler tomwheeler self-requested a review February 25, 2025 20:10
Copy link
Contributor

@tomwheeler tomwheeler left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@cretz I updated my local copy of your Go SDK to commit #362043e and then tried this again.

Following the approach previously described, I tested this with mTLS and found that it now works as expected.

I then attempted to do the same with API Key authentication. I unset the previous four environment variables and then set the following:

  • TEMPORAL_API_KEY=<redacted>
  • TEMPORAL_NAMESPACE=tom-dev.a2dd6
  • TEMPORAL_ADDRESS=us-east-1.aws.api.temporal.io:7233

When I started the Worker, it failed:

$ go run worker/main.go
2025/02/25 13:53:31 INFO  No logger configured for temporal client. Created default one.
2025/02/25 13:53:32 Unable to create Temporal client. failed reaching server: rpc error: code = Unauthenticated desc = Jwt is expired
exit status 1

I then looked at the code and realized that perhaps I needed to run this:

$ export TEMPORAL_TLS=true

I then repeated the previous command and found that the Worker now started. I did the same in the other terminal before running the command to start the Workflow, which then completed successfully.

In the end, it seems to work fine, although I think we'd be wise to emphasize in the docs that you must set TEMPORAL_TLS=true when using API Key authentication with Temporal Cloud, because I think people might overlook it otherwise.

@cretz
Copy link
Member Author

cretz commented Feb 25, 2025

Jwt is expired

@tomwheeler - that is the error you got originally, I am not sure how TEMPORAL_TLS=true affects that. For env config, we are defaulting TLS to true if you set the API key (all of this is not documented yet because so much may change during this review process), so you should not have to set it manually. I will confirm this is the case. Also, feel free to contact via internal channels if that'd be easier.

EDIT: @tomwheeler - I cannot replicate. With the samples-go project checked out to temporalio/samples-go#392 and the SDK it references checked out to this PR, when I run this in one terminal (Windows, change set to export for nix):

set TEMPORAL_ADDRESS=us-west-2.aws.api.temporal.io:7233
set TEMPORAL_NAMESPACE=sdk-ci.a2dd6
set TEMPORAL_API_KEY=omitted
go run ./helloworld/worker

and in another terminal

set TEMPORAL_ADDRESS=us-west-2.aws.api.temporal.io:7233
set TEMPORAL_NAMESPACE=sdk-ci.a2dd6
set TEMPORAL_API_KEY=omitted
go run ./helloworld/starter

It works as expected. TEMPORAL_TLS is not needed.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

External client configuration
4 participants