Binary Protocol: Add authenticated redirection

[fantix]

Brief

EdgeDB cloud instances will have (relatively speaking) dynamic DSN to connect, given a static unique name like "org_name/instance_name". The Cloud API would provide a way for EdgeDB clients to resolve the cloud instance name into DSN securely. Now that many (if not most) programming languages don't have a built-in HTTPS client, we are proposing to add this support of authenticated redirection in the edgedb-binary protocol.

"The Client" can be either the CLI or the language bindings. The CLI could cloud login to get the token, while the language bindings could load the token from environment variable or the file where CLI stored the token.

Even though this proposal is targeting cloud redirection, but this could potentially be used in other scenarios too.

Proposal

The Client should start the JWT authentication with a ClientHandshake message. Instead of sending user and database in the connection parameters, the Client should include the following connection parameters in order to authenticate for a cloud redirect:

• token - the JWT token
• instance - the name of the cloud instance to connect

The Client doesn't have to look into the JWT token, which is an implementation detail of the server. As a reference, the claims in the token may contain information about allowed cloud organizations or instances, as well as the authorization to do redirection.

If the JWT authentication is a success, the cloud server should return the following messages sequencially:

• AuthenticationOK - indicating the JWT token is accepted
• ServerKeyData - not used for now
• StateDataDescription - not used
• ParameterStatus with name credentials - the DSN/credentials to connect to the actual instance, value explained later
• Terminate - indicating the current connection must be terminated

If the JWT authentication is a failure, a single ErrorResponse message is returned. In either case, the server will actively close the connection after returning.

The value of the credentials ParameterStatus is similar to the ParameterStatus_SystemConfig:

struct ParameterStatus_Credentials {
  // Type descriptor prefixed with type
  // descriptor uuid.
  uint32          num_typedesc;
  uint8           typedesc[num_typedesc];

  // Configuration settings data.
  DataElement     data[1];
};

And DataElement is again the same as it is in the Data message:

struct DataElement {
  // Encoded output data.
  uint32          num_data;
  uint8           data[num_data];
};

An typical credentials type descriptor looks like this:

scalar type TlsSecurity extending enum<'INSECURE', 'NO_HOST_VERIFICATION', 'STRICT', 'DEFAULT'>;

type Credentials {
    required property host -> str;
    required property port -> int32;
    property user -> str;
    property password -> str;
    property database -> str;
    property tls_ca -> str;
    required property tls_security -> TlsSecurity;
}

On receiving credentials, the client could then establish a new connection to the target server.

The Client is suggested to cache the resolved credentials in memory (for language bidnings) or on disk (for the CLI) for a reasonable time (e.g. 30 seconds in memory, or maybe days on disk because 1) the DSN shouldn't change often and 2) connection failure invalidates the cache, explained later), in order to reduce round-trip latency in cases when connections are established frequently. On a failed connection to the target server, the Client should consider invalidating the cache and re-resolve the cloud instance name. This should be transparently done without raising an error to the end-user like all other retry logic, until max retry attempts reached. Ideally, language bindings should not try to resolve the same cloud instance name concurrently, as the result would be the same.

[elprans]

• instance - the name of the cloud instance to connect

No need for this, the target instance should be encoded in JWT claims instead.

[fantix]

Does that mean the CLI edgedb -I org/inst would have 3 steps:

1. Call HTTP API with logged-in token to get an instance-connecting token;
2. Use binary protocol to exchange the connect token for credentials;
3. Use the credentials to establish the connection to the instance.

[elprans]

I think that's what the proposed diagram is showing? The instance-connecting token can be a long-lived one, so you don't need to renew it often.

[fantix]

Oh that works too ... I was trying to reuse the single edgedb cloud login token for all instances, but a per-instance token is better! Even surviving a cloud account change.

[elprans]

Call HTTP API with logged-in token to get an instance-connecting token;

Except this defeats the point of not having to use HTTP. I'm rethinking this, we do need to include org/inst in the connection request and allow arbitrarily scoped tokens for auth.

[1st1]

Sorry, a few things are not clear to me. I see the diagram, but I don't understand what parts of it are HTTP and what parts are using ClientHandshake. In fact the current diagram is quite confusing at that.

Will the get token part implemented via a REST request?

The connect org/inst part is ClientHandshake, right?

Why can't the get token part return DSN?

[fantix]

Oh the green arrows are binary protocol, forget to add a legend.

get token is explained as:

The CLI could cloud login to get the token, while the language bindings could load the token from environment variable or the file where CLI stored the token.

That means the language bindings could go directly into the green arrows with an environment variable.

The connect org/inst part is ClientHandshake, right?

Yes

Why can't the get token part return DSN?

It's not an API call, but rather a general step to get the token. But the comment from Elvis above may change the part about CLI.

[elprans]

The get token part is optional. You can generate an instance-scoped token in Cloud's Web dashboard, in which case the client can go straight to the connect part.

[1st1]

OK, so the idea here is to basically hijack our protocol to also be used by client<>cloud backend to be able to resolve org/inst into a DSN?

[fantix]

Yes, so that we don't have to introduce heavy HTTPS client libraries into client bindings.

[1st1]

Alright, sounds good!

[elprans]

In off-thread discussions we reached a consensus that this approach would insert too much of a control plane surface (and hence failure modes) into the user path, so we'll be going with consistent instance endpoint naming and adding direct support for token authentication to server instead.