Research the best way to support advanced configuration per sync provider

[toddbaert]

flagd supports various sync providers (filepath, kubernetes, http, grpc), and also supports the usage of multiple providers simultaneously (just as an example, flagd can be started to listen to an HTTP endpoint, watch a file, and connect to 2 gRPC servers).

Each of these providers may require individual configuration. For instance, each gRPC stream might need different certs, or http syncs may each require different auth headers. We currently do not have a solution to handle this sort of configuration from the command line.

We need to decide on a way to support this. It should also work well with CRs we define in the OpenFeature operator.

Ideas:

• the use of multiple syncProvider args, supporting key/value encoding of some kind
• defining a configuration file structure and passing that to flagd
• ?

Edit: See Jame's research/suggestions below. I'm somewhat partial to this one.

Implementation issue: open-feature/open-feature-operator#382

[james-milligan]

I like the idea of a config file, its a pretty common pattern that would allow for developers to very easily interpret configurations (compared to a multi line list of startup flags), we can also create a schema definition for it to allow for linting + validation

[james-milligan]

Ive done some more investigation into a clean method for handling per-sync configurations:

Currently we are using viper to handle our flag values, out of the box viper allows for the parsing of a config file (.json/.yaml), we already have this configuration set up within flagd under the --config flag.
This means we can provide a config file such as:

port: 8080
sync-provider-args:
    food: bars
    cat: dog

resulting in each sync provider receiving the following map[string]string: map[cat:dog food:bars]

This flag structure feels inappropriate for our use case, instead we can update the sync.ProviderArgs type from map[string]string to map[string]interface{} and decode the --sync-provider-args flag value as map[string]sync.ProviderArgs

conf := &map[string]sync.ProviderArgs{}
err = viper.UnmarshalKey(providerArgsFlagName, conf)

this means we can now pass the following configuration file:

sync-provider-args:
    sync-uri-1:
        food: 5
        cat: dog
        fizz: 
            buzz: 7

this allows us to define configuration on a per-sync basis, using the sync URI as the map key, which we can filter for on startup, with each configuration defined as map[string]interface{}, to be handled internally by each sync as required.

IMO this implementation keeps out configurations extremely flexible, while ensuring each syncs configuration is kept separate. It also allows us to use both flag arguments and config files (if not are used the configuration from flag arguments overwrites any duplicated keys in the configuration file)

This will also make it easy to introduce the watching of these configurations: https://github.com/spf13/viper#watching-and-re-reading-config-files

[james-milligan]

Another option is to extend the functionality of the '--uri' flag to take optionally take a json configuration object.

We can identify any of the existing uri flags using the regex found in pkg/runtime/from_config.go, if none are matched then we can attempt to parse the argument into a struct similar to the following:

type UriConfiguration struct {
	URI             string `json:"uri"`
	ProviderID      string `json:"providerID,omitempty"`
	HTTPBearerToken string `json:"httpBearerToken,omitempty"`
}

If the string argument provided matches the regex, then we can create an 'empty configuration'.

We then pass this configuration to each of the providers, deprecating the existing '--sync-provider-args' flag and maintaining the backwards compatibility of the --uri flag.

Ive looked into implementing this with a viper.DecodeHook, and while possible, it will likely be far simpler to implement this using a helper function in the start command definition.

[james-milligan]

Im going to implement and open a PR for the string-ified object approach:

you could either pass:

--uri file://my-file.json

or:

--uri '{"uri":"my-file.json","arg":1}'

internally we would parse the response as a map[string]interface{} configuration object if the argument fails all existing sync provider regex. if they pass, a new object will be created, and the uri field set.

[beeme1mr]

Hey @james-milligan, I think the config object makes sense but I'm not sure if repurposing the --uri flag makes sense. Perhaps we should consider leaving uri as is and reintroducing --sync-provider and have that take the config object. It would also be a good opportunity to deprecate sync-provider-args.

What do you think @toddbaert?

[toddbaert]

Hey @james-milligan, I think the config object makes sense but I'm not sure if repurposing the --uri flag makes sense. Perhaps we should consider leaving uri as is and reintroducing --sync-provider and have that take the config object. It would also be a good opportunity to deprecate sync-provider-args.

What do you think @toddbaert?

I agree with this idea. --sources might be better, but that's up for debate.