[cmd/opampsupervisor] Support supervisor being ran "headlessly" from an OpAmp server

[dpaasman00]

Component(s)

cmd/opampsupervisor

Is your feature request related to a problem? Please describe.

Not all deployments of the collector warrant having an OpAmp connection or a connection isn't possible, and in such situations the supervisor isn't usable. This can be an issue for users as the supervisor isn't strictly a mechanism for OpAmp communication i.e. running the collector independently removes the watch dog benefits the supervisor provides.

Additionally, providing options to run the collector with or without the supervisor means maintaining multiple installation flows and packages. If users wish to switch between OpAmp management being enabled or not enabled, they need to uninstall one and install the other. The overhead of maintaining multiple installation flows & packages and then needing to execute these flows multiple times is prone to failure.

Describe the solution you'd like

The ability to run the supervisor disconnected from an OpAmp server would solve the above issues. Simply changing part of the supervisor's config file and then restarting the process would eliminate needing to maintain multiple installations flows & packages.

In order to support running the supervisor disconnected from an OpAmp server there are a couple key things that need to be changed.

1. The parameter server.endpoint should no longer be required. If this parameter is missing from the configuration, this is how the supervisor knows it's running while disconnected from a management server. How the supervisor handles this can be discussed more, but currently I think the supervisor should just create what's effectively a nop OpAmp client.
2. How the supervisor manages the collector's config will need to be changed. Currently it stores the last received remote-config's protobuf on disk and checks for it on startup. When present it writes this config to the agent's config file and when it isn't present it writes it's own minimal/nop config for the collector to run until it receives a remote config. This behavior needs to change so that users can create their own config on disk the supervisor can run the collector with, without needing to have received the config from an OpAmp server. This would also help simplify deployments of the supervisor & collector as you wouldn't need to ensure a connection to an external platform to provide a configuration.

Describe alternatives you've considered

If the supervisor remains as is, an alternative to multiple installation flows could be manually modifying a current installation to fit the user's needs. However this requires having the necessary knowledge and permissions to modify the existing installation's configuration and service files which could quickly prove tedious to customers.

Additional context

I'm happy to implement this feature if accepted.

[github-actions]

Pinging code owners:

• cmd/opampsupervisor: @evan-bradley @atoulme @tigrannajaryan

See Adding Labels via Comments if you do not have permissions to add labels yourself.

[github-actions]

This issue has been inactive for 60 days. It will be closed in 60 days if there is no activity. To ping code owners by adding a component label, see Adding Labels via Comments, or if you are unsure of which component this issue relates to, please ping @open-telemetry/collector-contrib-triagers. If this issue is still relevant, please ping the code owners or leave a comment explaining why it is still relevant. Otherwise, please close it.

Pinging code owners:

• cmd/opampsupervisor: @evan-bradley @atoulme @tigrannajaryan

See Adding Labels via Comments if you do not have permissions to add labels yourself.

[odubajDT]

Nice idea! To your second point regarding the config, maybe this PR will be a solution, since as it gets merged, users will be able to provide their own custom config.

WDYT?

If you agree I am willing to take the implementation after the linked PR gets merged.

cc @evan-bradley