Etcd based cluster config

[samutamm]

Close #233. Based on this design #233 (comment).

• Minor refactoring of URL parsing in pkg/cluster/config.go
• Add alternative way to configure cluster so that it start's all members at once (example config) VS the current way, which starts members one by one (example legacy config)
• Add unittests for new configuration

[codecov-commenter]

Codecov Report

Merging #382 (9400277) into main (0e61435) will increase coverage by 0.24%.
The diff coverage is 76.92%.

@@            Coverage Diff             @@
##             main     #382      +/-   ##
==========================================
+ Coverage   80.57%   80.82%   +0.24%     
==========================================
  Files          60       60              
  Lines        6888     6961      +73     
==========================================
+ Hits         5550     5626      +76     
- Misses       1043     1044       +1     
+ Partials      295      291       -4     

Impacted Files	Coverage Δ
pkg/cluster/layout.go	100.00% <ø> (ø)
pkg/object/mqttproxy/mqttproxy.go	15.18% <0.00%> (-1.03%)	⬇️
pkg/cluster/cluster.go	50.59% <63.88%> (+1.82%)	⬆️
pkg/cluster/member.go	78.12% <71.42%> (+0.43%)	⬆️
pkg/cluster/config.go	88.88% <90.54%> (+19.02%)	⬆️
pkg/object/mqttproxy/client.go	79.87% <0.00%> (-1.22%)	⬇️

---

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 0e61435...9400277. Read the comment docs.

[localvar]

please also fix the issues reported by Github actions.

[xxx7xxxx]

I don't go into detail about the code in the first review, I would like to specify my vision about this PR along with reasons.

1. Support older and newer roles by changing older names to newer ones in startup options. (You have already fixed it.)
2. The layout of the option is not a good design IMHO, I'd like to give a clean layout. Example without the trivial:

Primary member:

name: primary-1
cluster-name: cluster-test
cluster-role: primary
cluster-primary:
  listen-peer-urls: [http://127.0.0.1:2380]
  listen-client-urls: [http://127.0.0.1:2379]
  advertise-client-urls: [http://127.0.0.1:2379]
  initial-advertise-peer-urls: [http://127.0.0.1:2380]
  initial-cluster-state: new
  initial-cluster:
   - primary-1: http://127.0.0.1:2380
   - primary-2: http://127.0.0.1:2378
   - primary-3: http://127.0.0.1:2376

Secondary member:

name: secondary-1
cluster-name: cluster-test
cluster-role: secondary
cluster-secondary:
  primary-listen-peer-urls: [http://127.0.0.1:2380]

The reason I designed them like that is to make it cleaner to write/read config by splitting them up.

3. Please adapt the reader config to new rules.

4. Please don't add things to cluster/member.go which all will be deleted. What you changed in it will break existing cluster with member file.

[samutamm]

@xxx7xxxx Thanks for the comment! I think that the example you proposed makes it simpler to add new reader/secondary members. I would develop it further like this:

Almost the same config for primary member.

name: primary-1
cluster-name: cluster-test
cluster-role: primary # writer
cluster: # only primary member defines cluster entry 
  listen-peer-urls: [http://127.0.0.1:2380]
  listen-client-urls: [http://127.0.0.1:2379]
  advertise-client-urls: [http://127.0.0.1:2379]
  initial-advertise-peer-urls: [http://127.0.0.1:2380]
  initial-cluster-state: new
  initial-cluster:
   - primary-1: http://127.0.0.1:2380
   - primary-2: http://127.0.0.1:2378
   - primary-3: http://127.0.0.1:2376

and then secondary

name: secondary-1
cluster-name: cluster-test
cluster-role: secondary # reader
# secondary member defines, where to connect to primary
primary-listen-peer-urls: [http://127.0.0.1:2380]

[samutamm]

@xxx7xxxx I removed the extra logic from members.go, so it's easier to remove in future. The configuration is also as described above.

[xxx7xxxx]

1. As I commented last time, please adapt older examples of reader-004/reader-005.
2. Have you tested adding members, then shutdown all members, then restart all nodes, will it restart successfully?