Restart processes only once when doing a setting rollout and include uptime from processes

[johscheuer]

Description

Fixes: #2161

Type of change

Please select one of the options below.

• Bug fix (non-breaking change which fixes an issue)

Discussion

Testing

I added a new operator test and performed some manual tests and monitored the fdb-monitor-log:

Time="1731500867.138682" Severity="10" LogGroup="jscheuermann-jdev" Process="fdbmonitor": Could not remove inotify conf file watch, continuing...
Time="1731500867.138731" Severity="10" LogGroup="jscheuermann-jdev" Process="fdbmonitor": Watching conf file /var/dynamic-conf/fdbmonitor.conf
Time="1731500867.138742" Severity="10" LogGroup="jscheuermann-jdev" Process="fdbmonitor": Watching conf dir /var/dynamic-conf/ (10)
Time="1731500867.138749" Severity="10" LogGroup="jscheuermann-jdev" Process="fdbmonitor": Loading configuration /var/dynamic-conf/fdbmonitor.conf
Time="1731500867.138925" Severity="10" LogGroup="jscheuermann-jdev" Process="fdbmonitor": Starting fdbserver.1
Time="1731500867.139186" Severity="10" LogGroup="jscheuermann-jdev" Process="fdbmonitor": Starting fdbserver.2
Time="1731500867.139418" Severity="10" LogGroup="jscheuermann-jdev" Process="fdbserver.1": Launching /usr/bin/fdbserver (27) for fdbserver.1
Time="1731500867.139462" Severity="10" LogGroup="jscheuermann-jdev" Process="fdbserver.2": Launching /usr/bin/fdbserver (28) for fdbserver.2
Time="1731500867.189768" Severity="10" LogGroup="jscheuermann-jdev" Process="fdbserver.2": FDBD joined cluster.
Time="1731500867.190228" Severity="10" LogGroup="jscheuermann-jdev" Process="fdbserver.1": FDBD joined cluster.
Time="1731500914.453225" Severity="10" LogGroup="jscheuermann-jdev" Process="fdbmonitor": Could not remove inotify conf file watch, continuing...
Time="1731500914.453272" Severity="10" LogGroup="jscheuermann-jdev" Process="fdbmonitor": Watching conf file /var/dynamic-conf/fdbmonitor.conf
Time="1731500914.453283" Severity="10" LogGroup="jscheuermann-jdev" Process="fdbmonitor": Watching conf dir /var/dynamic-conf/ (12)
Time="1731500914.453289" Severity="10" LogGroup="jscheuermann-jdev" Process="fdbmonitor": Loading configuration /var/dynamic-conf/fdbmonitor.conf
Time="1731500914.453415" Severity="10" LogGroup="jscheuermann-jdev" Process="fdbmonitor": Updated configuration for fdbserver.2
Time="1731500914.453485" Severity="10" LogGroup="jscheuermann-jdev" Process="fdbmonitor": Updated configuration for fdbserver.1
Time="1731500931.652931" Severity="10" LogGroup="jscheuermann-jdev" Process="fdbmonitor": Could not remove inotify conf file watch, continuing...
Time="1731500931.652976" Severity="10" LogGroup="jscheuermann-jdev" Process="fdbmonitor": Watching conf file /var/dynamic-conf/fdbmonitor.conf
Time="1731500931.652985" Severity="10" LogGroup="jscheuermann-jdev" Process="fdbmonitor": Watching conf dir /var/dynamic-conf/ (14)
Time="1731500931.652993" Severity="10" LogGroup="jscheuermann-jdev" Process="fdbmonitor": Loading configuration /var/dynamic-conf/fdbmonitor.conf
Time="1731500931.653108" Severity="10" LogGroup="jscheuermann-jdev" Process="fdbmonitor": Found new configuration for fdbserver.2
Time="1731500931.653156" Severity="10" LogGroup="jscheuermann-jdev" Process="fdbmonitor": Found new configuration for fdbserver.1
Time="1731501053.633942" Severity="20" LogGroup="jscheuermann-jdev" Process="fdbserver.1": Process 27 exited 0, restarting in 0 seconds
Time="1731501053.634312" Severity="10" LogGroup="jscheuermann-jdev" Process="fdbserver.1": Launching /usr/bin/fdbserver (217) for fdbserver.1
Time="1731501053.634625" Severity="20" LogGroup="jscheuermann-jdev" Process="fdbserver.2": Process 28 exited 0, restarting in 0 seconds
Time="1731501053.634899" Severity="10" LogGroup="jscheuermann-jdev" Process="fdbserver.2": Launching /usr/bin/fdbserver (218) for fdbserver.2
Time="1731501053.679621" Severity="10" LogGroup="jscheuermann-jdev" Process="fdbserver.1": FDBD joined cluster.
Time="1731501053.680075" Severity="10" LogGroup="jscheuermann-jdev" Process="fdbserver.2": FDBD joined cluster.

Documentation

Follow-up

[foundationdb-ci]

Result of fdb-kubernetes-operator-pr on Linux CentOS 7

• Commit ID: 665bc33
• Duration 3:19:18
• Result: ❌ FAILED
• Error: Error while executing command: if $fail_test; then exit 1; fi. Reason: exit status 1
• Build Log terminal output (available for 30 days)
• Build Workspace zip file of the working directory (available for 30 days)

[foundationdb-ci]

Result of fdb-kubernetes-operator-pr on Linux CentOS 7

• Commit ID: 511dc55
• Duration 3:17:29
• Result: ❌ FAILED
• Error: Error while executing command: if $fail_test; then exit 1; fi. Reason: exit status 1
• Build Log terminal output (available for 30 days)
• Build Workspace zip file of the working directory (available for 30 days)

[foundationdb-ci]

Result of fdb-kubernetes-operator-pr on Linux CentOS 7

• Commit ID: 1f17d4f
• Duration 3:03:32
• Result: ✅ SUCCEEDED
• Error: N/A
• Build Log terminal output (available for 30 days)
• Build Workspace zip file of the working directory (available for 30 days)

[nicmorales9]

LGTM

[johscheuer]

Merging since only the name of a test case was corrected.

[foundationdb-ci]

Result of fdb-kubernetes-operator-pr on Linux CentOS 7

• Commit ID: bbeb6ae
• Duration 3:07:46
• Result: ✅ SUCCEEDED
• Error: N/A
• Build Log terminal output (available for 30 days)
• Build Workspace zip file of the working directory (available for 30 days)