Backoff on etcd errors

[EngHabu]

Tracking issue

fixes #5770

Why are the changes needed?

Propeller should backoff to avoid thrashing struggling etcserver even further.

What changes were proposed in this pull request?

Account for etcdserver errors when checking whether we should backoff or not.

How was this patch tested?

Setup process

Screenshots

Check all the applicable boxes

• I updated the documentation accordingly.
• All new and existing tests passed.
• All commits are signed-off.

[codecov]

Codecov Report

Attention: Patch coverage is 33.33333% with 2 lines in your changes missing coverage. Please review.

Project coverage is 36.17%. Comparing base (7a91799) to head (d0e327d).
Report is 185 commits behind head on master.

Files with missing lines	Patch %	Lines
...eller/pkg/controller/nodes/task/backoff/handler.go	33.33%	2 Missing ⚠️

Additional details and impacted files

@@           Coverage Diff           @@
##           master    #5710   +/-   ##
=======================================
  Coverage   36.17%   36.17%           
=======================================
  Files        1303     1303           
  Lines      109663   109665    +2     
=======================================
+ Hits        39667    39670    +3     
+ Misses      65851    65850    -1     
  Partials     4145     4145           

Flag	Coverage Δ
unittests-datacatalog	51.37% <ø> (ø)
unittests-flyteadmin	55.28% <ø> (ø)
unittests-flytecopilot	12.17% <ø> (ø)
unittests-flytectl	62.22% <ø> (+0.04%)	⬆️
unittests-flyteidl	7.12% <ø> (ø)
unittests-flyteplugins	53.34% <ø> (ø)
unittests-flytepropeller	41.76% <33.33%> (-0.01%)	⬇️
unittests-flytestdlib	55.35% <ø> (ø)

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.