Block PRs from getting merged if they decrease coverage

[thomas-moulard]

Description

CodeCov can block PRs from getting merged if a certain threshold is not met (either relative or absolute), see https://docs.codecov.io/docs/commit-status

We should agree on some value and enable this.

Test Plan

• Manual testing - set the threshold very high, verify that the build fails, then set it back to an appropriate value.

Documentation Plan

n/a

Release Plan

n/a

Acceptance Criteria

• Code has been implemented, reviewed and merged.

Once all above items are checked, this story can be moved to done.

Resources

• Codecov doc

[thomas-moulard]

@dabonnie PTAL

[dabonnie]

I agree, though I'm not sure what value is reasonable. Should we initially try to enforce no decreased coverage?

[thomas-moulard]

👍 That'd be great. We may allow some decrease to avoid blocking PRs containing code really hard to trigger (e.g. actually reading /proc).

[dabonnie]

I initially tried blocking with the default setting (no coverage) and it was too strict. I think we need to understand all our of partial hits, especially in the unit test files, before we enable blocking PRs based on coverage.

[mm318]

I have a feeling that every ASSERT_/EXPECT_ statement we use in our tests contribute to missed branch coverage, because the failure branches are never hit since our tests are always passing.