Move compute_ctl structs used in HTTP API and spec file to separate crate.

[hlinnaka]

This is in preparation of using compute_ctl to launch postgres nodes in the neon_local control plane. And seems like a good idea to separate the public interfaces anyway.

One non-mechanical change here is that we now use a RwLock rather than atomics to protect the ComputeNode::metrics field. We were not using atomics for performance but for convenience here, and an RwLock is now more convenient.

(extracted to separate PR from PR #3886)

[ololobus]

Moving structures in a separate crate sounds as good first step. Also moving metrics under the same lock should be fine too, as actually they are set now only during initial startup, so it's basically the same writer all the time

Yet, I made state to be under Mutex in the #3923 because of two reasons:

1. Condvar was super convenient for my use case
2. Write lock in the RwLock anyway blocks all readers, IIUC. And we don't have a case when there are a lot of concurrent readers

What do you think?

We should also agree on the order of PRs, probably :) Yours seems to be important to write better tests in the neon repo. Mine is needed to unblock control-plane work on reconfiguration without restart and maintaining the pool of pre-created computes. So not sure, just your refactoring looks relatively huge in general (if we consider all needed PRs)

[github-actions]

Test results for f0b2e07:

---

debug build: 212 tests run: 202 passed, 0 failed, 10 (full report)

---

release build: 212 tests run: 202 passed, 0 failed, 10 (full report)

---

[hlinnaka]

Moving structures in a separate crate sounds as good first step. Also moving metrics under the same lock should be fine too, as actually they are set now only during initial startup, so it's basically the same writer all the time

Yet, I made state to be under Mutex in the #3923 because of two reasons:

1. Condvar was super convenient for my use case

2. Write lock in the `RwLock` anyway blocks all readers, IIUC. And we don't have a case when there are a lot of concurrent readers

What do you think?

Sure, Mutex is fine.

We should also agree on the order of PRs, probably :) Yours seems to be important to write better tests in the neon repo. Mine is needed to unblock control-plane work on reconfiguration without restart and maintaining the pool of pre-created computes. So not sure, just your refactoring looks relatively huge in general (if we consider all needed PRs)

First to push wins ;-).

I suggest:

1. Move compute_ctl structs used in HTTP API and spec file to separate crate. #3937
2. Rename "Postgres nodes" in control_plane to endpoints. #3938
3. Implement live reconfiguration in the compute_ctl #3923

I can rebase #3923 over those two, I think the conflicts are pretty mechanical

[hlinnaka]

Rebased, fixing conflicts with commit e42982f.

I think this is ready to be merged. Please review and approve if you agree.

[ololobus]

LGTM, one comment / suggestion about files layout and this text

One non-mechanical change here is that we now use a RwLock rather than atomics to protect the ComputeNode::metrics field. We were not using atomics for performance but for convenience here, and an RwLock is now more convenient.

should be updated as we use Mutex now.