Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Ethstats operators SBAT receive stats from all elected validators #458

Closed
timmoreton opened this issue Sep 14, 2019 · 8 comments · Fixed by #543 or celo-org/ethstats-server#6
Closed
Assignees

Comments

@timmoreton
Copy link
Contributor

  • Right now, there is a single global token allowing anyone to publish. (So only CeloLabs validators appear on the list).
  • We should use the list of validators in groups which have one or more validator elected as the basis for whose signature to authenticate for stats -- this would allow any validator to publish in a permissionless way.
  • Also support blacklisting of addresses to prevent malicious validators spamming ethstats instances
@diminator
Copy link
Contributor

diminator commented Sep 24, 2019

The description was slightly vague, so I made a sketch how I think the situation is/could be:

image

  • Full nodes and validators specify a ethstats-operator for pushing their stats to
  • The ethstats-operator now accepts any inbound stats publisher but should only accept "validators in groups which have one or more validator elected"
  • "whose signature to authenticate for stats": stats are signed by the validator and this is used as a basis for authentication at the ethstats-operator side
  • When a node wants to publish stats to a specific ethstats-operator authentication at the ethstats-operator side verifies the signature and other attributes of the validator on-chain, such as the state of the validator group
  • spammers or malicious actors can be detected and put on a blacklist. this blacklist is maintained locally

@timmoreton
Copy link
Contributor Author

Thanks for writing this up! The description was a little vague because I hadn't figured out the details :-) There are a several open questions I think:

  • Signing the stats: we should probably use the node key, rather than the validator's consensus key.
  • Determining from whom to accept stats: ethstats could either (a) talk directly to a tx node (complicates how+where we get info from and complicates configuration) or (b) have a whitelisted set of validators and get them to periodically tell ethstats who they think the active validator set is, and use that to determine which stats from other nodes to accept.
  • how to treat stats from sentries and validators: I will raise a subsequent ticket for this

@diminator
Copy link
Contributor

diminator commented Sep 24, 2019

Signing the stats: we should probably use the node key, rather than the validator's consensus key.

How do you associate the node key with the keys associated to the active validator set?

Determining from whom to accept stats

Maybe ethstats-operators can verify the signed auth token with some geth calls to a local full node (check validator group and sig, maybe on-chain blacklist)?

@diminator
Copy link
Contributor

Any more design to be done on this issue?

@timmoreton
Copy link
Contributor Author

When a node wants to publish stats to a specific ethstats-operator authentication at the ethstats-operator side verifies the signature and other attributes of the validator on-chain, such as the state of the validator group

We could have ethstats get the details from on-chain, but an alternative approach would be that ethstats has a 'trustedlist' of validators that it relies on as the source of truth for which nodes to accept stats from. So, how about every validator sends ethstats the list of enodes of all currently elected validators. This is available as sb.valEnodeTable in consensus/istanbul/backend/backend.go - but looks like you'll need to make a function to retrieve all its entries at once. When ethstats receives this list from a validator on its 'trusted' list, it updates the list of enodes its willing to receive stats from.

Two extensions:

  • if a validator hasn't been seen for a long time, it could drop off the list - probably don't need this now.
  • Probably want ethstats to persist this list locally periodically and reload it on startup so it can get started quickly.

@timmoreton
Copy link
Contributor Author

So to be clear, this approach uses the nodekey and not the validator key. Stats would be signed by the nodekey. The enode includes the node address which is derived from the nodekey, so we use that to check sigs.

@diminator
Copy link
Contributor

Thanks for the additional clarifications.

Currently ethstats is a single stateless instance listening to a websocket.

  • do we create a minimal backend to store the trusted list? Or maybe we reuse the Explorer backend of blockscout (in the assumption that celostats equals "ethstats" + "blockscout" + "...")
  • will multiple ethstats operators exist? If so:
    • will they reuse a single repo for the trusted list or each stores the list by themselves (assuming the latter)
    • should validators be able to multicast to ethstats operators or only broadcast to a single one?

@timmoreton
Copy link
Contributor Author

Thanks for the additional clarifications.

Currently ethstats is a single stateless instance listening to a websocket.

  • do we create a minimal backend to store the trusted list? Or maybe we reuse the Explorer backend of blockscout (in the assumption that celostats equals "ethstats" + "blockscout" + "...")

I think the "trusted list" can be config that is read at startup (e.g a command line parameter)?

  • will multiple ethstats operators exist?

Yes, I am assuming there could be multiple operators but it's not essential if we defer supporting this to later.

If so:

  • will they reuse a single repo for the trusted list or each stores the list by themselves (assuming the latter)

The latter - every operator probably trusts a different subset of validators

  • should validators be able to multicast to ethstats operators or only broadcast to a single one?

Eventually, they should be able to contact multiple, but I would defer this to a later ticket

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment