Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Increase miner sampling for each step #249

Merged
merged 5 commits into from
Jun 5, 2024

Conversation

bkb2135
Copy link
Collaborator

@bkb2135 bkb2135 commented Jun 5, 2024

Sampling 5% of the network every step introduces a lot of noise into the ema calculation, which leads to unstable incentives. This problem is mitigated by sampling all uids each step.

This PR has currently been run on testnet, but needs a mainnet run to examine stability, as this could potentially be a large change.

Copy link

@Unkownman086 Unkownman086 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do you really think making 1024 connections is good idea? I believe default ubuntu limit is 1024 file descriptors if not modified.

I believe querying all endpoints in unnecessary

@bkb2135
Copy link
Collaborator Author

bkb2135 commented Jun 5, 2024

I believe default ubuntu limit is 1024 file descriptors if not modified.

In practice we should only query 1001 miners, because validators do not query each other. The rationale for querying all miners is that when we query only a small subset of miners, because of the different distributions of rewards on tasks, 2 models running the exact same code with the exact same intelligence can be on opposite ends of the incentive distribution based on who gets lucky with getting which queries, which does not incentivize intelligence.

@p-ferreira p-ferreira changed the title Sample All Miners Each Step Increase miner sampling for each step Jun 5, 2024
@p-ferreira p-ferreira merged commit 2aa77c8 into staging Jun 5, 2024
2 checks passed
@p-ferreira p-ferreira mentioned this pull request Jun 5, 2024
@Hollyqui Hollyqui deleted the feature/remove-random-uid-sampling branch August 2, 2024 08:17
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants