Skip to content

Release v2.10.0
Compare
Choose a tag to compare
@RodrigoLPA RodrigoLPA released this 04 Mar 14:40
· 65 commits to main since this release
v2.10.0
d22404d
This commit was created on github.com and signed with GitHub’s verified signature.
GPG key ID: B5690EEEBB952194
Verified
Learn about vigilant mode.

Announcing Release 2.10.0

Following the drop in synthetic mmlu samples generated by SN1 (due to the increase in organic queries), our synthetic_mmlu and if_eval tasks were not fed enough samples to function properly. This is a hotfix that stabilises the subnet:

Changes

  • New task weights: synthetic_mmlu 30%, word_sorting 5%, fineweb 35%, if_eval_v2 30%.
  • Validators now sample 10 configs of the macrobench dataset and then samples the individual text samples from this subset.
  • Common errors in subprocess handling addressed. Mainly the too many open files error.

Notes to validators

Please make sure to increase the number of open files limit before running the validator, then update the dependencies:

$ ulimit -n 64000
$ python -m pip install -e .
Assets 2
Loading