Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Results for legal-bert-small #8

Closed
atreyasha opened this issue Dec 15, 2021 · 1 comment
Closed

Results for legal-bert-small #8

atreyasha opened this issue Dec 15, 2021 · 1 comment

Comments

@atreyasha
Copy link

atreyasha commented Dec 15, 2021

Hi @iliaschalkidis,

As mentioned previously, I have been running some experiments on LexGLUE benchmarks and will soon be finishing with the runs for legal-bert-small. It is mentioned that it would be useful to report the results of this smaller model.

Should I just post the results here, or would you prefer another medium?

@atreyasha atreyasha changed the title Results for LegalBERT-small Results for legal-bert-small Dec 15, 2021
@iliaschalkidis
Copy link
Collaborator

Hi @atreyasha,

That's really cool! Thanks for running those experiments 🥳

I would really appreciate it if you could report the new results in the newly created Discussions space under the "New Results 🥳" category (https://github.com/coastalcph/lex-glue/discussions/new?category=new-results).

Ideally, we would like to have:

  • The validation and test results per dataset (task) in a table format, similar to the one presented in the main README.md file.
  • The generated log files all_results.json for all five seeds and datasets (tasks). You can simply attach those in your message (post). To do so, you can simply drag and drop the files (or better a .zip file with all of them).

I suspect there are no changes in the code, and it was as smooth as replacing the model path, so no need for pull requests or any other reporting.

I'll rapidly review the results and then update the leaderboard with your reported results with a reference to your name (@atreyasha) or another identifier (full name, article) that you may find more appropriate (if this is ok with you of course).

Thanks again and let me know if you have any other question or you need help with the above steps. 🤗

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants