Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Prioritizing classes and precission/recall for sequence tagger #1344

Closed
klasocki opened this issue Jan 11, 2020 · 1 comment
Closed

Prioritizing classes and precission/recall for sequence tagger #1344

klasocki opened this issue Jan 11, 2020 · 1 comment

Comments

@klasocki
Copy link
Contributor

klasocki commented Jan 11, 2020 •
edited
Loading

Following @apohllo 's idea, I implemented two ways of prioritizing precission/recall during training

  • Using F-beta as the evaluation metric, has impact on logging and annealing schedule

  • Adding class weights to cross-entropy loss, enabling class prioritization (e.g less weight to errors on 'O' tag if we want more recall). Works only if using standard loss - no CRF

I'll submit a pull request in case you'd want to include these changes in your library :)
@klasocki klasocki changed the title Prioritising classes and precission/recall for sequence tagger Prioritizing classes and precission/recall for sequence tagger Jan 11, 2020
klasocki added a commit to klasocki/flair that referenced this issue Jan 11, 2020
@klasocki
flairNLPGH-1344: Add F-beta score as Flair evaluation metric
44fd68c
klasocki added a commit to klasocki/flair that referenced this issue Jan 11, 2020
@klasocki
flairNLPGH-1344: Add weights for tags in loss function
1f3e12f 
Add logging of new training parameters - weights and beta
Fix name errors in F-beta metric
klasocki added a commit to klasocki/flair that referenced this issue Jan 11, 2020
@klasocki
flairNLPGH-1344: Add warning when specified weights don't take effect
3cef4ab
klasocki added a commit to klasocki/flair that referenced this issue Jan 11, 2020
@klasocki
flairNLPGH-1344: Add warning when specified weights don't take effect
d2c7c69
klasocki added a commit to klasocki/flair that referenced this issue Jan 11, 2020
@klasocki
flairNLPGH-1344: Add warning when specified weights don't take effect
e3d3233
klasocki added a commit to klasocki/flair that referenced this issue Jan 11, 2020
@klasocki
flairNLPGH-1344: Fix logging errors
97becbf 
Log new parameter values only for SequenceTagger instances
klasocki added a commit to klasocki/flair that referenced this issue Jan 13, 2020
@klasocki
flairNLPGH-1344: Add weights to TextClassifier
7bdafb5 
Add weights to state_dict for loading and saving models
Remove unnecessary isinstance calls while logging by overriding __str__
klasocki added a commit to klasocki/flair that referenced this issue Jan 13, 2020
@klasocki
flairNLPGH-1344: Add weights to TextClassifier
3a51e68 
Add weights to state_dict for loading and saving models
Remove unnecessary isinstance calls while logging by overriding __str__
Minor refactor
klasocki added a commit to klasocki/flair that referenced this issue Jan 13, 2020
@klasocki
flairNLPGH-1344: Add weights to TextClassifier
25e7614 
Add weights to state_dict for loading and saving models
Remove unnecessary isinstance calls while logging by overriding __str__
Minor refactor
alanakbik added a commit that referenced this issue Jan 22, 2020
@alanakbik
Merge pull request #1345 from klasocki/prioritizing-classes
f7dd29d 
GH-1344: Prioritizing classes
@alanakbik
Copy link
Collaborator

New version of flair released that contains this feature - thanks again @klasocki!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
2 participants
@alanakbik @klasocki