Micro average precision and recall not listed for non-span sequence labelers

[alanakbik]

The current implementation of the SequenceLabeler evaluation evaluates non-span sequence labeling such that a prediction is made for each word. This is correct in many cases like POS tagging where each word must have one POS tag.

However, in some cases - like word or frame disambiguation - only some words have a predicted sense tag. Many words will have no prediction. Our evaluation currently counts "no prediction" as a class and so no micro average precision and recall are computed.

To reproduce:

# load English universal proposition bank
corpus = UP_ENGLISH().downsample(0.001)

# make tag dictionary
tag_dictionary = corpus.make_tag_dictionary('frame')

# init simple tagger
tagger: SequenceTagger = SequenceTagger(
    hidden_size=256,
    embeddings=WordEmbeddings('glove'),
    tag_dictionary=tag_dictionary,
    tag_type='frame',
    use_crf=False, # there are too many classes for CRF
)

# train model
trainer = ModelTrainer(tagger, corpus)

trainer.train(f'resources/taggers/frame-test-output',
              max_epochs=50,
              mini_batch_size=8,
              )

The output then does not list micro-averaged precision and recall.

(Note that this only affects non-span sequence labeling, so NER for instance is not affected.)