implement pure Python "span-F1" evaluation routine applicable to BIO/BIOS

[miracle24]

Hello. Thanks for your wonderful code. I find you also calculate the F1 score using the perl script. I think there is a problem if this script is used to calculate the F1 score for the BIOS tag scheme, since this perl script cannot deal with those tag start with 'S-'. So I wonder if you guys correct the script code or you convert the tag scheme of tokens before using the script? If I am not right, please let me know. Thanks again.

[alanakbik]

Hi Miracle, many thanks for spotting this - the script indeed does not have any special provisions for the S-tags, but seems to default to chunkstart=True and chunkend=True for any non-BIE entity tag which is good. I've ran some tests and found some rare cases where it fails to work as intended, so this will probably have a minor effect on evaluation numbers and means that I have to take a closer look. I think for the upcoming release we might end up just implementing our own evaluation routine in Python, which will make a lot of this easier!

[tabergma]

in release-0.3