Add comet

[devrimcavusoglu]

Closes #99.

[devrimcavusoglu]

Currently CI is failing and one potential reason to that is not disabling the multiprocessing while scoring. There is a bug in the comet implementation, see this issue.

[devrimcavusoglu]

Any ideas for CI failing @Sophylax ? Some notes below:

• Test preserves around 4.5 GB memory, this is a similar case with Prism
• The PR addressing bug fix for the issue on unbabel/COMET is merged. Still with num_workers=0 it is failing.

Any similar situations experienced with your tests @ricardorei ?

[ricardorei]

Hi @devrimcavusoglu thanks for tagging me on this. Let's see if I can help you

I had some problems building the CI for COMET. My solution is to create tests using lightweight encoder models.

For example [wmt21-cometinho-da](https://github.com/Unbabel/COMET/blob/master/METRICS.md)?

[ricardorei]

which version of COMET are you currently using?

[devrimcavusoglu]

Thank you for the quick response @ricardorei . I missed the Metrics doc at first sight and only glanced to README, and didn't think there was a lighter model for COMET. Currently with your suggestion the CI problem is solved, thank you very much. One suggesstion from me could be to add/link METRICS.md in README.md.

[ricardorei]

Nice! I am happy to help, tag me if you need something else.

I'll do that. Actually, I need to update all the documentation, it is extremely outdated...

PS: cool repo, I think it's important to have tools to evaluate several metrics and stop people from reporting just BLEU :)

[devrimcavusoglu]

Thanks @ricardorei, appreciate it. Thanks for the comment about the repo as well, yes one of the milestone is that supplying a diverse set of metrics to people to make the evaluation in NLP more properly :).