GH-921: fine-tune FlairEmbeddings

[alanakbik]

Closes #921

This PR makes FlairEmbeddings task-trainable. This allows users to (a) fine-tune an existing language model on task data and (b) train a new model only on task data.

• You can fine-tune an existing LM by simply passing the fine_tune parameter in the FlairEmbeddings constructor, like this:

embeddings = FlairEmbeddings('news-foward', fine_tune=True)

• You can task-train a wholly new language model by passing an empty LanguageModel to the FlairEmbeddings constructor and the fine_tune parameter, like this:

# make an empty language model
language_model = LanguageModel(
    Dictionary.load('chars'),
    is_forward_lm=True,
    hidden_size=256,
    nlayers=1)

# init FlairEmbeddings to task-train this model
embeddings = FlairEmbeddings(language_model, fine_tune=True)

• Also closes AttributeError: 'WordEmbeddings' object has no attribute 'embeddings' #880 by fixing the embedding printout.

• This PR also removes the FlairEmbeddings-specific disk-caching mechanism. In the future, a more general caching mechanism applicable to all embedding types should be added on a different level of logic, potentially as a feature in the ModelTrainer.

[yosipk]

👍

[alanakbik]

👍

[jasminesjlee]

For clarification -- what exactly does it mean to "task-train a wholly new language model"? Is this different from training a LM on this data from scratch?

[alanakbik]

Hello @jasminesjlee yes this would be different since the training objective is different. A standard language model is trained to predict the next character given the previous characters. However, what we do here is wrap it into FlairEmbeddings and train it on a downstream task (such as NER). So we train a character-level RNN to produce useful features specifically for the downstream task (see the paper by Liu et al. who originally proposed this). Due to the different objective, such an LM cannot be used to generate text.

[jasminesjlee]

Ah, I understand. Thank you @alanakbik ! :)

[DecentMakeover]

@alanakbik What branch should i be in to access these changes i am currently on

      origin/GH-538-attention
  origin/GH-822-xlnet
  origin/GH-873-pytorch-transformers
  origin/HEAD -> remotes/origin/master
  origin/SimilarityLearner
  origin/master
  origin/release-0.4.1

Thanks

[pommedeterresautee]

As this PR is merged it is available on master

[DecentMakeover]

@pommedeterresautee Thanks .