GH-1309: WorEmbeddings replacement that stores vectors in sqlite data…

[timnon]

Note: reopenend this PR, since i accidentially based the old one on my master branch

#1309

The ner-tagger of flair eats a few gigs of memory when e.g. run as the backend of a simple flask-application. This is not necessary, since the main memory consumer are word embeddings which are simple vector lookups. These can be externalized to some indexed database, e.g. sqlite, but any database does the job.

I did this by creating a class WordEmbeedingsStore which can be used to replace a WordEmbeddings-instance in a flair model via duck-typing. By using this, i was able to reduce our ner-servers memory consumption from 6gig to 600mb (10x decrease) by adding a few lines of code. It can be tested using the following lines (also in the docstring). First create a headless version of a model without word embeddings:

from flair.inference_utils import WordEmbeddingsStore
from flair.models import SequenceTagger
import pickle
tagger = SequenceTagger.load("multi-ner-fast")
WordEmbeddingsStore.create_stores(tagger)
pickle.dump(tagger, open("multi-ner-fast-headless.pickle", "wb"))

and then to run the stored headless model without word embeddings, use:

from flair.data import Sentence
tagger = pickle.load(open("multi-ner-fast-headless.pickle", "rb"))
WordEmbeddingsStore.load_stores(tagger)
text = "Schade um den Ameisenbären. Lukas Bärfuss veröffentlicht Erzählungen aus zwanzig Jahren."
sentence = Sentence(text)
tagger.predict(sentence)

Please give some advice where to add this functionality, there seems to be no place for inference-related stuff, therefore i created inference_utils.py, but some other place might be more appropriate.

I like the current structure such that it is a very light integration via duck-typing, so no flair base classes need to get changed, but a tighter integration might also be possible.

[timnon]

one more thing, the outsourcing is not restricted to sqlite, this could be handled by any database or also redis, so a general "outsourcing connector" to different backends would be appropriate.

[alanakbik]

@timnon thanks a lot - this is a good solution to reduce the memory footprint of WordEmbeddings which many people will find useful! I've made a few small changes: the db variable was not always self.db causing errors and I've changed the zero tensor to avoid a torch warning.

I think a more integrated solution of this would be great, but I'm not sure how to best do this. I'll think a bit about this and let you know! Thanks again!

[timnon]

Cool, thanks for the fixes, tested and everything seems to be good. As written above, any backend to store vectors would be good, but the elegance of sqlite is to have all in files, which suits the current storing mechanism.

[alanakbik]

Great, will merge now! Thanks again!

[alanakbik]

👍