DensePassageRetriever: Add Training, Refactor Inference to FARM modules

[kolk]

• DPR training from haystack
• DPR inference modules using FARM instead of transformers
• modify test_dpr_retirever with correct test cases
• Update tutorials
• Update docs
• Use FARM==0.5.0

1. query_encoder files are saved in ../saved_models/dpr-tutorial/lm1
2. passage_encoder files are saved in ../saved_models/dpr-tutorial/lm2
3. prediction_head, tokenizer, vocab details are saved in ../saved_models/

Intializing DPR:

doc_store = ElasticsearchDocumentStore()
retriever = DensePassageRetriever(
                 document_store=doc_store,
                 query_embedding_model="bert-base-uncased",
                 passage_embedding_model="bert-base-uncased",
                 use_gpu=True,
                 batch_size=16,
                 embed_title=True)

Note: remove_sep_tok_from_untitled_passages argument has been removed as TextSimilarityProcessor in FARM encodes untitled passages as [CLS] [SEP] [SEP] <passage_tok_1> <passage_tok_2> ... <passage_tok_n>[SEP] without significant changes to accuracy

sample script to train DPR:

retriever.train(data_dir="data/retriever",
              train_filename="nq-dev.json",
              dev_filename="nq-dev.json",
              test_filename="nq-dev.json",
              batch_size=2,
              embed_title=True,
              num_hard_negatives=1,
              num_negatives=0,
              n_epochs=3,
              evaluate_every=1000,
              n_gpu=1,
              learning_rate=1e-5,
              epsilon=1e-08,
              weight_decay=0.0,
              num_warmup_steps=100,
              grad_acc_steps=1,
              optimizer_name="TransformersAdamW",
              optimizer_correct_bias=True,
              save_dir="../saved_models/dpr-tutorial")

[lalitpagaria]

just suggestion to include corresponding update in DPR tutorials and documentation as well in the task list.

[tholor]

Awesome. Good job! Let's merge it

[mchari]

@tholor, because the Retriever is initialized with a query_embedding_model and document_embedding_model, may I assume that retriever.train() will incrementally train on the set of documents given as input ?
So :

• Retriever is initialized with query_embedding_model, document_embedding_model <- trained on a huge corpus of documents
• incrementally update weights based on a new(much smaller) set of documents.

Is there a sample data file for train_filename(i can use as a template) that is input to train() ? Is it the same as in https://github.com/facebookresearch/DPR/ : Receiver Input data format ?
Thanks!