Loading custom fine-tuned model in REST-API demo

[Renuk9390]

Dear Community,

Recently I was working with a REST-API demo app.
Therein, I would like to use my custom fine-tuned model (present in my local directory). I have tried changing the model name in "pipelines.haystack-pipeline.yml", but still it is loading the default model (deepset/roberta-base-squad2 ) when the docker is up.

Also, a follow-up question, how can I delete the existing "Capital and Countries" data from Document Store and set up a new Document store?

Thank you very much in advance, any help would be highly appreciated.

[bilgeyucel]

Hi @Renuk9390,

I believe the problem you have is about the docker-compose.yml file. Currently, it is not picking up the changes from your updated pipelines.haystack-pipeline.yml file but uses the image provided in the file as deepset/haystack:cpu-main. There are specific changes that you need to make to change this behavior:

1. Add volumes with the local path to the pipelines.haystack-pipeline.yml file:

   ...
     haystack-api:
       image: "deepset/haystack:cpu-main" 
       volumes:
         - /<path_to_haystack_repo>/haystack/rest_api/rest_api/pipeline:/home/user/rest_api/pipeline
   ...

2. Update the PIPELINE_YAML_PATH value as /home/user/rest_api/pipeline/pipelines.haystack-pipeline.yml

   ...
   haystack-api:
     ...
     environment:
     - DOCUMENTSTORE_PARAMS_HOST=elasticsearch
     - PIPELINE_YAML_PATH=/home/user/rest_api/pipeline/pipelines.haystack-pipeline.yml
   ...

After these changes, your pipelines.haystack-pipeline.yml file should work without any problem.

For the second part of your question, you need to again make changes in docker-compose.yml. In the file, there is again another docker image "deepset/elasticsearch-countries-and-capitals" which contains indexed files. Instead, you can change the image of the elasticsearch and have an empty index for your own files. It should look like this:

...
elasticsearch:
  image: "docker.elastic.co/elasticsearch/elasticsearch:7.9.2"
...

I hope these answers help, feel free to ask if you have any further questions 🙌 As a side note, we are working on a tutorial on "Haystack with REST API" that might be interesting for you 👀

Btw, I am not sure if you're a member but we also have discord if you'd like to see more questions/discussions from the community 🚀

[Renuk9390]

Hi @bilgeyucel,

Thank you very much for your answer. It worked for me.