efficiency of remove_ids and updating vector

[Dr1rD]

Hi, I'm using IndexBinaryIVF index, which currently has 300 million vectors. Search, add_with_ids and remove_ids requests happen randomly all the time. My questions are:

1. Remove_ids method is time-consuming. It takes 400ms to remove a vector.
   id_sel = faiss.IDSelectorBatch(id.size, faiss.swig_ptr(id)) index.remove_ids(id_sel)
   Is there a way to improve the efficiency of remove_ids?
2. As far as I know, faiss does not support unique id. So I will use remove_ids before using add_with_ids to achieve the purpose of updating one vector.
   Is there a better way to implement vector updates?

Thanks.

[XueRonger]

I have the same questions.

[mdouze]

As stated in
https://github.com/facebookresearch/faiss/wiki/Special-operations-on-indexes#removing-elements-from-an-index
remove_ids does a full pass over the index, which is why it is so slow.
There is a update_vectors but it is implemented only for IVFFlat.

It is possible to extend the update_vectors to other index types and implement efficient single-element remove. I'll mark this as an enhancement.

[mdouze]

Supported in 1.6.3 to some extent. See
https://github.com/facebookresearch/faiss/wiki/Special-operations-on-indexes#removing-elements-from-an-index