ElasticVectorSearch: exclude metadata filters not working due to syntax error in filter creation

[choeller]

Checked other resources

• I added a very descriptive title to this issue.
• I searched the LangChain.js documentation with the integrated search.
• I used the GitHub search to find a similar question and didn't find it.
• I am sure that this is a bug in LangChain.js rather than my code.
• The bug is not resolved by updating to the latest stable version of LangChain (or the specific integration package).

Example Code

The following code

const retriever = vectorStore.asRetriever({
      k: input.k ?? 100,
      searchType: 'similarity',
      filter: [
        {
        field: 'gender.keyword',
        value: 'Male',
        operator: 'exclude',
      }]
    });

where vectorStore is an instance of ElasticVectoreStore, leads to the error message:

The following error occured while executing the query: x_content_parse_exception\n\tCaused by:\n\t\tx_content_parse_exception: [1:17197] [bool] failed to parse field [must_not]\n\tRoot causes:\n\t\tparsing_exception: [terms] query does not support [metadata.gender.keyword]

Error Message and Stack Trace (if applicable)

The following error occured while executing the query: x_content_parse_exception\n\tCaused by:\n\t\tx_content_parse_exception: [1:17197] [bool] failed to parse field [must_not]\n\tRoot causes:\n\t\tparsing_exception: [terms] query does not support [metadata.gender.keyword]

Description

I'm using version 0.0.57 of @langchain/community package and I'm using the metadata filtering functionality of the ElasticVectorStore. This works for ordinary terms filters, but throws an error for exclude filters. The reason is, that the method ElasticVectorSearch.buildMetadataTerms contains a syntax error when creating the must_not query part:

In this snippet

if (condition.operator === "exclude") {
        must_not.push({
          terms: {
            [metadataField]: condition.value,
          },
        });
      }

the terms keyword needs to be replaced with term:

if (condition.operator === "exclude") {
        must_not.push({
          term: {
            [metadataField]: condition.value,
          },
        });
      }

see e.g. https://discuss.elastic.co/t/elasticsearch-query-multiple-must-nots/349570/6

System Info

npm version 10.5.0
node version 18.20.2
mac

[dosubot]

Hey there, @choeller! I'm here to help you with any bugs, questions, or contributions you have. Let's squash some bugs together! 🐞

To resolve the syntax error in the ElasticVectorSearch.buildMetadataTerms method of the ElasticVectorStore class, you need to replace the terms keyword with term for the exclude condition. Here is the corrected code snippet:

if (condition.operator === "exclude") {
  must_not.push({
    term: {
      [metadataField]: condition.value,
    },
  });
}

This change ensures that the must_not clause is constructed correctly, avoiding the parsing exception you encountered ^[1].

To continue talking to Dosu, mention @dosu.