add tutorials for cross encoder models on Amazon Bedrock #3278

tkykenmt · 2024-12-15T14:28:21Z

Description

Add tutorials for Amazon Bedrock cross-encoder model (Amazon Rerank 1.0 and Cohere Rerank 3.5)

Related Issues

Resolves #3245

Check List

New functionality includes testing.
New functionality has been documented.
API changes companion pull request created.
Commits are signed per the DCO using --signoff.
Public documentation issue/PR created.

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
For more information on following Developer Certificate of Origin and signing off your commits, please check here.

Signed-off-by: tkykenmt <[email protected]>

…n recieve empty result from a model. Signed-off-by: tkykenmt <[email protected]>

dhrubo-os · 2024-12-15T21:54:31Z

docs/tutorials/rerank/rerank_pipeline_with_Amazon_Rerank_model_on_Amazon_Bedrock.md

+        "content-type": "application/json"
+      },
+      "pre_process_function": """
+        def query_text = params.query_text;


What do you think about raising a PR for pre & post processor functions for Bedrock rerank model like these.

In that way, customer can just mention the name of the pre/post process function rather than writing painless script in the connector payload.

I think built-in pre & post processor is good solution to reduce overhead and ease implementation. I've submitted issues for the topic.

#3254

I'm not familiar with code development, but I'm happy to contribute in this area.

I'm going to implement built-in function in #3254 but please let me leave custom pre and post function as it is in this PR.

docs/tutorials/rerank/rerank_pipeline_with_Amazon_Rerank_model_on_Amazon_Bedrock.md

brianf-aws

I agree with what was authored just minor feedback. Thank you for the contribution!

docs/tutorials/rerank/rerank_pipeline_with_Amazon_Rerank_model_on_Amazon_Bedrock.md

brianf-aws · 2024-12-16T19:32:07Z

docs/tutorials/rerank/rerank_pipeline_with_Amazon_Rerank_model_on_Amazon_Bedrock.md

+    {
+      "action_type": "predict",
+      "method": "POST",
+      "url": "https://bedrock-runtime.${parameters.region}.amazonaws.com/model/amazon.rerank-v1:0/invoke",


Similar feedback like before :)

fixed on 9482b47

brianf-aws · 2024-12-16T19:37:37Z

docs/tutorials/rerank/rerank_pipeline_with_Cohere_Rerank_model_on_Amazon_Bedrock.md

+    {
+      "action_type": "predict",
+      "method": "POST",
+      "url": "https://bedrock-runtime.${parameters.region}.amazonaws.com/model/cohere.rerank-v3-5:0/invoke",


Similar feedback like before

fixed on 9482b47

kolchfa-aws

Some text suggestions for one of the tutorials. Since the text is the same, please apply to both tutorials. Thanks!

kolchfa-aws · 2024-12-23T20:12:47Z

docs/tutorials/rerank/rerank_pipeline_with_Amazon_Rerank_model_on_Amazon_Bedrock.md

+# Topic
+
+[Reranking pipeline](https://opensearch.org/docs/latest/search-plugins/search-relevance/reranking-search-results/) is a feature released in OpenSearch 2.12.
+It can rerank search results, providing a relevance score for each document in the search results with respect to the search query.


Suggested change

It can rerank search results, providing a relevance score for each document in the search results with respect to the search query.

It can rerank search results, providing a relevance score with respect to the search query for each matching document.

Thanks, I'm going to merge your recommendation on next commit.

fixed on 9482b47

kolchfa-aws · 2024-12-23T20:13:06Z

docs/tutorials/rerank/rerank_pipeline_with_Amazon_Rerank_model_on_Amazon_Bedrock.md

+It can rerank search results, providing a relevance score for each document in the search results with respect to the search query.
+The relevance score is calculated by a cross-encoder model. 
+
+This tutorial explains how to use the [Amazon Rerank 1.0 model in Amazon Bedrock](https://docs.aws.amazon.com/bedrock/latest/userguide/rerank-supported.html) in a reranking pipeline. 


Suggested change

This tutorial explains how to use the [Amazon Rerank 1.0 model in Amazon Bedrock](https://docs.aws.amazon.com/bedrock/latest/userguide/rerank-supported.html) in a reranking pipeline.

This tutorial illustrates using the [Amazon Rerank 1.0 model in Amazon Bedrock](https://docs.aws.amazon.com/bedrock/latest/userguide/rerank-supported.html) in a reranking pipeline.

Thanks, I'm going to merge your recommendation on next commit.

fixed on 9482b47

kolchfa-aws · 2024-12-23T20:13:22Z

docs/tutorials/rerank/rerank_pipeline_with_Amazon_Rerank_model_on_Amazon_Bedrock.md

+# Steps
+
+## 0. Test the model on Amazon Bedrock
+You can perform a reranking test with the following code.


Suggested change

You can perform a reranking test with the following code.

You can perform a reranking test using the following code:

Thanks, I'm going to merge your recommendation on next commit.

fixed on 9482b47

kolchfa-aws · 2024-12-23T20:14:45Z

docs/tutorials/rerank/rerank_pipeline_with_Amazon_Rerank_model_on_Amazon_Bedrock.md

+}
+```
+
+If using the Amazon Opensearch Service, you can provide an IAM role arn that allows access to the bedrock service. Refer to this [AWS doc](https://docs.aws.amazon.com/opensearch-service/latest/developerguide/ml-amazon-connector.html)


Suggested change

If using the Amazon Opensearch Service, you can provide an IAM role arn that allows access to the bedrock service. Refer to this [AWS doc](https://docs.aws.amazon.com/opensearch-service/latest/developerguide/ml-amazon-connector.html)

If using the Amazon Opensearch Service, you can provide an IAM role ARN that allows access to the Amazon Bedrock service. For more information, see [AWS documentation](https://docs.aws.amazon.com/opensearch-service/latest/developerguide/ml-amazon-connector.html):

I'll update the sentence based on your recommendation

fixed on 9482b47

kolchfa-aws · 2024-12-23T20:16:21Z

docs/tutorials/rerank/rerank_pipeline_with_Amazon_Rerank_model_on_Amazon_Bedrock.md

+}
+```
+
+Each item in the array comprises a query_text and a text_docs string, separated by a .


This sentence is incomplete. Separated by what?

Suggested change

Each item in the array comprises a query_text and a text_docs string, separated by a .

Each item in the array comprises a `query_text` and a `text_docs` string, separated by a .

" separated by a , ." is correct sentence. I'm going to fix it.

fixed on 9482b47

kolchfa-aws · 2024-12-23T20:17:08Z

docs/tutorials/rerank/rerank_pipeline_with_Amazon_Rerank_model_on_Amazon_Bedrock.md

+
+The connector `pre_process_function` transforms the input into the format required by parameters shown previously.
+
+By default, Amazon Bedrock Rerank API output has the following format:


Suggested change

By default, Amazon Bedrock Rerank API output has the following format:

By default, the Amazon Bedrock Rerank API output has the following format:

I'm going to apply your recommendation.

fixed on 9482b47

kolchfa-aws · 2024-12-23T20:17:38Z

docs/tutorials/rerank/rerank_pipeline_with_Amazon_Rerank_model_on_Amazon_Bedrock.md

+]
+```
+
+The connector `post_process_function` transforms the model's output into a format that the [Reranker processor](https://opensearch.org/docs/latest/search-plugins/search-pipelines/rerank-processor/) can interpret, and orders result by index. This adapted format is as follows:


Suggested change

The connector `post_process_function` transforms the model's output into a format that the [Reranker processor](https://opensearch.org/docs/latest/search-plugins/search-pipelines/rerank-processor/) can interpret, and orders result by index. This adapted format is as follows:

The connector `post_process_function` transforms the model's output into a format that the [Reranker processor](https://opensearch.org/docs/latest/search-plugins/search-pipelines/rerank-processor/) can interpret, and orders the results by index. This adapted format is as follows:

I'm going to apply your recommendation.

fixed on 9482b47

kolchfa-aws · 2024-12-23T20:18:09Z

docs/tutorials/rerank/rerank_pipeline_with_Amazon_Rerank_model_on_Amazon_Bedrock.md

+
+Explanation of the response:
+1. The response contains two `similarity` outputs. For each `similarity` output, the `data` array contains a relevance score of each document against the query.
+2. The `similarity` outputs are provided in the order of the input documents; the first result of similarity pertains to the first document.


Suggested change

2. The `similarity` outputs are provided in the order of the input documents; the first result of similarity pertains to the first document.

2. The `similarity` outputs are provided in the order of the input documents; the first similarity result pertains to the first document.

I'm going to apply your recommendation.

fixed on 9482b47

kolchfa-aws · 2024-12-23T20:18:27Z

docs/tutorials/rerank/rerank_pipeline_with_Amazon_Rerank_model_on_Amazon_Bedrock.md

+}
+```
+
+Note: if you provide multiple filed names in `document_fields`, the values of all fields are first concatenated and then reranking is performed.


Suggested change

Note: if you provide multiple filed names in `document_fields`, the values of all fields are first concatenated and then reranking is performed.

Note: if you provide multiple file names in `document_fields`, the values of all fields are first concatenated and then reranking is performed.

filed is typo. field is correct. I'm going to fix.

fixed on 9482b47

kolchfa-aws · 2024-12-23T20:19:03Z

docs/tutorials/rerank/rerank_pipeline_with_Amazon_Rerank_model_on_Amazon_Bedrock.md

+}
+```
+
+Note: You can avoid writing the query twice by using query_text_path instead of query_text, as follows:


Suggested change

Note: You can avoid writing the query twice by using query_text_path instead of query_text, as follows:

Note: You can avoid writing the query twice by using the `query_text_path` instead of `query_text` as follows:

I'm going to apply your recoomendation.

ylwu-amzn · 2024-12-24T21:15:38Z

docs/tutorials/rerank/rerank_pipeline_with_Amazon_Rerank_model_on_Amazon_Bedrock.md

+    body=body
+)
+results = json.loads(response.get('body').read())["results"]
+print(json.dumps(sorted(results, key=lambda x: x['index']),indent=2))


How about just print the model response directly? Adding sorted may confuse user as this is some processed response.

print(json.dumps(results, indent=2))

Response

[ { "index": 2, "relevance_score": 0.7711548724998493 }, { "index": 0, "relevance_score": 0.0025114635138098534 }, { "index": 1, "relevance_score": 2.4876490010363496e-05 }, { "index": 3, "relevance_score": 6.339210403977635e-06 } ]

Thanks, I'm going to updated based on your recommendation on next commit.

fixed on 9482b47

ylwu-amzn · 2024-12-24T21:30:50Z

docs/tutorials/rerank/rerank_pipeline_with_Amazon_Rerank_model_on_Amazon_Bedrock.md

+
+Test the model by using the Predict API:
+```json
+POST _plugins/_ml/models/your_model_id/_predict


Please make sure every step in the tutorial works. I tested and this step failed

{ "error": { "root_cause": [ { "type": "script_exception", "reason": "compile error", "script_stack": [ "... resultBuilder.append(' {\\\"name\\\": \\\"similarity ...", " ^---- HERE" ], "script": "\n if (params.result == null || params.result.length > 0) {\n throw new IllegalArgumentException(\"Post process function input is empty.\");\n }\n def outputs = params.results;\n def relevance_scores = new Double[outputs.length];\n for (int i=0; i<outputs.length; i++) {\n def index = new BigDecimal(outputs[i].index.toString()).intValue();\n relevance_scores[index] = outputs[i].relevance_score;\n }\n def resultBuilder = new StringBuilder('[');\n for (int i=0; i<relevance_scores.length; i++) {\n resultBuilder.append(' {\\\"name\\\": \\\"similarity\\\", \\\"data_type\\\": \\\"FLOAT32\\\", \\\"shape\\\": [1],');\n resultBuilder.append('\\\"data\\\": [');\n resultBuilder.append(relevance_scores[i]);\n resultBuilder.append(']}');\n if (i<outputs.length - 1) {\n resultBuilder.append(',');\n }\n }\n resultBuilder.append(']');\n return resultBuilder.toString();\n ", "lang": "painless", "position": { "offset": 598, "start": 573, "end": 623 } } ], "type": "m_l_exception", "reason": "m_l_exception: Fail to execute PREDICT in aws connector", "caused_by": { "type": "script_exception", "reason": "compile error", "script_stack": [ "... resultBuilder.append(' {\\\"name\\\": \\\"similarity ...", " ^---- HERE" ], "script": "\n if (params.result == null || params.result.length > 0) {\n throw new IllegalArgumentException(\"Post process function input is empty.\");\n }\n def outputs = params.results;\n def relevance_scores = new Double[outputs.length];\n for (int i=0; i<outputs.length; i++) {\n def index = new BigDecimal(outputs[i].index.toString()).intValue();\n relevance_scores[index] = outputs[i].relevance_score;\n }\n def resultBuilder = new StringBuilder('[');\n for (int i=0; i<relevance_scores.length; i++) {\n resultBuilder.append(' {\\\"name\\\": \\\"similarity\\\", \\\"data_type\\\": \\\"FLOAT32\\\", \\\"shape\\\": [1],');\n resultBuilder.append('\\\"data\\\": [');\n resultBuilder.append(relevance_scores[i]);\n resultBuilder.append(']}');\n if (i<outputs.length - 1) {\n resultBuilder.append(',');\n }\n }\n resultBuilder.append(']');\n return resultBuilder.toString();\n ", "lang": "painless", "position": { "offset": 598, "start": 573, "end": 623 }, "caused_by": { "type": "illegal_argument_exception", "reason": "unexpected character [' {\\\"]. The only valid escape sequences in strings starting with ['] are [\\\\] and [\\'].", "caused_by": { "type": "lexer_no_viable_alt_exception", "reason": "lexer_no_viable_alt_exception: null" } } } }, "status": 500 }

I'm going to remove unnecessary escape character and test again on my side.

fixed on 9482b47

ylwu-amzn · 2024-12-24T22:05:04Z

docs/tutorials/rerank/rerank_pipeline_with_Amazon_Rerank_model_on_Amazon_Bedrock.md

+        def text_docs = params.text_docs;
+        def textDocsBuilder = new StringBuilder('[');
+        for (int i=0; i<text_docs.length; i++) {
+          textDocsBuilder.append('\"');


Suggested change

textDocsBuilder.append('\"');

textDocsBuilder.append('"');

I'm going to remove unnecessary escape character and test again on my side.

fixed on 9482b47

ylwu-amzn · 2024-12-24T22:05:15Z

docs/tutorials/rerank/rerank_pipeline_with_Amazon_Rerank_model_on_Amazon_Bedrock.md

+        for (int i=0; i<text_docs.length; i++) {
+          textDocsBuilder.append('\"');
+          textDocsBuilder.append(text_docs[i]);
+          textDocsBuilder.append('\"');


Suggested change

textDocsBuilder.append('\"');

textDocsBuilder.append('"');

I'm going to remove unnecessary escape character and test again on my side.

fixed on 9482b47

ylwu-amzn · 2024-12-24T22:05:32Z

docs/tutorials/rerank/rerank_pipeline_with_Amazon_Rerank_model_on_Amazon_Bedrock.md

+          }
+        }
+        textDocsBuilder.append(']');
+        def parameters = '{ \"query\": \"' + query_text + '\",  \"documents\": ' + textDocsBuilder.toString() + ' }';


Suggested change

def parameters = '{ \"query\": \"' + query_text + '\", \"documents\": ' + textDocsBuilder.toString() + ' }';

def parameters = '{ "query": "' + query_text + '", "documents": ' + textDocsBuilder.toString() + ' }';

I'm going to remove unnecessary escape character and test again on my side.

fixed on 9482b47

ylwu-amzn · 2024-12-24T22:05:44Z

docs/tutorials/rerank/rerank_pipeline_with_Amazon_Rerank_model_on_Amazon_Bedrock.md

+        }
+        textDocsBuilder.append(']');
+        def parameters = '{ \"query\": \"' + query_text + '\",  \"documents\": ' + textDocsBuilder.toString() + ' }';
+        return  '{\"parameters\": ' + parameters + '}';


Suggested change

return '{\"parameters\": ' + parameters + '}';

return '{"parameters": ' + parameters + '}';

I'm going to remove unnecessary escape character and test again on my side.

fixed on 9482b47

ylwu-amzn · 2024-12-24T22:05:58Z

docs/tutorials/rerank/rerank_pipeline_with_Amazon_Rerank_model_on_Amazon_Bedrock.md

+      """,
+      "request_body": "{ \"query\": \"${parameters.query}\", \"documents\": ${parameters.documents} }",
+      "post_process_function": """
+        if (params.result == null || params.result.length > 0) {


Suggested change

if (params.result == null || params.result.length > 0) {

if (params.results == null || params.results.length == 0) {

I'm going to update condition

fixed on 9482b47

brianf-aws

I think everything looks good just that one part where you keep mentioning SageMaker since you are using bedrock models

brianf-aws · 2024-12-26T22:13:30Z

docs/tutorials/rerank/rerank_pipeline_with_Amazon_Rerank_model_on_Amazon_Bedrock.md

+```
+### 2.2 Create a reranking pipeline
+```json
+PUT /_search/pipeline/rerank_pipeline_sagemaker


Why is written for Sagemaker here but the models are hosted on bedrock. Am I missing something?

Thanks, fixed at a219218

brianf-aws · 2024-12-26T22:13:51Z

docs/tutorials/rerank/rerank_pipeline_with_Cohere_Rerank_model_on_Amazon_Bedrock.md

+```
+### 2.2 Create a reranking pipeline
+```json
+PUT /_search/pipeline/rerank_pipeline_sagemaker


Same here for the name and description

Thanks, fixed at a219218

brianf-aws · 2024-12-26T22:14:42Z

docs/tutorials/rerank/rerank_pipeline_with_Cohere_Rerank_model_on_Amazon_Bedrock.md

+
+Note: You can avoid writing the query twice by using the `query_text_path` instead of `query_text` as follows:
+```json
+POST my-test-data/_search?search_pipeline=rerank_pipeline_sagemaker


Make sure to change it where you write SageMaker too

Thanks, fixed at a219218

…n Bedrock opensearch-project#3278 Signed-off-by: tkykenmt <[email protected]>

ylwu-amzn · 2024-12-27T17:35:31Z

docs/tutorials/rerank/rerank_pipeline_with_Amazon_Rerank_model_on_Amazon_Bedrock.md

+}
+```
+
+Each item in the array comprises a `query_text` and a `text_docs` string, separated by a  ` , `.


Move this to line 295, query_text and a text_docs string exist in the example of

POST _plugins/_ml/_predict/text_similarity/your_model_id

Thank you for your review. Please let me delete this note because it's bit inaccurate. Deleted comment on commit 0a549fb

ylwu-amzn · 2024-12-27T17:37:06Z

docs/tutorials/rerank/rerank_pipeline_with_Amazon_Rerank_model_on_Amazon_Bedrock.md

+```
+
+Explanation of the response:
+1. The response contains two `similarity` outputs. For each `similarity` output, the `data` array contains a relevance score of each document against the query.


Suggested change

1. The response contains two `similarity` outputs. For each `similarity` output, the `data` array contains a relevance score of each document against the query.

1. The response contains four `similarity` outputs. For each `similarity` output, the `data` array contains a relevance score of each document against the query.

Thank you for your review. Modified at commit 0a549fb

_predict call doesn't work in the tutorial. "texts" param should be replaced with "documents" to match the request_body in the connector payload.

POST _plugins/_ml/models/your_model_id/_predict { "parameters": { "query": "What is the capital city of America?", **"texts":** [. ### This should be "documents" "Carson City is the capital city of the American state of Nevada.", "The Commonwealth of the Northern Mariana Islands is a group of islands in the Pacific Ocean. Its capital is Saipan.", "Washington, D.C. (also known as simply Washington or D.C., and officially as the District of Columbia) is the capital of the United States. It is a federal district.", "Capital punishment (the death penalty) has existed in the United States since beforethe United States was a country. As of 2017, capital punishment is legal in 30 of the 50 states." ] } }

…opensearch-project#3278 Signed-off-by: tkykenmt <[email protected]>

docs/tutorials/rerank/rerank_pipeline_with_Amazon_Rerank_model_on_Amazon_Bedrock.md

ylwu-amzn · 2025-01-03T23:02:46Z

docs/tutorials/rerank/rerank_pipeline_with_Amazon_Rerank_model_on_Amazon_Bedrock.md

+  },
+  "parameters": {
+    "service_name": "bedrock",
+    "service_code": "bedrock-runtime",


service_code looks confusing, how about change to endpoint
Check https://docs.aws.amazon.com/bedrock/latest/APIReference/welcome.html

Amazon Bedrock provides the following service endpoints.

bedrock – Contains control plane APIs for managing, training, and deploying models. For more information, see Amazon Bedrock Actions and Amazon Bedrock Data Types.

bedrock-runtime – Contains data plane APIs for making inference requests for models hosted in Amazon Bedrock. For more information, see Amazon Bedrock Runtime Actions and Amazon Bedrock Runtime Data Types.

docs/tutorials/rerank/rerank_pipeline_with_Amazon_Rerank_model_on_Amazon_Bedrock.md

docs/tutorials/rerank/rerank_pipeline_with_Cohere_Rerank_model_on_Amazon_Bedrock.md

Signed-off-by: Yaliang Wu <[email protected]>

docs/tutorials/rerank/rerank_pipeline_with_Amazon_Rerank_model_on_Amazon_Bedrock.md

docs/tutorials/rerank/rerank_pipeline_with_Cohere_Rerank_model_on_Amazon_Bedrock.md

Signed-off-by: Yaliang Wu <[email protected]>

* add tutorials for cross encoder models on Amazon Bedrock Signed-off-by: tkykenmt <[email protected]> * fix bedrock url to reflect region parameter Signed-off-by: tkykenmt <[email protected]> * update error handling to throw exception when post processing function recieve empty result from a model. Signed-off-by: tkykenmt <[email protected]> * fix tutorials for cross encoder models on Amazon Bedrock based on review comments on #3278 Signed-off-by: tkykenmt <[email protected]> * fix wrong service name on tutorials for cross encoder models on Amazon Bedrock #3278 Signed-off-by: tkykenmt <[email protected]> * fix some notes on tutorials for cross encoder models on Amazon Bedrock #3278 Signed-off-by: tkykenmt <[email protected]> * some minor fix Signed-off-by: Yaliang Wu <[email protected]> * change cross-encoder to rerank Signed-off-by: Yaliang Wu <[email protected]> --------- Signed-off-by: tkykenmt <[email protected]> Signed-off-by: Yaliang Wu <[email protected]> Co-authored-by: Yaliang Wu <[email protected]> (cherry picked from commit 32a5788)

* add tutorials for cross encoder models on Amazon Bedrock Signed-off-by: tkykenmt <[email protected]> * fix bedrock url to reflect region parameter Signed-off-by: tkykenmt <[email protected]> * update error handling to throw exception when post processing function recieve empty result from a model. Signed-off-by: tkykenmt <[email protected]> * fix tutorials for cross encoder models on Amazon Bedrock based on review comments on #3278 Signed-off-by: tkykenmt <[email protected]> * fix wrong service name on tutorials for cross encoder models on Amazon Bedrock #3278 Signed-off-by: tkykenmt <[email protected]> * fix some notes on tutorials for cross encoder models on Amazon Bedrock #3278 Signed-off-by: tkykenmt <[email protected]> * some minor fix Signed-off-by: Yaliang Wu <[email protected]> * change cross-encoder to rerank Signed-off-by: Yaliang Wu <[email protected]> --------- Signed-off-by: tkykenmt <[email protected]> Signed-off-by: Yaliang Wu <[email protected]> Co-authored-by: Yaliang Wu <[email protected]> (cherry picked from commit 32a5788) Co-authored-by: Takayuki Enomoto <[email protected]>

tkykenmt and others added 5 commits December 3, 2024 14:41

add tutorials for cross encoder models on Amazon Bedrock

af61b9d

Signed-off-by: tkykenmt <[email protected]>

fix bedrock url to reflect region parameter

ecef600

Signed-off-by: tkykenmt <[email protected]>

Merge branch 'opensearch-project:main' into tutorial-rerank-bedrock

0db3484

Merge branch 'opensearch-project:main' into tutorial-rerank-bedrock

3ec0772

update error handling to throw exception when post processing functio…

8183b42

…n recieve empty result from a model. Signed-off-by: tkykenmt <[email protected]>

tkykenmt requested review from b4sjoo, dhrubo-os, jngz-es, model-collapse, rbhavna, ylwu-amzn, zane-neo, Zhangxunmt, austintlee, HenryL27 and xinyual as code owners December 15, 2024 14:28

tkykenmt temporarily deployed to ml-commons-cicd-env-require-approval December 15, 2024 14:28 — with GitHub Actions Inactive

tkykenmt had a problem deploying to ml-commons-cicd-env-require-approval December 15, 2024 14:28 — with GitHub Actions Failure

dhrubo-os reviewed Dec 15, 2024

View reviewed changes

tkykenmt temporarily deployed to ml-commons-cicd-env-require-approval December 16, 2024 17:40 — with GitHub Actions Inactive

mingshl reviewed Dec 16, 2024

View reviewed changes

docs/tutorials/rerank/rerank_pipeline_with_Amazon_Rerank_model_on_Amazon_Bedrock.md Show resolved Hide resolved

brianf-aws suggested changes Dec 16, 2024

View reviewed changes

kolchfa-aws reviewed Dec 23, 2024

View reviewed changes

ylwu-amzn reviewed Dec 24, 2024

View reviewed changes

tkykenmt requested review from kolchfa-aws, ylwu-amzn, brianf-aws and mingshl December 25, 2024 14:47

tkykenmt temporarily deployed to ml-commons-cicd-env-require-approval December 26, 2024 18:09 — with GitHub Actions Inactive

brianf-aws suggested changes Dec 26, 2024

View reviewed changes

fix wrong service name on tutorials for cross encoder models on Amazo…

a219218

…n Bedrock opensearch-project#3278 Signed-off-by: tkykenmt <[email protected]>

tkykenmt had a problem deploying to ml-commons-cicd-env-require-approval December 27, 2024 02:45 — with GitHub Actions Failure

tkykenmt temporarily deployed to ml-commons-cicd-env-require-approval December 27, 2024 02:45 — with GitHub Actions Inactive

tkykenmt temporarily deployed to ml-commons-cicd-env-require-approval December 27, 2024 16:54 — with GitHub Actions Inactive

ylwu-amzn reviewed Dec 27, 2024

View reviewed changes

fix some notes on tutorials for cross encoder models on Amazon Bedrock …

0a549fb

…opensearch-project#3278 Signed-off-by: tkykenmt <[email protected]>

tkykenmt had a problem deploying to ml-commons-cicd-env-require-approval December 29, 2024 14:28 — with GitHub Actions Failure

ylwu-amzn reviewed Jan 3, 2025

View reviewed changes

some minor fix

e4c55ce

Signed-off-by: Yaliang Wu <[email protected]>

ylwu-amzn had a problem deploying to ml-commons-cicd-env-require-approval January 3, 2025 23:31 — with GitHub Actions Failure

ylwu-amzn reviewed Jan 3, 2025

View reviewed changes

change cross-encoder to rerank

72643da

Signed-off-by: Yaliang Wu <[email protected]>

ylwu-amzn had a problem deploying to ml-commons-cicd-env-require-approval January 3, 2025 23:37 — with GitHub Actions Failure

ylwu-amzn approved these changes Jan 3, 2025

View reviewed changes

ylwu-amzn added the backport 2.x label Jan 3, 2025

rbhavna approved these changes Jan 3, 2025

View reviewed changes

ylwu-amzn merged commit 32a5788 into opensearch-project:main Jan 3, 2025
5 of 7 checks passed

opensearch-trigger-bot bot mentioned this pull request Jan 3, 2025

[Backport 2.x] add tutorials for cross encoder models on Amazon Bedrock #3332

Merged

	It can rerank search results, providing a relevance score for each document in the search results with respect to the search query.
	It can rerank search results, providing a relevance score with respect to the search query for each matching document.

	This tutorial explains how to use the [Amazon Rerank 1.0 model in Amazon Bedrock](https://docs.aws.amazon.com/bedrock/latest/userguide/rerank-supported.html) in a reranking pipeline.
	This tutorial illustrates using the [Amazon Rerank 1.0 model in Amazon Bedrock](https://docs.aws.amazon.com/bedrock/latest/userguide/rerank-supported.html) in a reranking pipeline.

	You can perform a reranking test with the following code.
	You can perform a reranking test using the following code:

	If using the Amazon Opensearch Service, you can provide an IAM role arn that allows access to the bedrock service. Refer to this [AWS doc](https://docs.aws.amazon.com/opensearch-service/latest/developerguide/ml-amazon-connector.html)
	If using the Amazon Opensearch Service, you can provide an IAM role ARN that allows access to the Amazon Bedrock service. For more information, see [AWS documentation](https://docs.aws.amazon.com/opensearch-service/latest/developerguide/ml-amazon-connector.html):

	Each item in the array comprises a query_text and a text_docs string, separated by a .
	Each item in the array comprises a `query_text` and a `text_docs` string, separated by a .


		The connector `pre_process_function` transforms the input into the format required by parameters shown previously.

		By default, Amazon Bedrock Rerank API output has the following format:

	By default, Amazon Bedrock Rerank API output has the following format:
	By default, the Amazon Bedrock Rerank API output has the following format:

	The connector `post_process_function` transforms the model's output into a format that the [Reranker processor](https://opensearch.org/docs/latest/search-plugins/search-pipelines/rerank-processor/) can interpret, and orders result by index. This adapted format is as follows:
	The connector `post_process_function` transforms the model's output into a format that the [Reranker processor](https://opensearch.org/docs/latest/search-plugins/search-pipelines/rerank-processor/) can interpret, and orders the results by index. This adapted format is as follows:

	2. The `similarity` outputs are provided in the order of the input documents; the first result of similarity pertains to the first document.
	2. The `similarity` outputs are provided in the order of the input documents; the first similarity result pertains to the first document.

	Note: if you provide multiple filed names in `document_fields`, the values of all fields are first concatenated and then reranking is performed.
	Note: if you provide multiple file names in `document_fields`, the values of all fields are first concatenated and then reranking is performed.

	Note: You can avoid writing the query twice by using query_text_path instead of query_text, as follows:
	Note: You can avoid writing the query twice by using the `query_text_path` instead of `query_text` as follows:

	def parameters = '{ \"query\": \"' + query_text + '\", \"documents\": ' + textDocsBuilder.toString() + ' }';
	def parameters = '{ "query": "' + query_text + '", "documents": ' + textDocsBuilder.toString() + ' }';

	return '{\"parameters\": ' + parameters + '}';
	return '{"parameters": ' + parameters + '}';

	if (params.result == null \|\| params.result.length > 0) {
	if (params.results == null \|\| params.results.length == 0) {

	1. The response contains two `similarity` outputs. For each `similarity` output, the `data` array contains a relevance score of each document against the query.
	1. The response contains four `similarity` outputs. For each `similarity` output, the `data` array contains a relevance score of each document against the query.

add tutorials for cross encoder models on Amazon Bedrock #3278

add tutorials for cross encoder models on Amazon Bedrock #3278

Conversation

tkykenmt commented Dec 15, 2024

Description

Related Issues

Check List

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

brianf-aws left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

kolchfa-aws left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

ylwu-amzn Dec 24, 2024 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

tkykenmt Dec 25, 2024 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

brianf-aws left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

ylwu-amzn Dec 24, 2024 •

edited

Loading

tkykenmt Dec 25, 2024 •

edited

Loading