forked from tmc/langchaingo
-
Notifications
You must be signed in to change notification settings - Fork 0
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
Merge branch 'main' into testcontainers-go
* main: googleai: add initial Vertex (GCP) implementation of Model (tmc#540) docs: add pgvector page
- Loading branch information
Showing
9 changed files
with
612 additions
and
46 deletions.
There are no files selected for viewing
84 changes: 84 additions & 0 deletions
84
docs/docs/modules/data_connection/vector_stores/pgvector.mdx
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,84 @@ | ||
| --- | ||
| sidebar_label: pgvector | ||
| sidebar_position: 1 | ||
| draft: true | ||
| --- | ||
|
|
||
| import CodeBlock from "@theme/CodeBlock"; | ||
| import ExamplePGVector from "@examples/pgvector-vectorstore-example/pgvector_vectorstore_example.go"; | ||
|
|
||
| # Getting Started: pgvector | ||
|
|
||
| [PGVector](https://github.com/pgvector/pgvector) is an open-source vector similarity search for Postgres | ||
|
|
||
| PGVector supports: | ||
| * exact and approximate nearest neighbor search | ||
| * L2 distance, inner product, and cosine distance | ||
| * IVFFlat and HNSW index types | ||
|
|
||
| See the [installation instructions](https://github.com/pgvector/pgvector#installation-notes). | ||
|
|
||
| ## Usage with Langchain Go | ||
|
|
||
| In code, create an embedder based on an LLM (OpenAI, Ollama, etc.): | ||
| ```go | ||
| llm, _:= openai.New() | ||
| emb, _ := embeddings.NewEmbedder(llm) | ||
| ``` | ||
|
|
||
| For OpenAI embeddings, you will need obtain an API key and provide as an environment variable to the program: | ||
|
|
||
| ```bash | ||
| export OPENAI_API_KEY=your_openai_api_key_here | ||
| ``` | ||
|
|
||
| Create a vector store: | ||
| ```go | ||
| ctx := context.Background() | ||
| store, err := pgvector.New( | ||
| ctx, | ||
| pgvector.WithConnectionURL("postgres://testuser:testpass@localhost:5432/testdb?sslmode=disable"), | ||
| pgvector.WithEmbedder(emb), | ||
| ) | ||
| ``` | ||
|
|
||
| Document tables will be created automatically. | ||
|
|
||
| Add documents: | ||
| ```go | ||
| _, err = store.AddDocuments(context.Background(), []schema.Document{ | ||
| { | ||
| PageContent: "Tokyo", | ||
| Metadata: map[string]any{ | ||
| "population": 38, | ||
| "area": 2190, | ||
| }, | ||
| }, | ||
| { | ||
| PageContent: "Sao Paulo", | ||
| Metadata: map[string]any{ | ||
| "population": 22.6, | ||
| "area": 1523, | ||
| }, | ||
| }, | ||
| }) | ||
| ``` | ||
|
|
||
| Run a similarity search using cosine distance (`<=>`): | ||
|
|
||
| ```go | ||
| filter := map[string]any{"area": "1523"} | ||
|
|
||
| docs, err = store.SimilaritySearch(ctx, "only cities in south america", | ||
| 10, | ||
| vectorstores.WithScoreThreshold(0.80), | ||
| vectorstores.WithFilters(filter), | ||
| ) | ||
| ``` | ||
|
|
||
| For now, pgvector integration only supports simple key-value filters and cosine distance search. | ||
|
|
||
| ## Full example | ||
|
|
||
| Here is the entire program (from [pgvector-vectorstore-example](https://github.com/tmc/langchaingo/blob/main/examples/pgvector-vectorstore-example/pgvector_vectorstore_example.go)): | ||
| <CodeBlock language="go">{ExamplePGVector}</CodeBlock> |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Oops, something went wrong.