-
Notifications
You must be signed in to change notification settings - Fork 495
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
[NOID] Fixes #3971: Check how to integrate vector databases via rest …
…APIs (#4059) (#4237) * [NOID] Fixes #3971: Check how to integrate vector databases via rest APIs (#4059) * Fixes #3971: Check how to integrate vector databases via rest APIs * fixed CI errors and removed unused imports * Changes review: added weaviate db, removed vector idx autocreation and vector as a default result * code clean * Changes review: added systemdb store, removed constraint creation * code clean * 2nd changes review * fixed qdrant filename typo and removed info procs from docs * [NOID] solved compile error * [NOID] spotless and licence changes * [NOID] updated license files * [NOID] various changes for 4.4 * [NOID] fixes tests and format-checks
- Loading branch information
Showing
47 changed files
with
4,644 additions
and
646 deletions.
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
157 changes: 157 additions & 0 deletions
157
docs/asciidoc/modules/ROOT/pages/database-integration/vectordb/chroma.adoc
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,157 @@ | ||
|
|
||
| == ChromaDB | ||
|
|
||
| Here is a list of all available ChromaDB procedures, | ||
| note that the list and the signature procedures are consistent with the others, like the Qdrant ones: | ||
|
|
||
| [opts=header, cols="1, 3"] | ||
| |=== | ||
| | name | description | ||
| | apoc.vectordb.chroma.createCollection(hostOrKey, collection, similarity, size, $config) | | ||
| Creates a collection, with the name specified in the 2nd parameter, and with the specified `similarity` and `size`. | ||
| The default endpoint is `<hostOrKey param>/api/v1/collections`. | ||
| | apoc.vectordb.chroma.deleteCollection(hostOrKey, collection, $config) | | ||
| Deletes a collection with the name specified in the 2nd parameter. | ||
| The default endpoint is `<hostOrKey param>/api/v1/collections/<collection param>`. | ||
| | apoc.vectordb.chroma.upsert(hostOrKey, collection, vectors, $config) | | ||
| Upserts, in the collection with the name specified in the 2nd parameter, the vectors [{id: 'id', vector: '<vectorDb>', medatada: '<metadata>'}]. | ||
| The default endpoint is `<hostOrKey param>/api/v1/collections/<collection param>/upsert`. | ||
| | apoc.vectordb.chroma.delete(hostOrKey, collection, ids, $config) | | ||
| Deletes the vectors with the specified `ids`. | ||
| The default endpoint is `<hostOrKey param>/api/v1/collections/<collection param>/delete`. | ||
| | apoc.vectordb.chroma.get(hostOrKey, collection, ids, $config) | | ||
| Gets the vectors with the specified `ids`. | ||
| The default endpoint is `<hostOrKey param>/api/v1/collections/<collection param>/get`. | ||
| | apoc.vectordb.chroma.query(hostOrKey, collection, vector, filter, limit, $config) | | ||
| Retrieve closest vectors from the defined `vector`, `limit` of results, in the collection with the name specified in the 2nd parameter. | ||
| The default endpoint is `<hostOrKey param>/api/v1/collections/<collection param>/query`. | ||
| | apoc.vectordb.chroma.getAndUpdate(hostOrKey, collection, ids, $config) | | ||
| Gets the vectors with the specified `ids`, and optionally creates/updates neo4j entities. | ||
| The default endpoint is `<hostOrKey param>/api/v1/collections/<collection param>/get`. | ||
| | apoc.vectordb.chroma.queryAndUpdate(hostOrKey, collection, vector, filter, limit, $config) | | ||
| Retrieve closest vectors from the defined `vector`, `limit` of results, in the collection with the name specified in the 2nd parameter, and optionally creates/updates neo4j entities. | ||
| The default endpoint is `<hostOrKey param>/api/v1/collections/<collection param>/query`. | ||
| |=== | ||
|
|
||
| where the 1st parameter can be a key defined by the apoc config `apoc.chroma.<key>.host=myHost`. | ||
| With hostOrKey=null, the default is 'http://localhost:8000'. | ||
|
|
||
| === Examples | ||
|
|
||
| .Create a collection (it leverages https://docs.trychroma.com/usage-guide#creating-inspecting-and-deleting-collections[this API]) | ||
| [source,cypher] | ||
| ---- | ||
| CALL apoc.vectordb.chroma.createCollection($host, 'test_collection', 'Cosine', 4, {<optional config>}) | ||
| ---- | ||
|
|
||
|
|
||
| .Delete a collection (it leverages https://docs.trychroma.com/usage-guide#creating-inspecting-and-deleting-collections[this API]) | ||
| [source,cypher] | ||
| ---- | ||
| CALL apoc.vectordb.chroma.deleteCollection($host, '<collection_id>', {<optional config>}) | ||
| ---- | ||
|
|
||
|
|
||
| .Upsert vectors (it leverages https://docs.trychroma.com/usage-guide#adding-data-to-a-collection[this API]) | ||
| [source,cypher] | ||
| ---- | ||
| CALL apoc.vectordb.qdrant.upsert($host, '<collection_id>', | ||
| [ | ||
| {id: 1, vector: [0.05, 0.61, 0.76, 0.74], metadata: {city: "Berlin", foo: "one"}, text: 'ajeje'}, | ||
| {id: 2, vector: [0.19, 0.81, 0.75, 0.11], metadata: {city: "London", foo: "two"}, text: 'brazorf'} | ||
| ], | ||
| {<optional config>}) | ||
| ---- | ||
|
|
||
|
|
||
| .Get vectors (it leverages https://docs.trychroma.com/usage-guide#querying-a-collection[this API]) | ||
| [source,cypher] | ||
| ---- | ||
| CALL apoc.vectordb.chroma.get($host, '<collection_id>', ['1','2'], {<optional config>}), text | ||
| ---- | ||
|
|
||
|
|
||
| .Example results | ||
| [opts="header"] | ||
| |=== | ||
| | score | metadata | id | vector | text | entity | ||
| | null | {city: "Berlin", foo: "one"} | null | null | null | null | ||
| | null | {city: "Berlin", foo: "two"} | null | null | null | null | ||
| | ... | ||
| |=== | ||
|
|
||
|
|
||
| .Get vectors with `{allResults: true}` | ||
| [source,cypher] | ||
| ---- | ||
| CALL apoc.vectordb.chroma.get($host, '<collection_id>', ['1','2'], {<optional config>}), text | ||
| ---- | ||
|
|
||
|
|
||
| .Example results | ||
| [opts="header"] | ||
| |=== | ||
| | score | metadata | id | vector | text | entity | ||
| | null | {city: "Berlin", foo: "one"} | 1 | [...] | ajeje | null | ||
| | null | {city: "Berlin", foo: "two"} | 2 | [...] | brazorf | null | ||
| | ... | ||
| |=== | ||
|
|
||
|
|
||
| .Query vectors (it leverages https://docs.trychroma.com/usage-guide#querying-a-collection[this API]) | ||
| [source,cypher] | ||
| ---- | ||
| CALL apoc.vectordb.chroma.query($host, | ||
| '<collection_id>', | ||
| [0.2, 0.1, 0.9, 0.7], | ||
| {city: 'London'}, | ||
| 5, | ||
| {allResults: true, <optional config>}), text | ||
| ---- | ||
|
|
||
|
|
||
| .Example results | ||
| [opts="header"] | ||
| |=== | ||
| | score | metadata | id | vector | text | ||
| | 1, | {city: "Berlin", foo: "one"} | 1 | [...] | ajeje | ||
| | 0.1 | {city: "Berlin", foo: "two"} | 2 | [...] | brazorf | ||
| | ... | ||
| |=== | ||
|
|
||
|
|
||
| [NOTE] | ||
| ==== | ||
| To optimize performances, we can choose what to `YIELD` with the apoc.vectordb.chroma.query and the `apoc.vectordb.chroma.get` procedures. | ||
| For example, by executing a `CALL apoc.vectordb.chroma.query(...) YIELD metadata, score, id`, the RestAPI request will have an {"include": ["metadatas", "documents", "distances"]}, | ||
| so that we do not return the other values that we do not need. | ||
| ==== | ||
|
|
||
|
|
||
| In the same way as other procedures, we can define a mapping, to fetch the associated nodes and relationships and optionally create them, | ||
| by leveraging the vector metadata. For example: | ||
|
|
||
| .Query vectors | ||
| [source,cypher] | ||
| ---- | ||
| CALL apoc.vectordb.chroma.query($host, '<collection_id>', | ||
| [0.2, 0.1, 0.9, 0.7], | ||
| {}, | ||
| 5, | ||
| { mapping: { | ||
| embeddingKey: "vect", | ||
| nodeLabel: "Test", | ||
| entityKey: "myId", | ||
| metadataKey: "foo" | ||
| } | ||
| }) | ||
| ---- | ||
|
|
||
|
|
||
|
|
||
| .Delete vectors (it leverages https://docs.trychroma.com/usage-guide#deleting-data-from-a-collection[this API]) | ||
| [source,cypher] | ||
| ---- | ||
| CALL apoc.vectordb.chroma.delete($host, '<collection_id>', [1,2], {<optional config>}) | ||
| ---- | ||
|
|
Oops, something went wrong.