Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[NOID] Fixes #4244: Make apoc.dv.* procedures work in clusters (#4281) #4324

Merged
merged 4 commits into from
Jan 16, 2025
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Original file line number Diff line number Diff line change
Expand Up @@ -5,7 +5,7 @@ This file is generated by DocsTest, so don't change it!
= apoc.dv.catalog.add
:description: This section contains reference documentation for the apoc.dv.catalog.add procedure.

label:procedure[] label:apoc-full[]
label:procedure[] label:apoc-full[] label:deprecated[]

[.emphasis]
Add a virtualized resource configuration
Expand All @@ -17,6 +17,8 @@ Add a virtualized resource configuration
apoc.dv.catalog.add(name :: STRING?, config = {} :: MAP?) :: (name :: STRING?, type :: STRING?, url :: STRING?, desc :: STRING?, labels :: LIST? OF STRING?, query :: STRING?, params :: LIST? OF STRING?)
----

include::partial$/dv/deprecated.adoc[]

[WARNING]
====
This procedure is not intended to be used in a cluster environment, and may act unpredictably.
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -5,7 +5,7 @@ This file is generated by DocsTest, so don't change it!
= apoc.dv.catalog.list
:description: This section contains reference documentation for the apoc.dv.catalog.list procedure.

label:procedure[] label:apoc-full[]
label:procedure[] label:apoc-full[] label:deprecated[]

[.emphasis]
List all virtualized resource configuration
Expand All @@ -17,6 +17,8 @@ List all virtualized resource configuration
apoc.dv.catalog.list() :: (name :: STRING?, type :: STRING?, url :: STRING?, desc :: STRING?, labels :: LIST? OF STRING?, query :: STRING?, params :: LIST? OF STRING?)
----

include::partial$/dv/deprecated.adoc[]

== Output parameters
[.procedures, opts=header]
|===
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -5,7 +5,7 @@ This file is generated by DocsTest, so don't change it!
= apoc.dv.catalog.remove
:description: This section contains reference documentation for the apoc.dv.catalog.remove procedure.

label:procedure[] label:apoc-full[]
label:procedure[] label:apoc-full[] label:deprecated[]

[.emphasis]
Remove a virtualized resource config by name
Expand All @@ -17,6 +17,8 @@ Remove a virtualized resource config by name
apoc.dv.catalog.remove(name :: STRING?) :: (name :: STRING?, type :: STRING?, url :: STRING?, desc :: STRING?, labels :: LIST? OF STRING?, query :: STRING?, params :: LIST? OF STRING?)
----

include::partial$/dv/deprecated.adoc[]

[WARNING]
====
This procedure is not intended to be used in a cluster environment, and may act unpredictably.
Expand Down
17 changes: 10 additions & 7 deletions docs/asciidoc/modules/ROOT/pages/virtual-resource/index.adoc
Original file line number Diff line number Diff line change
Expand Up @@ -2,6 +2,8 @@
= Virtual Resource
:description: This chapter describes how to handle external data sources as virtual resource without persisting them in the database

include::partial$systemdbonly.note.adoc[]

[NOTE]
====
There are situations where we would like to enrich/complement the results of a cypher query in a Neo4j graph with additional
Expand Down Expand Up @@ -40,10 +42,11 @@ image::apoc.dv.imported-graph-from-RDB.png[scaledwidth="100%"]
== Managing a Virtualized Resource via JDBC

=== Creating a Virtualized Resource (JDBC)
Before we can query a Virtualized Resource, we need to define it. We do this using the `apoc.dv.catalog.add` procedure.
The procedure takes two parameters:
Before we can query a Virtualized Resource, we need to define it. We do this using the `apoc.dv.catalog.install` procedure.
The procedure takes three parameters:

* a name that uniquely identifies the virtualized resource and can be used to query that resource
* the database name where we want to use the resource (default is `'neo4j'`)
* a set of parameters indicating the type of the resource (type), the access point (url), the parameterised query
that will be run on the access point (query) and the labels that will be applied to the generated virtual nodes (labels).

Expand All @@ -56,7 +59,7 @@ Here is the cypher that creates such virtualized resource:

[source,cypher]
----
CALL apoc.dv.catalog.add("fr-towns-by-dept", {
CALL apoc.dv.catalog.install("fr-towns-by-dept", "neo4j", {
type: "JDBC",
url: "jdbc:postgresql://localhost/communes?user=jb&password=jb",
labels: ["Town","PopulatedPlace"],
Expand Down Expand Up @@ -124,19 +127,19 @@ RETURN path
----

=== Listing the Virtualized Resource Catalog
The apoc.dv.catalog.list procedure returns a list with all the existing Virtualized resources and their descriptions. It takes no parameters.
The apoc.dv.catalog.list procedure returns a list with all the existing Virtualized resources and their descriptions. It accepts one parameter: i.e. the database name where we want to use the resource (default is 'neo4j').

[source,cypher]
----
CALL apoc.dv.catalog.list()
CALL apoc.dv.catalog.show()
----

=== Removing Virtualized Resources from the Catalog
When a Virtualized Resource is no longer needed it can be removed from the catalog by using the apoc.dv.catalog.remove procedure passing as parameter the unique name of the VR.

[source,cypher]
----
CALL apoc.dv.catalog.remove("vr-name")
CALL apoc.dv.catalog.drop("vr-name", <dbName>)
----

=== Export metadata
Expand Down Expand Up @@ -165,7 +168,7 @@ Here is the cypher that creates such virtualized resource:

[source,cypher]
----
CALL apoc.dv.catalog.add("prod-details-by-id", {
CALL apoc.dv.catalog.install("prod-details-by-id", "neo4j", {
type: "CSV",
url: "http://data.neo4j.com/northwind/products.csv",
labels: ["ProductDetails"],
Expand Down
19 changes: 19 additions & 0 deletions docs/asciidoc/modules/ROOT/partials/dv/deprecated.adoc
Original file line number Diff line number Diff line change
@@ -0,0 +1,19 @@
[WARNING]
====
Please note that this procedure is deprecated.

Use the following ones instead, which allow for better support in a cluster:

[opts="header"]
|===
| deprecated procedure | new procedure
| `apoc.dv.catalog.add(<name>, $config)` | `apoc.dv.catalog.install('<name>', '<dbName>', $config)`
| `apoc.dv.catalog.remove('<name>')` | `apoc.dv.catalog.drop('<name>', '<dbName>')`
| `apoc.dv.catalog.list()` | `apoc.dv.catalog.show('<dbName>')`
|===

where `<dbName>` is the database where we want to execute the procedure

xref::virtual-resource/index.adoc[See here for more info].

====
2 changes: 2 additions & 0 deletions full-it/build.gradle
Original file line number Diff line number Diff line change
Expand Up @@ -11,6 +11,8 @@ dependencies {
testImplementation project(':core').sourceSets.test.output
testImplementation project(':full').sourceSets.test.output

testImplementation group: 'us.fatehi', name: 'schemacrawler-mysql', version: '16.20.8'

testImplementation group: 'com.amazonaws', name: 'aws-java-sdk-s3', version: '1.12.770'
testImplementation group: 'org.xmlunit', name: 'xmlunit-core', version: '2.9.1'

Expand Down
Loading
Loading