APOC doesn't read config file on Docker install

[gbhag]

I am using docker Neo 3.2 AND APOC 3.2.0.3 jar. I have been trying to import xml or graphml but no success.

I have changed the neo4j.conf with these
apoc.export.file.enabled=true
apoc.import.file.enabled=true
apoc.import.file.use_neo4j_config=true
dbms.security.procedures.unrestricted=apoc.*
dbms.security.procedures.white_list=apoc.load.*

restarted neo4j

But I keep getting the error
Neo.ClientError.Procedure.ProcedureCallFailed
Failed to invoke procedure apoc.load.xml: Caused by: java.lang.RuntimeException: Import from files not enabled, please set apoc.import.file.enabled=true in your neo4j.conf

Anything I am doing wrong?

I am very excited to use all the apoc procedures which are very useful but kind of stuck in this.

Thanks

[jexp]

Are you on osx? With community edition? Check the readme for the plugin locations please Von meinem iPhone gesendet

…

Am 20.05.2017 um 19:42 schrieb gbhag ***@***.***>: I am using docker Neo 3.2 AND APOC 3.2.0.3 jar. I have been trying to import xml or graphml but no success. I have changed the neo4j.conf with these apoc.export.file.enabled=true apoc.import.file.enabled=true apoc.import.file.use_neo4j_config=true dbms.security.procedures.unrestricted=apoc.* dbms.security.procedures.white_list=apoc.load.* restarted neo4j But I keep getting the error Neo.ClientError.Procedure.ProcedureCallFailed Failed to invoke procedure apoc.load.xml: Caused by: java.lang.RuntimeException: Import from files not enabled, please set apoc.import.file.enabled=true in your neo4j.conf Anything I am doing wrong? I am very excited to use all the apoc procedures which are very useful but kind of stuck in this. Thanks — You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub, or mute the thread.

[gbhag]

Running in docker

Have plugin in $home/plugin directory

Neo4j. Conf is at $home/conf

[jexp]

but you mount them into the docker environment at the right places? Can you share all the details, your call to start it, the config content and what's in your plugin directory.

…

On Sat, May 20, 2017 at 10:31 PM, gbhag ***@***.***> wrote: Running in docker Have plugin in $home/plugin directory Neo4j. Conf is at $home/conf — You are receiving this because you commented. Reply to this email directly, view it on GitHub <#451 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AAEHY8GNcyqosv5L2ltK5MoH0ze6bXwPks5r702LgaJpZM4NhXXQ> .

[gbhag]

The plugin is located at /var/lib/neo4j/plugins/apoc-3.2.0.3-all.jar
There are no other files in the plugins directory

I could see all the apoc procedures from the browser UI. call apoc.*

This is the call I am making - call apoc.load.xml("file:///test.xml")

The conf file is located at /var/lib/neo4j/conf/neo4j.congf
To restart : ./var/lib/bin/neo4j restart

Here is the content (default config file with apoc configs)

#*****************************************************************
# Neo4j configuration
#
# For more details and a complete list of settings, please see
# https://neo4j.com/docs/operations-manual/current/reference/configuration-settings/
#*****************************************************************

# The name of the database to mount
#dbms.active_database=graph.db

# Paths of directories in the installation.
#dbms.directories.data=data
#dbms.directories.plugins=plugins
#dbms.directories.certificates=certificates
#dbms.directories.logs=logs
#dbms.directories.lib=lib
#dbms.directories.run=run

# This setting constrains all `LOAD CSV` import files to be under the `import` directory. Remove or comment it out to
# allow files to be loaded from anywhere in the filesystem; this introduces possible security problems. See the
# `LOAD CSV` section of the manual for details.
#dbms.directories.import=import


apoc.export.file.enabled=true
apoc.import.file.enabled=true
apoc.import.file.use_neo4j_config=true
apoc.trigger.enabled=true
apoc.ttl.enabled=true
dbms.security.procedures.unrestricted=apoc.*
dbms.security.procedures.white_list=apoc.load.*





# Whether requests to Neo4j are authenticated.
# To disable authentication, uncomment this line
dbms.security.auth_enabled=false

# Enable this to be able to upgrade a store from an older version.
#dbms.allow_format_migration=true

# Java Heap Size: by default the Java heap size is dynamically
# calculated based on available system resources.
# Uncomment these lines to set specific initial and maximum
# heap size.
dbms.memory.heap.initial_size=512M
dbms.memory.heap.max_size=512M
# The amount of memory to use for mapping the store files, in bytes (or
# kilobytes with the 'k' suffix, megabytes with 'm' and gigabytes with 'g').
# If Neo4j is running on a dedicated server, then it is generally recommended
# to leave about 2-4 gigabytes for the operating system, give the JVM enough
# heap to hold all your transaction state and query context, and then leave the
# rest for the page cache.
# The default page cache memory assumes the machine is dedicated to running
# Neo4j, and is heuristically set to 50% of RAM minus the max Java heap size.
dbms.memory.pagecache.size=512M

#*****************************************************************
# Network connector configuration
#*****************************************************************

# With default configuration Neo4j only accepts local connections.
# To accept non-local connections, uncomment this line:
dbms.connectors.default_listen_address=0.0.0.0

# You can also choose a specific network interface, and configure a non-default
# port for each connector, by setting their individual listen_address.

# The address at which this server can be reached by its clients. This may be the server's IP address or DNS name, or
# it may be the address of a reverse proxy which sits in front of the server. This setting may be overridden for
# individual connectors below.
#dbms.connectors.default_advertised_address=localhost

# You can also choose a specific advertised hostname or IP address, and
# configure an advertised port for each connector, by setting their
# individual advertised_address.

# Bolt connector
dbms.connector.bolt.enabled=true
#dbms.connector.bolt.tls_level=OPTIONAL
dbms.connector.bolt.listen_address=0.0.0.0:7687

# HTTP Connector. There must be exactly one HTTP connector.
dbms.connector.http.enabled=true
dbms.connector.http.listen_address=0.0.0.0:7474

# HTTPS Connector. There can be zero or one HTTPS connectors.
dbms.connector.https.enabled=true
dbms.connector.https.listen_address=0.0.0.0:7473

# Number of Neo4j worker threads.
#dbms.threads.worker_count=

#*****************************************************************
# Logging configuration
#*****************************************************************

# To enable HTTP logging, uncomment this line
#dbms.logs.http.enabled=true

# Number of HTTP logs to keep.
#dbms.logs.http.rotation.keep_number=5

# Size of each HTTP log that is kept.
#dbms.logs.http.rotation.size=20m

# To enable GC Logging, uncomment this line
#dbms.logs.gc.enabled=true

# GC Logging Options
# see http://docs.oracle.com/cd/E19957-01/819-0084-10/pt_tuningjava.html#wp57013 for more information.
#dbms.logs.gc.options=-XX:+PrintGCDetails -XX:+PrintGCDateStamps -XX:+PrintGCApplicationStoppedTime -XX:+PrintPromotionF

# Number of GC logs to keep.
#dbms.logs.gc.rotation.keep_number=5

# Size of each GC log that is kept.
#dbms.logs.gc.rotation.size=20m

# Size threshold for rotation of the debug log. If set to zero then no rotation will occur. Accepts a binary suffix "k",
# "m" or "g".
#dbms.logs.debug.rotation.size=20m

# Maximum number of history files for the internal log.
#dbms.logs.debug.rotation.keep_number=7

#*****************************************************************
# Miscellaneous configuration
#*****************************************************************

# Enable this to specify a parser other than the default one.
#cypher.default_language_version=3.0

# Determines if Cypher will allow using file URLs when loading data using
# `LOAD CSV`. Setting this value to `false` will cause Neo4j to fail `LOAD CSV`
# clauses that load data from the file system.
dbms.security.allow_csv_import_from_file_urls=true

# Retention policy for transaction logs needed to perform recovery and backups.
dbms.tx_log.rotation.retention_policy=1 days

# Enable a remote shell server which Neo4j Shell clients can log in to.
#dbms.shell.enabled=true
# The network interface IP the shell will listen on (use 0.0.0.0 for all interfaces).
#dbms.shell.host=127.0.0.1
# The port the shell will listen on, default is 1337.
#dbms.shell.port=1337

# Only allow read operations from this Neo4j instance. This mode still requires
# write access to the directory for lock purposes.
#dbms.read_only=false

# Comma separated list of JAX-RS packages containing JAX-RS resources, one
# package name for each mountpoint. The listed package names will be loaded
# under the mountpoints specified. Uncomment this line to mount the
# org.neo4j.examples.server.unmanaged.HelloWorldResource.java from
# neo4j-server-examples under /examples/unmanaged, resulting in a final URL of
# http://localhost:7474/examples/unmanaged/helloworld/{nodeId}
#dbms.unmanaged_extension_classes=org.neo4j.examples.server.unmanaged=/examples/unmanaged

#********************************************************************
# JVM Parameters
#********************************************************************

# G1GC generally strikes a good balance between throughput and tail
# latency, without too much tuning.
dbms.jvm.additional=-XX:+UseG1GC

# Have common exceptions keep producing stack traces, so they can be
# debugged regardless of how often logs are rotated.
dbms.jvm.additional=-XX:-OmitStackTraceInFastThrow

# Make sure that `initmemory` is not only allocated, but committed to
# the process, before starting the database. This reduces memory
# fragmentation, increasing the effectiveness of transparent huge
# pages. It also reduces the possibility of seeing performance drop
# due to heap-growing GC events, where a decrease in available page
# cache leads to an increase in mean IO response time.
# Try reducing the heap memory, if this flag degrades performance.
dbms.jvm.additional=-XX:+AlwaysPreTouch

# Trust that non-static final fields are really final.
# This allows more optimizations and improves overall performance.
# NOTE: Disable this if you use embedded mode, or have extensions or dependencies that may use reflection or
# serialization to change the value of final fields!
dbms.jvm.additional=-XX:+UnlockExperimentalVMOptions
dbms.jvm.additional=-XX:+TrustFinalNonStaticFields

# Disable explicit garbage collection, which is occasionally invoked by the JDK itself.
dbms.jvm.additional=-XX:+DisableExplicitGC
# Remote JMX monitoring, uncomment and adjust the following lines as needed. Absolute paths to jmx.access and
# jmx.password files are required.
# Also make sure to update the jmx.access and jmx.password files with appropriate permission roles and passwords,
# the shipped configuration contains only a read only role called 'monitor' with password 'Neo4j'.
# For more details, see: http://download.oracle.com/javase/8/docs/technotes/guides/management/agent.html
# On Unix based systems the jmx.password file needs to be owned by the user that will run the server,
# and have permissions set to 0600.
# For details on setting these file permissions on Windows see:
#     http://docs.oracle.com/javase/8/docs/technotes/guides/management/security-windows.html
#dbms.jvm.additional=-Dcom.sun.management.jmxremote.port=3637
#dbms.jvm.additional=-Dcom.sun.management.jmxremote.authenticate=true
#dbms.jvm.additional=-Dcom.sun.management.jmxremote.ssl=false
#dbms.jvm.additional=-Dcom.sun.management.jmxremote.password.file=/absolute/path/to/conf/jmx.password
#dbms.jvm.additional=-Dcom.sun.management.jmxremote.access.file=/absolute/path/to/conf/jmx.access

# Some systems cannot discover host name automatically, and need this line configured:
#dbms.jvm.additional=-Djava.rmi.server.hostname=$THE_NEO4J_SERVER_HOSTNAME

# Expand Diffie Hellman (DH) key size from default 1024 to 2048 for DH-RSA cipher suites used in server TLS handshakes
# This is to protect the server from any potential passive eavesdropping.
dbms.jvm.additional=-Djdk.tls.ephemeralDHKeySize=2048

#********************************************************************
# Wrapper Windows NT/2000/XP Service Properties
#********************************************************************
# WARNING - Do not modify any of these properties when an application
#  using this configuration file has been installed as a service.
#  Please uninstall the service before modifying this section.  The
#  service can then be reinstalled.

# Name of the service
dbms.windows_service_name=neo4j

#********************************************************************
# Other Neo4j system properties
#********************************************************************
dbms.jvm.additional=-Dunsupported.dbms.udc.source=tarball
wrapper.java.additional=-Dneo4j.ext.udc.source=docker
ha.host.data=4fc3f199c844:6001
ha.host.coordination=4fc3f199c844:5001
dbms.tx_log.rotation.retention.policy=100M size
causal_clustering.transaction_listen_address=0.0.0.0:6000
causal_clustering.transaction_advertised_address=4fc3f199c844:6000
causal_clustering.raft_listen_address=0.0.0.0:7000
causal_clustering.raft_advertised_address=4fc3f199c844:7000
causal_clustering.discovery_listen_address=0.0.0.0:5000
causal_clustering.discovery_advertised_address=4fc3f199c844:5000

[gbhag]

Any idea, why it is not working. Does the apoc jar needs to be in different location or is there some other config params I am missing?

Thanks

[InverseFalcon]

This looks very similar to a past issue ( #313 ) regarding certain installs where APOC would not read from the config file.

Try executing this...if you get an empty list, then this is likely related to the same issue:

CALL apoc.config.list()

[gbhag]

Tried that earlier. It is an empty list

[InverseFalcon]

Sounds like the same bug as previously referenced.

Would be useful to know if this is 3.2 specific, or if we have the same issue with 3.1.x and its associated APOC version.

In the meantime, it may be useful to rename this issue to
APOC doesn't read config file on Docker install

[gbhag]

I am not sure about previous versions but I have been trying with 3.2. I agree, I can rename the issue as "APOC doesn't read config file on Docker install

[InverseFalcon]

Can you clarify if this is the enterprise or community version?

[gbhag]

community version

[gbhag]

tried neo 3.1.4 with apoc 3.1.3.7 same issue as above not reading the config file.

[gbhag]

Any ideas ?

Thanks

[omarlarus]

Hi @gbhag ,
which configuration file are you editing?
Can you try the following stuff, please?

Download this archive: https://www.dropbox.com/s/85jnk4n3cb7m7p7/apoc_451.tar.gz?dl=0

cp apoc_451.tar.gz /tmp
cd /tmp
tar xzf apoc_451.tar.gz
docker run \
-d \
--rm \
--name=apoc_451 \
--publish=7474:7474 --publish=7687:7687 \
-v /tmp/neo4j/data:/data \
-v /tmp/neo4j/logs:/logs \
-v /tmp/neo4j/plugins:/plugins \
-v /tmp/neo4j/conf/:/conf \
neo4j:3.1.4

Then go to the neo4j browser, setup the password and run

CALL apoc.config.list() yield key, value
where key =~ '.*apoc.*'
return key, value

If you see the configuration properties, you can edit the "/tmp/conf/neo4j.conf" to change the configuration. Otherwise, paste the version of docker and the "/tmp/logs/" contents, please.

[inserpio]

@gbhag ping :) did you solved?

[inserpio]

@jexp, should we close it? @omarlarus described how to solve that above.

[jexp]

Actually it's better to pass in configuration via docker environment variables:
like in here:
http://neo4j.com/docs/operations-manual/current/installation/docker/#docker-environment-variables

[JamesHagerman]

Right, so, I was also having issues with getting APOC installed. It'd be really nice to add some "Docker specific" notes about configuring APOC to it's install docs.

After a bunch of messing around, I basically came up with this:

docker run \
    --publish=7474:7474 --publish=7687:7687 \
    --volume=$PWD/hobbygraph/data:/data \
    --volume=$PWD/plugins:/plugins \
    --name mygraph-apoc-neo4j \
    -e NEO4J_apoc_export_file_enabled=true \
    -e NEO4J_apoc_import_file_enabled=true \
    -e NEO4J_apoc_import_file_use__neo4j__config=true \
    neo4j

Obviously, there are some issues with that ($PWD being the big one) but that at least gets CALL apoc.config.list(); to spit out the APOC configurations in cypher-shell...

After that, I was able to use something like the following (in this case, dump a .cypher format of my data):

CALL apoc.export.cypher.all("/data/hobby-graph.cypher", {format : "cypher-shell", cypherFormat: "create"});

So maybe this ticket is closed... unless there are docs to add about this process?

[jexp]

I add it to the readme.

[jexp]

Done.