Bug - Handle java.util.Date datatypes in Mongodb procedures

[InverseFalcon]

We're seeing errors when attempting to get a Mongodb document with a java.util.Date datatype.

An example of usage and errors:

CALL apoc.mongodb.get('mongodb://user:password, 127.0.0.1:27017', 'dbName', 'dbCollection', NULL) YIELD value UNWIND value.row AS Data RETURN Data LIMIT 5

Error encountered is:

"Unpackable value Sat Apr 30 12:27:12 UTC 2016 of type java.util.Date"

This looks similar to an error in a previous mongodb issue, #355 , so we may need to follow the same steps, as the Neo4jPacker doesn't seem to be handling this type correctly. Seems like we'll need to transform this into a Long timestamp.

[jexp]

The problem is that those problematic types can be at any level in the document, also packed or nested in arrays or nested maps.

This problem also only occurs if we want to return exactly those fields via bolt or if we want to store them in properties in the db unconverted.

So whoever knows which fields are there, can use any of the convert methods to turn them into other types (e.g. apoc.convert.toString) or just skip them as needed.

So we would need to traverse every document that we get and check every value if it is potentially offending, which could be quite expensive.

Currenly we only do it explicitely for the top-level objectId.

We could add an optional flag to the procedures that reads compatibleValues=true
and then we might have the easiest and fastest route via converting the whole document to json and reading it back with Jackson.

[albertodelazzari]

@jexp if we do something like that jsonMapper.readValue(jsonMapper.writeValueAsBytes(document), Map.class) where document is our MongoDB document, the "ObjectId", in the resulting Map, will be "decomposed" in all its fields (timestamp, machineidentifier and so on). Now we simply transform the ObjectId to its string representation.
Do we leave different conversions for ObjectId according to compatibleValues or not (converting it always as a string)?

[jexp]

Yep, but only if an optional flag is passed, that triggers the conversion.

We have to watch out though for LONG values that perhaps can't be represented in JSON, at least test that it works.

[albertodelazzari]

@jexp LONG values are converted to INTEGER if the number fits in an INTEGER, this is the default behaviour.
But we can do this: new ObjectMapper().enable(DeserializationFeature.USE_LONG_FOR_INTS) using jackson-databind package instead of jackson-mapper-asl. So we need another dependency for com.fasterxml.jackson.core:jackson-databind:2.9.0. With this "trick" integer numbers are correctly deserialized to LONG.