Fix #551: Only load content that is newer locally than on the server #552
Conversation
| files.select { |file_uri| | ||
| target_uri = xcc.build_target_uri(file_uri, options) | ||
|
|
||
| q = %Q{"" || xdmp:timestamp-to-wallclock(xdmp:document-timestamp("#{target_uri}"))} |
There was a problem hiding this comment.
Interesting, leverages xdmp:document-timestamp! Won't work on ML5, but does on 6+.
There was a problem hiding this comment.
Maybe I should add some error handling so it will not break on ML5?
Conflicts: deploy/lib/server_config.rb
|
I have now changed it so that it will:
I thought it could be an issue if you have an enormous list of files, but I simulated the XQuery call in qconsole and it was happy checking 10000 document timestamps at once. |
| uris_as_string = uris.map{|i| "\"#{i}\""}.join(",") | ||
| q = %Q{for $u in (#{uris_as_string}) return "" || xdmp:timestamp-to-wallclock(xdmp:document-timestamp($u))} | ||
|
|
||
| result = execute_query q, :db_name => @properties["ml.content-db"] |
There was a problem hiding this comment.
Your query is running against the content database. I didn't get any timestamps back until I changed this to ["ml.modules-db"].
There was a problem hiding this comment.
This function is looking at timestamps on documents in the content database, so isn't that the correct database to be running against?
There was a problem hiding this comment.
Is this being used to deploy content (data) or modules (code)? If you're deploying modules, they are sent to the modules database, so you'll need to check the timestamps there.
There was a problem hiding this comment.
It is only used for "deploy content"
There was a problem hiding this comment.
I think this feature would be very much appreciated for deploy modules as well, at the least. For that it would be useful if it would be more dynamic..
There was a problem hiding this comment.
I could potentially pass in the database name to get_db_timestamps, but it seems like unneeded flexibility at this point. Having incremental deployment of modules doesn't seem useful to me so I will leave that (and making get_db_timestamps more flexible) to someone else.
Fix #551: Only load content that is newer locally than on the server
|
Looks like what was identified previously has been fixed. |
#551
This change adds support for a command line flag that causes "deploy content" to compare modification times on local files and timestamps on deployed documents. It then only deploys files which are new or have newer modification dates locally.
The use case is the situation where the deployed files are a "baseline" set that could potentially change in a running system.
It's not optimized for speed and will be slow if there's a large number of small files, as an individual timestamp query is done for each.