DMSDK Support

Support starting in 3.0.0

Version 3.0.0 of ml-javaclient-util includes support for the new Data Movement SDK in version 4 of the MarkLogic Java Client API.

The main class for this support is QueryBatcherTemplate, a Spring-style Template class that simplifies common operations involving a QueryBatcher. A QueryBatcherTemplate depends on an instance of DatabaseClient, and then you can configure it and call the method that matches your use case. Example:

QueryBatcherTemplate t = new QueryBatcherTemplate(databaseClient);
t.setThreadCount(32); // defaults to 8 
t.setBatchSize(50); // defaults to 100
t.setApplyConsistentSnapshot(true); // defaults to true
t.setAwaitCompletion(true); // defaults to true
t.applyOnCollections(someListener, "collection1", "collection2");
t.applyOnUriPattern(someOtherListener, "**some-pattern*.xml"); // uses cts:uri-match under the hood

In addition to QueryBatcherTemplate, ml-javaclient-util provides implementations of the QueryBatchListener interface for common use cases. Implementations currently exist for adding, removing, setting, and deleting collections, and for adding, removing, and setting permissions.

New in 3.3.0

Version 3.3.0 of ml-javaclient-util adds a new QueryBatcherJob interface to simplify running a job that uses a QueryBatcher. The interface itself is very simple:

QueryBatcherJobTicket run(DatabaseClient client);

Usage should generally look like this:

DatabaseClient client = ... // create this any way you'd like
new AddCollectionsJob("blue", "green").setWhereCollections("red").run(client);

The intent is that an implementation of QueryBatcherJob can be instantiated with its required arguments, and then (if needed) a "setWhere*" method can be used to specify the URIs to select. Then, simply call "run(client)" to run the job. The Job classes will typically reuse the Listener implementations that were originally added in version 3.0.0, with new ones being added in subsequent releases.

Each QueryBatcherJob implementation is likely to extend AbstractQueryBatcherJob, which provides access to number of methods for configuring the job:

new AddCollectionsJob("blue", "green")
  .setApplyConsistentSnapshot(false)
  .setAwaitCompletion(true)
  .setStopAfterCompletion(true)
  .setJobName("my-job")
  .setBatchSize(500)
  .setThreadCount(32)
  .setForestConfig(new ForestConfiguration(...))
  .run(client);

Several "setWhere*" methods are available as well to select URIs, though only one should be used:

new AddCollectionsJob("blue", "green")
  .setWhereUris("doc1", "doc2")
  .setWhereCollections("coll1", "coll2")
  .setWhereUriPattern("/test/*.xml")
  .setWhereUrisQuery("cts:element-value-query(xs:QName('hello'), 'world')");

Version 3.3.0 includes the following jobs:

1. AddCollectionsJob
2. AddPermissionsJob
3. DeleteCollectionsJob
4. ExportToFileJob
5. ExportToZipJob
6. RemoveCollectionsJob
7. RemovePermissionsJob
8. SetCollectionsJob
9. SetPermissionsJob
10. SimpleExportJob (allows for using any Consumer with DMSDK's ExportListener)
11. SimpleQueryBatcherJob (allows for using any QueryBatchListener)

And of course you can create your own class, which is likely to extend AbstractQueryBatcherJob.