added dhf4 javascript only examples (#1796)

examples/dhf4-with-tests-javascript-only/.gitignore

@@ -0,0 +1,3 @@
+!gradle.properties
+!gradle-local.properties
+!hub-internal-config/

examples/dhf4-with-tests-javascript-only/README.md

@@ -0,0 +1,106 @@
+## Overview
+
+This project shows a basic setup for writing marklogic unit tests for the DHF using __*only javascript*__.
+
+This includes the insertion of the data into the database as well as running harmonization jobs (and testing the output). E.g.
+```javascript
+    const dtu = require('/test/lib/dhfTestUtils.sjs');
+
+    const results = dtu.mlHubRunFlow("Employee","sampleHarmonize",{"entity":"Employee"})
+
+    let testResults =  [
+        test.assertEqual(2, results.totalCount),
+        test.assertEqual(0, results.errorCount)
+    ]
+```
+
+Note that this exmaple __*does not*__ use a dedicated test database. Rather the tests runs using the staging and final databases that you have configured in your gradle-*.properties file
+
+## Trying the project out locally
+
+### Install
+
+To try this out locally, run the following:
+
+    ./gradlew mlDeploy
+
+This will deploy the datahub application and the tests
+
+### Running the tests via web gui -
+
+Then you can go to the following url to view the test web gui - 
+
+    http://server_name:staging_port/test/default.xqy
+
+e.g.
+
+    http://localhost:8010/test/default.xqy
+
+### Running the tests via Junit test - 
+
+    ./gradlew clean test
+
+This will generate the following
+*  JUnit XML tests results at `build\test-results\test`
+*  HTML test report at `build\reports\tests\test\index.html`
+
+## Details
+
+The main configuration / files to be aware of is as follows -
+
+| File / configuration | Details |
+| -------------        | --------|
+| src/test/ml-modules/root/test/lib/dhfTestUtils.sjs | Test utility library that contains useful helper functions for running dhf tests - e.g. `mlHubRunFlow` |
+| src/test/ml-modules/root/test/suites/EmployeeTest/setup.sjs | Setup script for the tests. It will insert the sample data in the staging database|
+| src/test/ml-modules/root/test/suites/EmployeeTest/teardown.sjs | Teardown script for the tests. It will delete the data in the staging and final databases|
+| src/test/ml-modules/root/test/suites/EmployeeTest/testSampleMapping.sjs | Example of how to test the content.sjs createContent mapping functionality for the Employee entity|
+| src/test/ml-modules/root/test/suites/EmployeeTest/testSampleHarmonization.sjs | Example of how to test an harmonization flow for the Employee entity|
+| src/test/java/org/example/RunUnitTestsTest.java | Simple Junit test class that will execute all of the javascript tests and output the results in JUNIT xml format and an html report. __Note__ if you want to run the tests against the __test__ env, you will need to set ```@ContextConfiguration(classes = {TestEnvDataHubTestConfig.class})``` in this java class.|
+| src/test/java/org/example/TestEnvDataHubTestConfig.java | Configuration to use when running tests against the __test__ env |
+| plugins/entities/Employee/* | The Employee entity and harmonization job (sampleHarmonize) configuration and code  |
+| lib/moment.js | A useful date parsing library |
+| build.gradle | The basic gradle build file to make these examples work. Note how the __isDeployUnitTestFramework__ property is used in this build file |
+| gradle.properties | The gradle config properties. The most important ones for this example are <br>mlTestDbName=data-hub-STAGING <br>mlTestPort=8010 <br>mlModulePaths=src/main/ml-modules,src/test/ml-modules <br>isDeployUnitTestFramework=true|
+
+
+## Other useful commands / configuration
+
+### Ensuring tests do not get deployed to production
+
+The file gradle-prod.properties contains the configuration required to ensure that the tests do not test deployed to production
+
+When running the gradle deployment commands, make sure you include the `-PenvironmentName=prod` argment. E.g. -
+
+    ./gradlew mlDeploy -PenvironmentName=prod
+
+### Deploying a dedicated test instance for running your unit tests
+
+The file gradle-test.properties contains the configuration required to setup a dedicated dhf test instance (even on your local host)
+
+When running the gradle deployment commands, make sure you include the `-PenvironmentName=test` argment. E.g. -
+
+    ./gradlew mlDeploy -PenvironmentName=test
+
+And then add ```@ContextConfiguration(classes = {TestEnvDataHubTestConfig.class})``` to the ```src/test/java/org/example/RunUnitTestsTest.java``` file
+
+And then run your tests
+
+    ./gradlew clean test -PenvironmentName=test
+
+## Points to be aware of 
+
+* You can only load the tests by using the gradle commands (the DHF quickstart gui does not load them). Commands you can use to load the tests - :
+
+    ```./gradlew mlReloadModules ``` 
+    or 
+    ```./gradlew mlLoadModules ``` 
+    or 
+    ```./gradlew mlDeploy ```
+or 
+    ```./gradlew mlReDeploy ```
+
+* Be aware that the harmonization job in the test will not generate job and trace documents
+
+* Using the `mlHubRunFlow` command is only intented for running harmonizing a small number of documents for testing purposes. __Do not use this for production code!__
+
+* Removing the ```src/test/ml-modules``` path from the mlModulePaths property will mean that the tests are not loaded

examples/dhf4-with-tests-javascript-only/build.gradle

@@ -0,0 +1,42 @@
+buildscript {
+  repositories {
+    jcenter()
+  }
+  dependencies {
+    classpath "com.marklogic:marklogic-unit-test-client:0.12.0"
+  }
+}
+
+plugins {
+    id "java"
+    id 'net.saliman.properties' version '1.4.6'
+    id 'com.marklogic.ml-data-hub' version '4.1.1'
+}
+
+repositories {
+    jcenter()
+}
+
+dependencies {
+    if ("true".equalsIgnoreCase(project.findProperty("isDeployUnitTestFramework"))) {
+        // Needed to execute tests written using marklogic-unit-test
+        mlRestApi "com.marklogic:marklogic-unit-test-modules:0.12.0"
+    }
+
+    // Supports testing against MarkLogic via JUnit5
+    testCompile "com.marklogic:marklogic-junit:0.11.0"
+
+    // Needed for verifying the deployment in a JUnit5 test
+    testCompile "com.marklogic:ml-app-deployer:3.10.1"
+
+    // Needed by Gradle 4.6+ to run "gradle test"
+    testRuntime "org.junit.jupiter:junit-jupiter-engine:5.3.0"
+}
+
+/**
+ * This is needed in order to run "gradle test" in Gradle 4.6+.
+ * See https://www.petrikainulainen.net/programming/testing/junit-5-tutorial-running-unit-tests-with-gradle/
+ */
+test {
+    useJUnitPlatform()
+}

examples/dhf4-with-tests-javascript-only/gradle-local.properties

@@ -0,0 +1,5 @@
+# Put your overrides from gradle.properties here
+# Don't check this in to version control
+
+ mlUsername=admin
+ mlPassword=admin

examples/dhf4-with-tests-javascript-only/gradle-prod.properties

@@ -0,0 +1,4 @@
+mlTestDbName=
+mlTestPort=
+isDeployUnitTestFramework=false
+mlModulePaths=src/main/ml-modules

examples/dhf4-with-tests-javascript-only/gradle-test.properties

@@ -0,0 +1,65 @@
+# This configures a dedicated test instance for running the unit tests
+
+# Properties for test resources
+mlHost=localhost
+mlTestDbName=test-data-hub-STAGING
+# better to use the STAGING db/server as the context for the tests
+mlTestPort=9010
+isDeployUnitTestFramework=true
+
+
+mlStagingAppserverName=test-data-hub-STAGING
+mlStagingPort=9010
+mlStagingDbName=test-data-hub-STAGING
+mlStagingForestsPerHost=3
+mlStagingAuth=digest
+# To enable SSL for staging
+# mlStagingSimpleSsl=true
+
+mlFinalAppserverName=test-data-hub-FINAL
+mlFinalPort=9011
+mlFinalDbName=test-data-hub-FINAL
+mlFinalForestsPerHost=3
+mlFinalAuth=digest
+# To enable SSL for final
+# mlFinalSimpleSsl=true
+
+
+mlJobAppserverName=test-data-hub-JOBS
+mlJobPort=9013
+mlJobDbName=test-data-hub-JOBS
+mlJobForestsPerHost=4
+mlJobAuth=digest
+# To enable SSL for jobs
+# mlJobSimpleSsl=true
+
+mlModulesDbName=test-data-hub-MODULES
+mlModulesForestsPerHost=1
+
+mlStagingTriggersDbName=test-data-hub-staging-TRIGGERS
+mlStagingTriggersForestsPerHost=1
+
+mlStagingSchemasDbName=test-data-hub-staging-SCHEMAS
+mlStagingSchemasForestsPerHost=1
+
+mlFinalTriggersDbName=test-data-hub-final-TRIGGERS
+mlFinalTriggersForestsPerHost=1
+
+mlFinalSchemasDbName=test-data-hub-final-SCHEMAS
+mlFinalSchemasForestsPerHost=1
+
+# You can override this to specify an alternate folder for your
+# custom forest info. Defaults to user-config/forests/
+# mlCustomForestPath=forests
+
+# The name of the Role to create for Hub Access
+mlHubUserRole=test-data-hub-role
+mlHubUserName=test-data-hub-user
+mlModulePermissions=rest-reader,read,rest-writer,insert,rest-writer,update,rest-extension-user,execute,test-data-hub-role,read,test-data-hub-role,execute
+
+# this password is autogenerated for you via the 'gradle hubInit' task
+mlHubUserPassword=i]::O}$:(cJ`QN('WYB`
+
+
+
+

examples/dhf4-with-tests-javascript-only/gradle.properties

@@ -0,0 +1,137 @@
+# Properties for test resources
+mlTestDbName=data-hub-STAGING
+# better to use the STAGING db/server as the context for the tests
+mlTestPort=8010
+#include the module path for the test suite
+mlModulePaths=src/main/ml-modules,src/test/ml-modules
+#indicate that the 
+isDeployUnitTestFramework=true
+
+
+# These settings are used by the Data Hub Framework when
+# communicating with MarkLogic.
+# The values in this file are meant as project-wide settings.
+# You can override these properties for a specific environment
+# by creating a gradle-{environment}.properties file.
+# For example, to create a properties file for your prod environment create a file
+# named gradle-prod.properties.
+#
+# ....
+mlDHFVersion=4.1.1
+mlHost=localhost
+
+# If you are working with a load balancer please specify it here
+# mlIsHostLoadBalancer=false
+
+# Your MarkLogic Username and Password
+mlUsername=admin
+mlPassword=admin
+
+# If specified, the manage username/password combo is used with the ML Management REST API for managing application
+# resources; this user must have the manage-admin and rest-admin roles.
+#
+# If these are not set, then mlUsername/mlPassword is used for managing application resources.
+# mlManageUsername=
+# mlManagePassword=
+#
+# To change the Manage Port
+# mlManagePort=8002
+#
+# To use SSL on the Manage appserver (port 8002 by default)
+# mlManageScheme=https
+# mlManageSimpleSsl=true
+#
+# If specified, mlSecurityUsername/mlSecurityPassword is used for talking to Security end points on port 8002; this 
+# user must have the "manage-admin" and "security" roles.
+#
+# mlSecurityUsername=
+# mlSecurityPassword=
+#
+# To change the Admin Port
+# mlAdminPort=8001
+#
+# To use SSL on the Admin appserver (port 8001 by default)
+# mlAdminScheme=https
+# mlAdminSimpleSsl=true
+
+
+# If specified, these values can override where the DHF thinks
+# MarkLogic default ports are at. You would only use this if you
+# have changed the ports on which MarkLogic listens
+#
+# mlAppServicesPort=8000
+#
+# To use SSL on the AppServices appserver (port 8000 by default)
+# mlAppServicesSimpleSsl=true
+
+
+mlStagingAppserverName=data-hub-STAGING
+mlStagingPort=8010
+mlStagingDbName=data-hub-STAGING
+mlStagingForestsPerHost=3
+mlStagingAuth=digest
+# To enable SSL for staging
+# mlStagingSimpleSsl=true
+
+mlFinalAppserverName=data-hub-FINAL
+mlFinalPort=8011
+mlFinalDbName=data-hub-FINAL
+mlFinalForestsPerHost=3
+mlFinalAuth=digest
+# To enable SSL for final
+# mlFinalSimpleSsl=true
+
+
+mlJobAppserverName=data-hub-JOBS
+mlJobPort=8013
+mlJobDbName=data-hub-JOBS
+mlJobForestsPerHost=4
+mlJobAuth=digest
+# To enable SSL for jobs
+# mlJobSimpleSsl=true
+
+mlModulesDbName=data-hub-MODULES
+mlModulesForestsPerHost=1
+
+mlStagingTriggersDbName=data-hub-staging-TRIGGERS
+mlStagingTriggersForestsPerHost=1
+
+mlStagingSchemasDbName=data-hub-staging-SCHEMAS
+mlStagingSchemasForestsPerHost=1
+
+mlFinalTriggersDbName=data-hub-final-TRIGGERS
+mlFinalTriggersForestsPerHost=1
+
+mlFinalSchemasDbName=data-hub-final-SCHEMAS
+mlFinalSchemasForestsPerHost=1
+
+# You can override this to specify an alternate folder for your
+# custom forest info. Defaults to user-config/forests/
+# mlCustomForestPath=forests
+
+# The name of the Role to create for Hub Access
+mlHubUserRole=data-hub-role
+mlHubUserName=data-hub-user
+# this password is autogenerated for you via the 'gradle hubInit' task
+mlHubUserPassword=i]::O}$:(cJ`QN('WYB`
+
+# The name of the role to create for hub deployment/development
+mlHubAdminRole=hub-admin-role
+mlHubAdminUserName=hub-admin-user
+mlHubAdminUserPassword=CQCQclX+0M;i|QT=A|3=
+
+# Deprecated property
+# If you are working with a load balancer please indicate so using
+# property "mlIsHostLoadBalancer"
+# When "mlIsHostLoadBalancer" is set to "true", the value specified for "mlHost" will be used as the load balancer.
+# You do not need to explicitly set the value of "mlLoadBalancerHosts" but if you do it must match the value of the property "mlHost"
+# mlLoadBalancerHosts=your-load-balancer-hostname
+
+# Default module permissions which allow data-hub-role to execute flows
+mlModulePermissions=rest-reader,read,rest-writer,insert,rest-writer,update,rest-extension-user,execute,data-hub-role,read,data-hub-role,execute
+
+# If DHF is running in a provisioned environment please specify it here
+# mlIsProvisionedEnvironment=false
+
+
+

examples/dhf4-with-tests-javascript-only/gradle/wrapper/gradle-wrapper.properties

@@ -0,0 +1,5 @@
+distributionBase=GRADLE_USER_HOME
+distributionPath=wrapper/dists
+distributionUrl=https\://services.gradle.org/distributions/gradle-4.10-bin.zip
+zipStoreBase=GRADLE_USER_HOME
+zipStorePath=wrapper/dists