Build FX bundled release GP tests using three Payment Managers with TTK as core-connectors

[sri-miriyala]

As an adopter of Mojaloop intending on offering a cross-currency service
I want to make sure that PM4ML is configured to run as FXP, PM4ML as payer, PM4ML as payee
so that I can rely on this functionality once deployed in an IaC environment.

Note: This story does not cover GP testing of Aborted (not timed-out) FX transfers, and fx Quotes

Acceptance Criteria:

• Verify that payer, payee and FXP are configured and deployed as PM4ML in the IaC env, that will be used for testing
• Verify that the existing TTK Rules can be leveraged for TTK with PM4ML
• Verify that the below tests that are part of FXP Golden Path can be run in an IaC deployed environment
  • timeout failures
  • schema failures
  • switch errors
  • fxp errors
  • Payee dfsp errors
  • payer dfsp errors

Complexity: <High|Medium|Low> > A short comment to remind the reason for the rating

Uncertainty: <High|Medium|Low> > A short comment to remind the reason for the rating

Tasks:

• Get Access to environment and run happy path [@vijayg10 ]
• Pull upstream changes from main branch to fx branch and refine the FX related changes [@vijayg10 ] Compared the changes in the feat/fx-impl branch against main branch and observed that the necessary changes are already merged to main. So we don't need fx branch.
• 3 new PM4MLs to be deployed using TTK as core connector, in dev env [@vijayg10 ]
• 3 new PM4MLs to be onboarded in the dev env. [@vijayg10 ]
• Add a provisioning collection for fxp and a sample env file [@vijayg10 ]
• Create helm development versions for FX [@vijayg10 ]
  • Add FX support for SDK helm chart (Pushed a commit to feat: fx and interscheme implementation helm#632)
  • Update SDK in PM4ML helm chart to support FX (feat: fx and interscheme implementation pm4ml/mojaloop-payment-manager-helm#151)
• Trouble shoot the happy path to check if there are any issues [@vijayg10 ]
  • Identified an issue with quoting service validation quoting-service/pull/340
  • Batch processing should be enabled and observed some issue with the fx branch of helm chart. Handled with the following commits
    • mojaloop/helm@657d508
    • mojaloop/helm@0feb52b
    • Update IaC repo for adding support to batch processing of timeout handlers iac-modules/pull/302
  • Version of the central ledger is not updated in batch position handler
  • Fixed an issue with migration scripts for transferParticipant table commit
  • Admin transfer is throwing an error (This is because of old version in the hlem chart, fixed it in the helm fx branch)
• Update scripts to point to PM4ML payer outbound endpoint 1 [@sri-miriyala ]
• Update scripts to point to PM4ML payer outbound endpoint2 [@sri-miriyala ]
• Verify TTK rules needed for Quotes error cases are still working with PM4ML as TTK[@sri-miriyala ]
• Verify TTK rules needed for transfers error cases are still working with PM4ML as TTK[@sri-miriyala ]
• Run Quotes Happy Path Scenario and update scripts as needed [@sri-miriyala ]
• Run Transfers Negative scenarios and update scripts as needed[@sri-miriyala ]
  - [ ] Set 1
  - [ ] Set 2
  - [ ] Set 3
  - [ ] Set 4
  • Set 5
• Run Payer errors
• Run payee error
• Report any issues

Acceptance Criteria pass

• Designs are up-to date
• Unit Tests pass
• Integration Tests pass
• Code Style & Coverage meets standards
• Changes made to config (default.json) are broadcast to team and follow-up tasks added to update helm charts and other deployment config.

[sri-miriyala]

Hey team! Please add your planning poker estimate with Zenhub @vijayg10 @kalinkrustev @geka-evk @oderayi

[vijayg10]

Mojaloop Helm:
mojaloop/helm#632
https://github.com/mojaloop/helm/releases/tag/v16.3.0-snapshot.18
sdk-scheme-adapter: 1.9.0-18461-26ef08b
mojaloop: 16.2.0-18561-510a080

PM4ML Helm
pm4ml/mojaloop-payment-manager-helm#151
https://github.com/pm4ml/mojaloop-payment-manager-helm/releases/tag/v9.2.0-fx.3
mojaloop-payment-manager: 9.2.0

IaC modules:

• feat: add ttk test cases for pm4ml iac-modules#300

TTK Tags
ttk_testcases_tag: 16.1.0-fx-snapshot
onboarding_collection_tag: 16.1.0-fx-snapshot.1

Observations:

• The above are the helm chart releases for mojaloop and payment manger.
• There will be a TTK collection for FXP to add currency accounts it supports. This collection should be run manually at the moment as there is no mechanism in connection manager to specify the participant is FXP and run different onboarding collection.
• The validation is not correct in the quoting service even after the story [FX] Adjust participant account validations in quoting service to support FX #3933. For the payer side currency conversion, the quoting service is validating payer against the currency in the quotes request. But its not correct, as the currency in the quotes request is the target currency in this case.

[vijayg10]

Deployment and Testing Notes:

1. Switch Deployment:

• custom-config/mojaloop-vars.yaml

  mojaloop_chart_version: 16.2.0-18561-510a080
  ttk_testcases_tag: 16.1.0-fx-snapshot
  onboarding_collection_tag: 16.1.0-fx-snapshot.1
  central_ledger_handler_transfer_position_batch_processing_enabled: true

• Use the following versions in custom-config/cluster-config.yaml

  ansible_collection_tag: v5.2.6
  iac_terraform_modules_tag: v5.4.0-fx.1

• custom-config/mojaloop-values-override.yaml

  quoting-service:
    quoting-service:
      ## This is required until the PR https://github.com/mojaloop/helm/pull/636 gets merged to fx branch
      configOverride:
        .QUOTErc: {
          "HUB_PARTICIPANT": {
            "ID": 1,
            "NAME": "Hub"
          }
        }
    quoting-service-handler:
      ## This is required until the PR https://github.com/mojaloop/helm/pull/636 gets merged to fx branch
      configOverride:
        .QUOTErc: {
          "HUB_PARTICIPANT": {
            "ID": 1,
            "NAME": "Hub"
          }
        }
  ## The following is required if we use older version of IaC. This is addressed in the recent PR https://github.com/mojaloop/iac-modules/pull/302
  centralledger:
    centralledger-handler-timeout:
      config:
        batch_processing_enabled: true

2. PM4ML Deployment:

• Use the following versions in the file custom-config/cluster-config.yaml

  ansible_collection_tag: v5.2.6
  iac_terraform_modules_tag: v5.3.5

• custom-config/pm4ml-vars.yaml

  pm4mls:
    - pm4ml: testpayer
      pm4ml_chart_version: 9.2.1-snapshot.1
      pm4ml_dfsp_id: testpayer
      core_connector_selected: ttk
      pm4ml_external_switch_oidc_url: <KEYCLOAK_URL_OF_SWITCH_HERE>
      pm4ml_external_switch_fqdn: <FSPIOP_FQDN_OF_SWITCH_HERE>
      pm4ml_external_mcm_public_fqdn: <MCM_FQDN_OF_SWITCH_HERE>
      supported_currencies: "EUR"
      fxp_id: "testfxp"
    - pm4ml: testpayee
      pm4ml_chart_version: 9.2.1-snapshot.1
      pm4ml_dfsp_id: testpayee
      core_connector_selected: ttk
      pm4ml_external_switch_oidc_url: <KEYCLOAK_URL_OF_SWITCH_HERE>
      pm4ml_external_switch_fqdn: <FSPIOP_FQDN_OF_SWITCH_HERE>
      pm4ml_external_mcm_public_fqdn: <MCM_FQDN_OF_SWITCH_HERE>
      supported_currencies: "CAD"
      fxp_id: "testfxp"
    - pm4ml: testfxp
      pm4ml_chart_version: 9.2.1-snapshot.1
      pm4ml_dfsp_id: testfxp
      core_connector_selected: ttk
      pm4ml_external_switch_oidc_url: <KEYCLOAK_URL_OF_SWITCH_HERE>
      pm4ml_external_switch_fqdn: <FSPIOP_FQDN_OF_SWITCH_HERE>
      pm4ml_external_mcm_public_fqdn: <MCM_FQDN_OF_SWITCH_HERE>
      supported_currencies: "EUR,CAD"
      fxp_id: "testfxp"

3. Onboarding:

• Follow the onboarding instructions and onboard the dfsps testpayer(EUR), testpayee(CAD) and testfxp(EUR) with the currencies mentioned.
• Now we need to further onboard the fxp with the additional currencies. For this use the TTK collection provisioning_fxp/from-switch.json and TTK environment file pm4ml_fxp_provisioning_from_switch.jsonexecute in the switch
• Provision party ID of the payee DFSP using the collection pm4ml/provisioning/sdk_post_accounts.json and with the default environment from the payee TTK. (NOTE: you may get the error callback but the party might be already provisioned. This is because of an issue that is already resolved in recent version of ALS )

4. Test an FX transfer

• Open the payer TTK and load the collection fxp2p (v16.1.0-fx-snapshot.4) from Import from Github option in the collection manager
• Run the collection and you should see all the tests passed
• Observe the position changes in the finance portal