Provide Possibility To Translate Messages In Tests / Assertions

[sprankhub]

Currently, asserting certain messages like the "add to cart" success message is language-specific. The core usually uses the default en_US locale. See e.g. here https://github.com/magento/magento2/blob/0dcb19e498e3563307a511a42933d37cb9af54ca/app/code/Magento/Catalog/Test/Mftf/ActionGroup/StorefrontAddToCartCustomOptionsProductPageActionGroup.xml#L23. This of course fails any time one uses this test for a store in another language. There should be a way to translate such messages into the locale, which is configured for the store the test is run in.

This could be implemented via an additional XML attribute translate="true" for the see XML tag. It may be useful for other tags as well (not sure).

Preconditions

1. Magento 2.3.3
2. MFTF 2.4.5

Steps to reproduce

1. Use the action group StorefrontAddToCartCustomOptionsProductPageActionGroup or write your own action group, which checks if a specific message is displayed.

Expected result

1. The message is checked in the locale, which is configured in the current store view.

Actual result

1. The message is checked in exactly the same way it occurs in the test. No translation is applied at all.

[jilu1]

Hello @sprankhub, Thank you for your request! We've created an internal ticket https://jira.corp.magento.com/browse/MQE-1860 for this feature enhancement. We will keep you updated with progress and feedback through this PR.

[sprankhub]

Any update?

[okolesnyk]

Hi @sprankhub
Currently, we don't have any updates on that issue. We either suggest using one single locale or creating a separate set of data used in tests that are executed against different locales.

[okolesnyk]

Please let me know if that can work for you.

[sprankhub]

This means that I cannot test an e.g. German-only shop with a German locale. I first need to change the locale to EN, so that the tests work. Hence, I do not test the system any more, which I would like to test, but a different one. This does not make much sense to me.

[adamkarnowka]

Any update of this? As author have noticed this is major blocker for having reliable test results - if we cannot assert against store specific texts (not only language-wise, but some texts are simply altered) - then there is no point in such assertion imo.

@sprankhub may I ask you if you got any solution for this or you just skip strings assertion?

[sprankhub]

I think we just skipped this. To be honest, MFTF became a nightmare for us over time. It is very unreliable, hard to set up and breaks way too easy. We'll switch to https://github.com/elgentos/magento2-cypress-testing-suite/ for future projects.