-
Notifications
You must be signed in to change notification settings - Fork 266
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Test Adapter: parametrized tests are discovered/logged to data collector as 1 test (no information about specific parameters) #340
Comments
@pkruk2 Having separate tests for each datarow is not in plan for the near future. Would it be possible to check the parameter value in your test code (or) does this need to be done outside the test code ? |
@cltshivash It must be outside of the test code. In case I need to add this myself to the test adapter, do you have any suggestions? Ideally it'd be accepted upstream. I understand it'd also change VS behavior, because currently when you discover/run such parametrized test, VS shows 1 test in Test Explorer, and all data rows results are visible in test details section. |
Tagging @pvlakshm , @cltshivash to see if and how can we bring this in? |
@jayaranigarg : Another reason why it would be really nice to have separate test cases is in #171. To be able to run/debug an individual test case would greatly improve dev efficiency especially if these are long running tests. |
@pkruk2 this is by design and it is a non-goal to have discovery by data row. But your ask for DataCollectors to be aware of iterations is a valid ask and can be solved (Given the constraint of not discovering each data row as a separate test method) is for us to add new events for test iteration start and test iteration end for data collectors to hook in to. But this will require a breaking change in the test Platform object model. A simple solution is for us to have custom events in the Test Platform for datacollectors. Anyone who wants to send custom events from their adapters (in our case MsTestv2) can use this Extensible List of events to register and listen to custom events like test iteration start and test iteration end. But this won't be a simple thing to implement. |
* Fix assembly loading in symbol loader. (microsoft#332) Instead of attempting to load the test assembly, try to query for it. It should have been already loaded by the test adapter. In netcoreapp1.1, attempting to load the same executable within itself throws. * Add error tracing for symbol parse failure. * Fix platform tests for objectmodel. * Fix debug type. Copy manifest file and comcomponents to output directory.
Description
Hi,
MsTest tests may be parametrizable, for example a method with DataTestMethod and DataRow attributes. For example:
Current behavior
For such test, MsTest Test Adapter:
Desired behavior
Would it be possible to have per data row discovery/logging resolution? That is, ideally:
In fact, the 2nd point is more important for me - I would like to change environment and monitor each execution of TestMethod (for each data row separately).
If it is not possible, could you please share the reason?
Thanks
Environment
mstest.testadapter.1.2.0
microsoft.testplatform.15.6.0-preview-20171211-02
The text was updated successfully, but these errors were encountered: