Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

On test failure, report more context from the logs to the Azure Pipeline report #91

Closed
kmahone opened this issue Dec 14, 2018 · 1 comment
Assignees
Labels
area-TestInfrastructure Issue in the test infrastructure (e.g. in Helix scripts) team-Controls Issue for the Controls team

Comments

@kmahone
Copy link
Member

kmahone commented Dec 14, 2018

When a test failure occurs, we report the results back to Azure Pipelines. The 'Tests' tab of the Pipeline shows the failing tests and an error message for each.

Currently, the error including 'Warnings' and 'Errors' from the test log. Often this is hard to interpret since it is missing so much context. See this test run for an example: https://microsoft.visualstudio.com/WinUI/_build/results?buildId=13356842&view=ms.vss-test-web.test-result-details

I tried including the full test log output, but the Error Message string gets truncated.

It is possible to get to the full test logs on Mission Control, but this requires a bunch of clicking through various links and it also gives you the complete log for the entire run (i.e. every test in the test class).

The best approach would probably be to report the Error Message to Azure Pipeline as the error log output pre-pended with the last few lines of standard output just before the error. This should be fairly easy to add to ConvertWttLogToXUnit.ps1 and would make interpreting the test reports a lot easier.

@jevansaks jevansaks added the area-TestInfrastructure Issue in the test infrastructure (e.g. in Helix scripts) label Dec 17, 2018
@kmahone kmahone self-assigned this Jan 15, 2019
@kmahone
Copy link
Member Author

kmahone commented Jan 15, 2019

Completed: #176

@kmahone kmahone closed this as completed Jan 15, 2019
@jevansaks jevansaks added the team-Controls Issue for the Controls team label Nov 7, 2019
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
area-TestInfrastructure Issue in the test infrastructure (e.g. in Helix scripts) team-Controls Issue for the Controls team
Projects
None yet
Development

No branches or pull requests

2 participants