Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[next] nilrt-snac: make test output ptest-compliant #735

Merged
merged 3 commits into from
Sep 20, 2024

Conversation

amstewart
Copy link
Contributor

@amstewart amstewart commented Sep 20, 2024

The NI ptest-parser (and ptests in general) use the AutoMake style of test output. The nilrt-snac integration tests uses pytest formatting, which is generally non-compliant and obscures individual testcase output from being parsed by the RTOS ptesting pipeline.

Use the python3-unittest-automake-output plugin for pytest, to output the test results in format ptest can parse.

AB#2776057

Testing

  • I have built the core package feed with this PR in place. (bitbake packagefeed-ni-core)
  • Tested the new package on a real crio-904x series. New test output looks like this:
admin@NI-cRIO-9049-01CEEE05:/usr/lib/nilrt-snac/ptest# ptest-runner nilrt-snac
START: ptest-runner
2024-09-20T15:09
BEGIN: /usr/lib/nilrt-snac/ptest
PASS: test_cli.py:test_help
PASS: test_cli.py:test_noargs
PASS: test_cli.py:test_verify
PASS: test_cli.py:test_version
PASS: test_installation.py:test_conflicts_ipk
PASS: test_installation.py:test_iptables
PASS: test_installation.py:test_opkg_binary
PASS: test_installation.py:test_supported_distro
PASS: test_installation.py:test_wireguard_conf
============================================================================
Testsuite summary
# TOTAL: 9
# PASS: 9
# SKIP: 0
# XFAIL: 0
# FAIL: 0
# XPASS: 0
# ERROR: 0
DURATION: 4
END: /usr/lib/nilrt-snac/ptest
2024-09-20T15:09
STOP: ptest-runner
TOTAL: 1 FAIL: 0

Procedure

The NI ptest-parser (and ptests in general) use the AutoMake style of
test output. The nilrt-snac integration tests uses pytest formatting,
which is generally non-compliant and obscures individual testcase output
from being parsed by the RTOS ptesting pipeline.

Use the python3-unittest-automake-output plugin for pytest, to output
the test results in format ptest can parse.

Signed-off-by: Alex Stewart <[email protected]>
@amstewart amstewart requested review from jpautler and a team September 20, 2024 15:19
The nirtcfg utility is installed to a non-standard path, for no good
reason. When running the nilrt-snac ptests from an interactive shell,
this nonstandard path is already searched. But when the ptests are run
from an non-interactive shell, they fail to resolve the path.

Add the nonstandard location to the search PATH.

Signed-off-by: Alex Stewart <[email protected]>
Python files use spaces for indentation.

Signed-off-by: Alex Stewart <[email protected]>
@amstewart
Copy link
Contributor Author

Patch V2

  • Fixed whitespace in ptest entrypoint.
  • Added env PATH fixup so that the verify workflow can find the nirtcfg binary when running non-interactively.

@amstewart amstewart merged commit 626231b into ni:nilrt/master/next Sep 20, 2024
@amstewart amstewart deleted the dev/nilrt-snac branch September 20, 2024 21:11
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants