Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

ci: Return custom exit code to indicate known shm leak failure in L0_backend_python bls test #7485

Merged
merged 14 commits into from
Jul 31, 2024

Conversation

krishung5
Copy link
Contributor

@krishung5 krishung5 commented Jul 30, 2024

What does the PR do?

Return custom exit code 123 to indicate known shm leak failure in L0_backend_python bls test. CI will catch the custom exit code and allow the L0_backend_python test to fail if the known issues occur.

In order to propagate the exit code correctly, Python unittest and Pytest can not be used at the same time. In this PR, the python_unittest.py is replaced with test_infer_shm_leak.py which uses Pytest directly.

Checklist

  • PR title reflects the change and is of format <commit_type>: <Title>
  • Changes are described in the pull request.
  • Related issues are referenced.
  • Populated github labels field
  • Added test plan and verified test passes.
  • Verified that the PR passes existing CI.
  • Verified copyright is correct on all changed files.
  • Added succinct git squash message before merging ref.
  • All template sections are filled out.
  • Optional: Additional screenshots for behavior/output changes with before/after.

Commit Type:

Check the conventional commit type
box here and add the label to the github PR.

  • build
  • ci
  • docs
  • feat
  • fix
  • perf
  • refactor
  • revert
  • style
  • test

Related PRs:

tritonserver MR !236.

Where should the reviewer start?

Test plan:

L0_backend_python should fail with allowed to fail if it's failing due to the known shm leak from bls sub test.

  • CI Pipeline ID: 17025308

Caveats:

Background

Related Issues: (use one of the action keywords Closes / Fixes / Resolves / Relates to)

  • closes GitHub issue: #xxx

@krishung5 krishung5 changed the title test: Return custom exit code to indicate known shm leak failure in L0_backend_python bls test ci: Return custom exit code to indicate known shm leak failure in L0_backend_python bls test Jul 30, 2024
@krishung5 krishung5 added PR: ci Changes to our CI configuration files and scripts PR: test Adding missing tests or correcting existing test labels Jul 30, 2024
@@ -151,7 +151,7 @@ apt-get update && apt-get -y install \
libboost-dev
rm -f /usr/bin/python3 && \
ln -s "/usr/bin/python3.${PYTHON_ENV_VERSION}" /usr/bin/python3
pip3 install --upgrade install requests numpy virtualenv protobuf
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nice find 🚀

@@ -441,6 +441,9 @@ def __exit__(self, type, value, traceback):
print(
f"Shared memory leak detected [{shm_region}]: {curr_shm_free_size} (curr free) < {prev_shm_free_size} (prev free)."
)
# Known shared memory leak of 480 bytes in BLS test.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do we have any ticket to eventually find/resolve this issue all together? Would be good to leave the FIXME DLIS-XXXX here if so.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good call, added the ticket reference.

Copy link
Contributor

@rmccorm4 rmccorm4 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM, just a minor comment on ticket reference

@krishung5 krishung5 merged commit e181662 into main Jul 31, 2024
3 checks passed
@krishung5 krishung5 deleted the krish-l0-backend-python branch July 31, 2024 19:52
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
PR: ci Changes to our CI configuration files and scripts PR: test Adding missing tests or correcting existing test
Development

Successfully merging this pull request may close these issues.

3 participants