-
Notifications
You must be signed in to change notification settings - Fork 1.5k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Fix benchmarking tests #7461
Fix benchmarking tests #7461
Conversation
It's out of scope for this PR, but it looks like there are some functions like |
else | ||
echo "Installed Open MPI version is not less than 5.0.1. Skipping the upgrade." | ||
echo "The installed Open MPI version ($CURRENT_VERSION) is 5.0.1 or higher. Skipping the upgrade." |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
DLFW uses specific version of Open MPI, by mocking and upgrading version within test we cause precedent that we start testing something that is won't be shipped to the customers.
cd $TENSORRTLLM_BACKEND_DIR && git lfs install && git submodule update --init --recursive | ||
} | ||
|
||
# Update Open MPI to a version compatible with SLURM. | ||
function upgrade_openmpi { | ||
cd /tmp/ | ||
local CURRENT_VERSION=$(mpirun --version 2>&1 | awk '/Open MPI/ {gsub(/rc[0-9]+/, "", $NF); print $NF}') | ||
|
||
if [ -n "$CURRENT_VERSION" ] && dpkg --compare-versions "$CURRENT_VERSION" lt "5.0.1"; then | ||
# Uninstall the current version of Open MPI |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I understand that this is due to the SLURM MPI limitation, but agree with Misha's comment that we should be using the exact same version of the one shipped in the container. Can be future enhancement.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
LGTM, great work!
This reverts commit 5320009.
This reverts commit a5ad309.
What does the PR do?
L0_perf_tensorrt_llm
L0_perf_vllm
Checklist
<commit_type>: <Title>
Commit Type:
Check the conventional commit type
box here and add the label to the github PR.
Related PRs:
Where should the reviewer start?
Test plan:
Caveats:
Background
Related Issues: (use one of the action keywords Closes / Fixes / Resolves / Relates to)