Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Reduce disk usage of the GPU compiler #23

Merged
merged 3 commits into from
Dec 12, 2022
Merged

Reduce disk usage of the GPU compiler #23

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
3 commits
Select commit Hold shift + click to select a range
65dc303
Remove install dir
sfiligoi Dec 12, 2022
97c6d51
Do not unpack things we do not use for unifrac in default mode
sfiligoi Dec 12, 2022
9ee8cc0
Do no hide errors
sfiligoi Dec 12, 2022
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion .github/workflows/main.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -56,7 +56,7 @@ jobs:
which h5c++
if [[ "$(uname -s)" == "Linux" ]];
then
./scripts/install_hpc_sdk.sh >/dev/null 2>/dev/null </dev/null
./scripts/install_hpc_sdk.sh </dev/null
source setup_nv_h5.sh
fi
df -h .
Expand Down
9 changes: 8 additions & 1 deletion scripts/install_hpc_sdk.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -41,6 +41,7 @@ if [ "x${NV_URL}" == "x" ]; then
NV_URL=https://developer.download.nvidia.com/hpc-sdk/21.7/nvhpc_2021_217_Linux_x86_64_cuda_multi.tar.gz
fi

echo "Downloading the NVIDIA HPC SDK"
# Defaults to using curl
# set USE_CURL=N if you want to use aria2 or wget
if [ "x${USE_CURL}" == "x" ]; then
Expand All @@ -51,7 +52,8 @@ if [ "x${USE_CURL}" == "x" ]; then
tar xpzf nvhpc.tgz
rm -f nvhpc.tgz
else
curl "${NV_URL}" | tar xpzf -
# Do not unpack things we do not use for unifrac
curl -s "${NV_URL}" | tar xpzf - --exclude '*libcublas*' --exclude '*libcufft*' --exclude '*libcusparse*' --exclude '*libcusolver*' --exclude '*libcurand*' --exclude '*profilers*' --exclude '*comm_libs*' --exclude '*/doc/*' --exclude '*/plugin*'
fi
elif [ "x${USE_ARIA2}" == "x" ]; then
aria2c "${NV_URL}"
Expand All @@ -63,6 +65,8 @@ else
rm -f nvhpc_*.tar.gz
fi

echo "Installing NVIDIA HPC SDK"

# must patch the install scripts to find the right gcc
for f in nvhpc_*/install_components/install nvhpc_*/install_components/*/*/compilers/bin/makelocalrc nvhpc_*/install_components/install_cuda; do
sed -i -e "s#PATH=/#PATH=$PWD/conda_nv_bins:/#g" $f
Expand Down Expand Up @@ -123,4 +127,7 @@ source $PWD/setup_scripts/setup_nv_hpc_bins.sh
PATH=${PWD}/conda_h5:\$PATH
EOF

# we don't need the install dir anymore
rm -fr nvhpc_*

echo "Setup script avaiabile in $PWD/setup_nv_h5.sh"