Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix: zip corpus files before uploading to s3 #4685

Merged
merged 4 commits into from
Aug 2, 2024
Merged

fix: zip corpus files before uploading to s3 #4685

Changes from 3 commits
Commits
Show all changes
4 commits
Select commit Hold shift + click to select a range
03b402f
zip corpus files
jouho Aug 2, 2024
7fc3c4d
fix syntax
jouho Aug 2, 2024
e5d46ec
start with 0 corpus if zip was not found
jouho Aug 2, 2024
cd72c23
address PR comment
jouho Aug 2, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
14 changes: 10 additions & 4 deletions tests/fuzz/runFuzzTest.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -84,8 +84,11 @@ if [ "$CORPUS_UPLOAD_LOC" != "none" ]; then
# The LD variables interferes with certificate validation when communicating with AWS S3.
unset LD_PRELOAD
unset LD_LIBRARY_PATH
printf "Copying corpus files from S3 bucket...\n"
aws s3 sync $CORPUS_UPLOAD_LOC/${TEST_NAME}/ "${TEMP_CORPUS_DIR}"
if aws s3 ls "${CORPUS_UPLOAD_LOC}/${TEST_NAME}/corpus.zip" > /dev/null 2>&1; then
jmayclin marked this conversation as resolved.
Show resolved Hide resolved
printf "corpus.zip found, downloading from S3 bucket and unzipping...\n"
aws s3 cp "${CORPUS_UPLOAD_LOC}/${TEST_NAME}/corpus.zip" "${TEMP_CORPUS_DIR}/corpus.zip"
unzip -o "${TEMP_CORPUS_DIR}/corpus.zip" -d "${TEMP_CORPUS_DIR}"
fi
)
else
cp -r ./corpus/${TEST_NAME}/. "${TEMP_CORPUS_DIR}"
Expand Down Expand Up @@ -225,8 +228,11 @@ then
unset LD_PRELOAD
unset LD_LIBRARY_PATH
if [ "$CORPUS_UPLOAD_LOC" != "none" ]; then
printf "Uploading corpus files to S3 bucket...\n"
aws s3 sync ./corpus/${TEST_NAME}/ $CORPUS_UPLOAD_LOC/${TEST_NAME}/
printf "Zipping corpus files...\n"
zip -r ./corpus/${TEST_NAME}.zip ./corpus/${TEST_NAME}/

printf "Uploading zipped corpus file to S3 bucket...\n"
aws s3 cp ./corpus/${TEST_NAME}.zip $CORPUS_UPLOAD_LOC/${TEST_NAME}/corpus.zip
fi
fi

Expand Down
Loading