Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

ci: refactor fuzz buildspec #4783

Merged
merged 2 commits into from
Sep 19, 2024
Merged

ci: refactor fuzz buildspec #4783

merged 2 commits into from
Sep 19, 2024

Conversation

jouho
Copy link
Contributor

@jouho jouho commented Sep 19, 2024

Resolved issues:

The s2n-tls release is currently blocked due to a fuzz test failure. This is likely caused by recent changes to runFuzzTest.sh, which introduced incompatibilities with the Make build in the Omnibus job.

Failed job: Link to CodeBuild

Instead of fixing the Make build, we can replace it with a CMake-based fuzz build, as introduced in #4743

Description of changes:

  • The Omnibus job runs a batch with a list of identifiers and their associated buildspec files. The fuzz test currently uses a buildspec that runs a batch job, but running a batch job within another batch job can cause issues. Therefore, this PR splits the logic from buildspec_fuzz.yml into two files:
    • buildspec_fuzz_batch.yml to manage the batch job with environment variables for each job.
    • buildspec_fuzz.yml to handle the actual CMake build commands.
  • The -DCMAKE_PREFIX_PATH=$LIBCRYPTO_ROOT flag was not loading the intended libcrypto correctly, and this PR addresses that issue.

Call-outs:

This PR requires changes to both the s2nFuzzBatch and s2nOmnibus jobs:

s2nFuzzBatch: Update the buildspec file from codebuild/spec/buildspec_fuzz.yml to codebuild/spec/buildspec_fuzz_batch.yml.

s2nOmnibus: In the buildspec, replace the following:

    - identifier: s2nFuzzerOpenSSL111Coverage
      buildspec: codebuild/spec/buildspec_ubuntu_fuzz_artifacts.yml
      env:
        privileged-mode: true
        compute-type: BUILD_GENERAL1_LARGE
        image: 024603541914.dkr.ecr.us-west-2.amazonaws.com/docker:ubuntu22codebuild
        variables:
          S2N_LIBCRYPTO: openssl-1.1.1
          LATEST_CLANG: true
          TESTS: fuzz
          FUZZ_TIMEOUT_SEC: 60
          FUZZ_COVERAGE: true

    - identifier: s2nFuzzerOpenSSL102FIPS
      buildspec: codebuild/spec/buildspec_ubuntu_fuzz_artifacts.yml
      env:
        privileged-mode: true
        compute-type: BUILD_GENERAL1_LARGE
        image: 024603541914.dkr.ecr.us-west-2.amazonaws.com/docker:ubuntu22codebuild
        variables:
          S2N_LIBCRYPTO: openssl-1.0.2-fips
          LATEST_CLANG: true
          TESTS: fuzz
          FUZZ_TIMEOUT_SEC: 60

with:

  - identifier: s2nFuzzerAWSLC
    buildspec: codebuild/spec/buildspec_omnibus_fuzz.yml
    debug-session: true
    env:
      compute-type: BUILD_GENERAL1_XLARGE
      image: 024603541914.dkr.ecr.us-west-2.amazonaws.com/docker:ubuntu22codebuild
      privileged-mode: true
      variables:
        S2N_LIBCRYPTO: awslc
        COMPILER: clang

  - identifier: s2nFuzzerOSSL_3_0
    buildspec: codebuild/spec/buildspec_omnibus_fuzz.yml
    debug-session: true
    env:
      compute-type: BUILD_GENERAL1_XLARGE
      image: 024603541914.dkr.ecr.us-west-2.amazonaws.com/docker:ubuntu22codebuild
      privileged-mode: true
      variables:
        S2N_LIBCRYPTO: openssl-3.0
        COMPILER: clang

Testing:

Tested changes in this PR by overriding CodeBuild jobs:

Omnibus job ran with the modified buildspec definition: Link to CodeBuild
(Note that the old fuzz tests have been removed, and the new s2nFuzzerAWSLC and s2nFuzzerOSSL_3_0 jobs have been added to the batch list.)

s2nFuzzBatch job using buildspec_fuzz_batch.yml: Link to CodeBuild

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.

@github-actions github-actions bot added the s2n-core team label Sep 19, 2024
@jouho jouho requested a review from lrstewart September 19, 2024 01:23
@jouho jouho marked this pull request as ready for review September 19, 2024 01:23
@jouho jouho requested a review from dougch as a code owner September 19, 2024 01:23
@jouho jouho requested a review from jmayclin September 19, 2024 01:25
codebuild/spec/buildspec_fuzz_batch.yml Outdated Show resolved Hide resolved
codebuild/spec/buildspec_fuzz_batch.yml Outdated Show resolved Hide resolved
- remove irrelavant comment
- back to using original identifiers
@jouho jouho enabled auto-merge (squash) September 19, 2024 16:45
@lrstewart lrstewart disabled auto-merge September 19, 2024 17:13
@jouho jouho merged commit 0bae2c5 into aws:main Sep 19, 2024
37 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants