Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Update CRT submodules to latest releases #692

Merged
merged 1 commit into from
Jan 8, 2024

Conversation

dannycjones
Copy link
Contributor

@dannycjones dannycjones commented Jan 5, 2024

Description of change

This pulls in the latest tagged releases of AWS CRT dependencies.

aws-c-auth, aws-c-common, and aws-c-s3 were all interesting, with a breaking change in aws-c-common for aws_uri type. Request telemetry callback was improved and no longer has the caveat that it may be called by HTTP IO threads.

There are some new features however they are excluded for this change. There are also some improvements/optimizations of interest. Here's some I picked out:

The full list of changes is quite long:

Submodule mountpoint-s3-crt-sys/crt/aws-c-auth baeffa7..3d9f13f:
  > Add an option to disable environment credentials provider in the default chain (#222)
  > Change `port` from uint16_t to uint32_t, to support VSOCK (#220)
  > Fix error when tests run in parallel. (#221)
Submodule mountpoint-s3-crt-sys/crt/aws-c-common 80f21b3..8eaa098:
  > Change `port` from uint16_t to uint32_t, to support VSOCK (#1079)
  > add a query string helper from byte cursor directly (#1080)
Submodule mountpoint-s3-crt-sys/crt/aws-c-http a082f8a..6a1c157:
  > Change `port` from uint16_t to uint32_t, to support VSOCK (#457)
  > correct the doc about unactivated stream and add test (#460)
  > Add support for cancel stream (#458)
  > sizeof('c')=4, not 1: fix overallocation (#459)
Submodule mountpoint-s3-crt-sys/crt/aws-c-io df64f57..749c87e:
  > Change `port` from uint16_t to uint32_t, to support VSOCK (#613)
Submodule mountpoint-s3-crt-sys/crt/aws-c-s3 fcd7a10..1dd55be:
  > Change `port` from uint16_t to uint32_t, to support VSOCK (#394)
  > Fix get-object with partNumber (#401)
  > Skip headRequest for RangeGet Part 1 (#389)
  > fix two bugs from metrics and cancel (#399)
  > Auto - Update S3 Ruleset & Partition (#400)
  > Metric callback improve (#390)
  > Cancel/Pause stream optimization  (#395)
  > support meta request level override of part size and mpu threshold  (#393)
Submodule mountpoint-s3-crt-sys/crt/aws-lc 444e478..dc4e281:
  > Release v1.19.0 (#1365)
  > Fix issue with older CMake when building FIPS static using Ninja build system (#1362)
  > Bump AWSLC_API_VERSION for EVP_chacha20_poly1305 (#1358)
  > Merge pull request #1356 from nebeid/upstream-merge-2023-12-11
  > 'apt-get update' before install (#1363)
  > Update function to avoid division (#1360)
  > Add print helper functions for RSA and DSA (#1354)
  > Add helper functions needed for NTP (#1355)
  > Add gRPC integration tests (#1265)
  > Fix accidental deletion in f3e49b01 (#1357)
  > Improve tcpdump support + integ test (#1351)
  > Add EVP_chacha20_poly1305() to speed.cc (#1339)
  > Initialize min/max use_default values (#1350)
  > Enable Arm SHA384 proofs in AWS-LC CI (#1348)
  > Mark bcm_redirector functions as local (#1352)
  > Merge pull request #1331 from dkostic/upstream-merge-2023-11-30
  > More caution with 'rm -rf' (#1346)
  > Enable 25519 s2n-bignum backend for FIPS build (#1344)
  > Dockerfile and structure change for SAW proofs on X86_64 (#1345)
  > Add support and CI for building with librelp (#1317)
  > Factor out the machine-optimised backend for 25519 algorithms (#1340)
  > Relax test pinging unauthorized OCSP responder (#1343)
  > Update OCSP responder integration test timeout (#1341)
  > Expand PEM/X509 test coverage (#1327)
  > Read fips scope marker addresses using adrp instead of adr to increase reach (#1332)
  > Implement support for EVP_chacha20_poly1305 (#1311)
Submodule mountpoint-s3-crt-sys/crt/s2n-tls 965bde2..a9a07a2:
  > bindings: match tcp EOF behavior (#4323)
  > (docs): Reordered and moved usage guide into an mdbook (#4300)
  > ktls: add method to enable TLS1.3 (#4331)
  > ci: fix flaky interning test (#4334)
  > Add CBMC proof for s2n_stuffer_printf (#4309)
  > docs: remove gitter references (#4332)
  > ktls: handle TLS1.3 key limits (#4318)
  > ci: pin home crate to fix rust build (#4330)
  > ci: switch autopep8 action (#4322)
  > ci: ignore cbmc prereleases (#4328)
  > ci: switch FreeBSD back to vmactions (#4326)
  > ktls: add TLS1.3 support (#4314)
  > ci: fix pep8 linting (#4319)
  > cleanup: add getter for sequence number (#4317)
  > Mark inline asm output as earlyclobber (#4310)
  > bindings: release rust bindings 0.1.0 (#4313)
  > ci: add workflow for rust bench crate (#4210)
  > Enforce security rules on security policies (#4311)
  > documentation: fix security policy table (#4304)

Relevant issues: N/A

Does this change impact existing behavior?

There are minor interface changes impacting consumers of mountpoint-s3-crt.

There are no expected breaking changes to Mountpoint file connector.


By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license and I agree to the terms of the Developer Certificate of Origin (DCO).

@dannycjones dannycjones temporarily deployed to PR integration tests January 5, 2024 14:55 — with GitHub Actions Inactive
@dannycjones dannycjones had a problem deploying to PR integration tests January 5, 2024 14:55 — with GitHub Actions Failure
@dannycjones dannycjones temporarily deployed to PR integration tests January 5, 2024 14:55 — with GitHub Actions Inactive
@dannycjones dannycjones temporarily deployed to PR integration tests January 5, 2024 14:55 — with GitHub Actions Inactive
@dannycjones dannycjones temporarily deployed to PR integration tests January 5, 2024 14:55 — with GitHub Actions Inactive
@dannycjones dannycjones temporarily deployed to PR integration tests January 5, 2024 14:55 — with GitHub Actions Inactive
@dannycjones dannycjones temporarily deployed to PR integration tests January 5, 2024 14:55 — with GitHub Actions Inactive
@dannycjones dannycjones added the performance PRs to run benchmarks on label Jan 5, 2024
Copy link
Contributor

@passaro passaro left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM!

@dannycjones dannycjones had a problem deploying to PR integration tests January 8, 2024 09:33 — with GitHub Actions Failure
@dannycjones dannycjones temporarily deployed to PR integration tests January 8, 2024 09:33 — with GitHub Actions Inactive
@dannycjones dannycjones temporarily deployed to PR integration tests January 8, 2024 09:33 — with GitHub Actions Inactive
@dannycjones dannycjones enabled auto-merge January 8, 2024 09:33
@dannycjones dannycjones temporarily deployed to PR integration tests January 8, 2024 09:33 — with GitHub Actions Inactive
@dannycjones dannycjones temporarily deployed to PR integration tests January 8, 2024 09:33 — with GitHub Actions Inactive
@dannycjones dannycjones temporarily deployed to PR integration tests January 8, 2024 09:33 — with GitHub Actions Inactive
@dannycjones dannycjones temporarily deployed to PR integration tests January 8, 2024 09:33 — with GitHub Actions Inactive
@dannycjones dannycjones temporarily deployed to PR integration tests January 8, 2024 12:05 — with GitHub Actions Inactive
@dannycjones dannycjones temporarily deployed to PR integration tests January 8, 2024 12:05 — with GitHub Actions Inactive
@dannycjones dannycjones temporarily deployed to PR integration tests January 8, 2024 12:05 — with GitHub Actions Inactive
@dannycjones dannycjones temporarily deployed to PR integration tests January 8, 2024 12:05 — with GitHub Actions Inactive
@dannycjones dannycjones temporarily deployed to PR integration tests January 8, 2024 12:05 — with GitHub Actions Inactive
@dannycjones dannycjones temporarily deployed to PR integration tests January 8, 2024 12:05 — with GitHub Actions Inactive
@dannycjones dannycjones temporarily deployed to PR integration tests January 8, 2024 12:05 — with GitHub Actions Inactive
@dannycjones dannycjones added this pull request to the merge queue Jan 8, 2024
Merged via the queue into awslabs:main with commit 261257b Jan 8, 2024
26 checks passed
@dannycjones dannycjones deleted the update-crt branch January 8, 2024 12:40
passaro added a commit to passaro/mountpoint-s3 that referenced this pull request Feb 5, 2025
Submodule mountpoint-s3-crt-sys/crt/aws-c-auth 5bc67797..b513db4b:
  > A bunch of CMake fixes (awslabs#258)
  > Add Account Id to Credentials (awslabs#260)
  > Skip Transfer-Encoding from signing (awslabs#261)
Submodule mountpoint-s3-crt-sys/crt/aws-c-cal fbbe2612..7299c6ab:
  > Fix Findcrypto.cmake (awslabs#205)
  > A bunch of CMake fixes (awslabs#203)
  > Switch CI to use roles (awslabs#202)
Submodule mountpoint-s3-crt-sys/crt/aws-c-common 7a6f5df2..0e7637fa:
  > A bunch of CMake fixes (awslabs#1178)
  > Fix heap overflow on uri parsing (awslabs#1185)
  > (take 2) Detect when AVX is disabled via OSXSAVE (awslabs#1184)
  > Fixup IPv6 validation logic (awslabs#1180)
  > Detect when AVX is disabled via OSXSAVE (awslabs#1182)
  > proof_ci.yaml must use latest upload-artifact (awslabs#1183)
  > change PR template to ask for clearer wording (awslabs#1177)
Submodule mountpoint-s3-crt-sys/crt/aws-c-compression c6c1191e..f951ab2b:
  > A bunch of CMake fixes (awslabs#72)
  > Switch CI to use roles (awslabs#71)
  > chore: Modified bug issue template to add checkbox to report potential regression. (awslabs#69)
Submodule mountpoint-s3-crt-sys/crt/aws-c-http fc3eded2..590c7b59:
  > A bunch of CMake fixes (awslabs#497)
  > Fix CI for GCC-13 on Ubuntu-18  (awslabs#496)
  > Switch CI to use roles (awslabs#494)
Submodule mountpoint-s3-crt-sys/crt/aws-c-io fcb38c80..3041dabf:
  > A bunch of CMake fixes (awslabs#701)
  > Event Loop & Socket Type Multi-Support (awslabs#692)
  > fix typo in log message (awslabs#702)
  > Fix CI for GCC-13 on Ubuntu-18 (awslabs#700)
  > Switch CI to use roles (awslabs#698)
Submodule mountpoint-s3-crt-sys/crt/aws-c-s3 a3b401bf..6eb8be53:
  > A bunch of CMake fixes (awslabs#480)
  > S3Express CreateSession Allowlist Headers (awslabs#492)
  > Auto - Update S3 Ruleset & Partition (awslabs#491)
Submodule mountpoint-s3-crt-sys/crt/aws-c-sdkutils 1ae8664f..ba6a28fa:
  > A bunch of CMake fixes (awslabs#50)
Submodule mountpoint-s3-crt-sys/crt/aws-checksums 3e4101b9..fb8bd0b8:
  > A bunch of CMake fixes (awslabs#101)
  > Switch CI to use roles (awslabs#100)
Submodule mountpoint-s3-crt-sys/crt/aws-lc ffd6fb71..138a6ad3:
  > Prepare AWS-LC v1.44.0 (#2153)
  > Fix issue with ML-DSA key parsing (#2152)
  > Add support for PKCS7_set/get_detached (#2134)
  > Prepare Docker image for CI integration jobs (#2126)
  > Delete OpenVPN mainline patch from our integration build (#2149)
  > SHA3/SHAKE Init Updates via FIPS202 API layer (#2101)
  > Support keypair calculation for PQDSA PKEY (#2145)
  > Optimize x86/aarch64 MD5 implementation (#2137)
  > Check for MIPSEB in target.h (#2143)
  > Ed25519ph and Ed25519ctx Support (#2120)
  > Support for ML-DSA public key generation from private key (#2142)
  > Avoid mixing SSE and AVX in XTS-mode AVX512 implementation (#2140)
  > Remove remaining support for Trusty and Fuchsia operating systems (#2136)
  > ACVP test harness for ML-DSA (#2127)
  > Minor symbols to work with Ruby's mainline (#2132)

Signed-off-by: Alessandro Passaro <[email protected]>
github-merge-queue bot pushed a commit that referenced this pull request Feb 5, 2025
Update the CRT libraries to the latest releases. In particular, include:
* S3Express CreateSession Allowlist Headers
([awslabs/aws-c-s3#492](awslabs/aws-c-s3#492))

<details>
  <summary>Full CRT changelog:</summary>
  
```
Submodule mountpoint-s3-crt-sys/crt/aws-c-auth 5bc67797..b513db4b:
  > A bunch of CMake fixes (#258)
  > Add Account Id to Credentials (#260)
  > Skip Transfer-Encoding from signing (#261)
Submodule mountpoint-s3-crt-sys/crt/aws-c-cal fbbe2612..7299c6ab:
  > Fix Findcrypto.cmake (#205)
  > A bunch of CMake fixes (#203)
  > Switch CI to use roles (#202)
Submodule mountpoint-s3-crt-sys/crt/aws-c-common 7a6f5df2..0e7637fa:
  > A bunch of CMake fixes (#1178)
  > Fix heap overflow on uri parsing (#1185)
  > (take 2) Detect when AVX is disabled via OSXSAVE (#1184)
  > Fixup IPv6 validation logic (#1180)
  > Detect when AVX is disabled via OSXSAVE (#1182)
  > proof_ci.yaml must use latest upload-artifact (#1183)
  > change PR template to ask for clearer wording (#1177)
Submodule mountpoint-s3-crt-sys/crt/aws-c-compression c6c1191e..f951ab2b:
  > A bunch of CMake fixes (#72)
  > Switch CI to use roles (#71)
  > chore: Modified bug issue template to add checkbox to report potential regression. (#69)
Submodule mountpoint-s3-crt-sys/crt/aws-c-http fc3eded2..590c7b59:
  > A bunch of CMake fixes (#497)
  > Fix CI for GCC-13 on Ubuntu-18  (#496)
  > Switch CI to use roles (#494)
Submodule mountpoint-s3-crt-sys/crt/aws-c-io fcb38c80..3041dabf:
  > A bunch of CMake fixes (#701)
  > Event Loop & Socket Type Multi-Support (#692)
  > fix typo in log message (#702)
  > Fix CI for GCC-13 on Ubuntu-18 (#700)
  > Switch CI to use roles (#698)
Submodule mountpoint-s3-crt-sys/crt/aws-c-s3 a3b401bf..6eb8be53:
  > A bunch of CMake fixes (#480)
  > S3Express CreateSession Allowlist Headers (#492)
  > Auto - Update S3 Ruleset & Partition (#491)
Submodule mountpoint-s3-crt-sys/crt/aws-c-sdkutils 1ae8664f..ba6a28fa:
  > A bunch of CMake fixes (#50)
Submodule mountpoint-s3-crt-sys/crt/aws-checksums 3e4101b9..fb8bd0b8:
  > A bunch of CMake fixes (#101)
  > Switch CI to use roles (#100)
Submodule mountpoint-s3-crt-sys/crt/aws-lc ffd6fb71..138a6ad3:
  > Prepare AWS-LC v1.44.0 (#2153)
  > Fix issue with ML-DSA key parsing (#2152)
  > Add support for PKCS7_set/get_detached (#2134)
  > Prepare Docker image for CI integration jobs (#2126)
  > Delete OpenVPN mainline patch from our integration build (#2149)
  > SHA3/SHAKE Init Updates via FIPS202 API layer (#2101)
  > Support keypair calculation for PQDSA PKEY (#2145)
  > Optimize x86/aarch64 MD5 implementation (#2137)
  > Check for MIPSEB in target.h (#2143)
  > Ed25519ph and Ed25519ctx Support (#2120)
  > Support for ML-DSA public key generation from private key (#2142)
  > Avoid mixing SSE and AVX in XTS-mode AVX512 implementation (#2140)
  > Remove remaining support for Trusty and Fuchsia operating systems (#2136)
  > ACVP test harness for ML-DSA (#2127)
  > Minor symbols to work with Ruby's mainline (#2132)
```
</details>


### Does this change impact existing behavior?

No.

### Does this change need a changelog entry? Does it require a version
change?

No.

---

By submitting this pull request, I confirm that my contribution is made
under the terms of the Apache 2.0 license and I agree to the terms of
the [Developer Certificate of Origin
(DCO)](https://developercertificate.org/).

Signed-off-by: Alessandro Passaro <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
performance PRs to run benchmarks on
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants