Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat(rust): Allow setting custom client options #21007

Open
wants to merge 1 commit into
base: main
Choose a base branch
from

Conversation

PrettyWood
Copy link

@PrettyWood PrettyWood commented Jan 30, 2025

We currently use Polars in our Rust web service and need to support custom certificates in our file connector.
While this functionality already exists in object_store via the ClientOptions struct (which allows setting root certificates through the with_root_certificate method), Polars does not yet expose this struct.
This PR aims to add this capability to Polars.

Polars CloudOptions implements PartialEq, Eq and Hash, which is not the case of ClientOptions in object_store. Furthermore the defaults in polars for ClientOptions were not the same as object_store
(see get_client_options function)

So I decided to create a new type PlClientOptions with those defaults and only add what we want to set: the 3 previous fields (pure refactoring) + certificates (feature)

Some questions:

  • is the overall approach good for you?
  • is it ok if we ignore those certificates for PartialEq, Eq and Hash? If yes do you prefer the current approach with custom implems or the use of derivative crate with #[derivative(PartialEq="ignore")], which is more explicit and readable but adds a new dependency.
  • should I simplify #[cfg(any(feature = "aws", feature = "gcp", feature = "azure", feature = "http"))] with #[cfg(feature = "cloud")]?

We can test this PR on AWS but not for GCP and Azure but since the code is the same, this test should be enough!

Thank you so much :)

@PrettyWood PrettyWood force-pushed the feat/custom-client-options branch from d6de45d to e71cff5 Compare January 30, 2025 14:25
@PrettyWood PrettyWood changed the title feat(rust): allow setting custom client options feat(rust): Allow setting custom client options Jan 30, 2025
@github-actions github-actions bot added enhancement New feature or an improvement of an existing feature rust Related to Rust Polars and removed title needs formatting labels Jan 30, 2025
@PrettyWood PrettyWood force-pushed the feat/custom-client-options branch 2 times, most recently from 26b7468 to ad25eb8 Compare January 30, 2025 15:56
@PrettyWood PrettyWood force-pushed the feat/custom-client-options branch from ad25eb8 to b2ac3e9 Compare January 30, 2025 16:25
@PrettyWood PrettyWood marked this pull request as ready for review January 30, 2025 16:31
Copy link

codecov bot commented Jan 30, 2025

Codecov Report

Attention: Patch coverage is 64.15094% with 19 lines in your changes missing coverage. Please review.

Project coverage is 79.20%. Comparing base (96a2d01) to head (b2ac3e9).
Report is 23 commits behind head on main.

Files with missing lines Patch % Lines
crates/polars-io/src/cloud/options.rs 64.15% 19 Missing ⚠️
Additional details and impacted files
@@            Coverage Diff             @@
##             main   #21007      +/-   ##
==========================================
- Coverage   79.34%   79.20%   -0.14%     
==========================================
  Files        1579     1583       +4     
  Lines      224319   225144     +825     
  Branches     2573     2581       +8     
==========================================
+ Hits       177976   178321     +345     
- Misses      45755    46233     +478     
- Partials      588      590       +2     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or an improvement of an existing feature rust Related to Rust Polars
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant