Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: Support writing Time type in json #21454

Merged
merged 1 commit into from
Feb 26, 2025
Merged

Conversation

mcrumiller
Copy link
Contributor

@mcrumiller mcrumiller commented Feb 25, 2025

Closes #21452.

I also removed two small branches that aren't relevant to Polars:

  1. ArrowDataType::Timestamp(s) - this is our Datetime dtype, which only uses ms, us, and ns, we have no s implementation.
  2. ArrowDataType::Duration(s) - same as Datetime units described above.

Copy link

codecov bot commented Feb 25, 2025

Codecov Report

Attention: Patch coverage is 87.09677% with 4 lines in your changes missing coverage. Please review.

Project coverage is 79.99%. Comparing base (5b48926) to head (e8821c6).
Report is 17 commits behind head on main.

Files with missing lines Patch % Lines
crates/polars-json/src/json/write/serialize.rs 87.09% 4 Missing ⚠️
Additional details and impacted files
@@            Coverage Diff             @@
##             main   #21454      +/-   ##
==========================================
+ Coverage   79.96%   79.99%   +0.02%     
==========================================
  Files        1597     1598       +1     
  Lines      229115   229228     +113     
  Branches     2618     2620       +2     
==========================================
+ Hits       183222   183381     +159     
+ Misses      45294    45248      -46     
  Partials      599      599              

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

@mcrumiller mcrumiller changed the title feat(rust, python): Support writing Time type in json feat: Support writing Time type in json Feb 25, 2025
@github-actions github-actions bot added enhancement New feature or an improvement of an existing feature python Related to Python Polars rust Related to Rust Polars and removed title needs formatting labels Feb 25, 2025
@mcrumiller mcrumiller marked this pull request as ready for review February 25, 2025 03:44
@@ -514,6 +537,33 @@ pub(crate) fn new_serializer<'a>(
take,
)
},
ArrowDataType::Time32(tu) => {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We only use one of time32 / time64 in Polars. (I don't know on top of head which one, but we only should implement and compile one)(

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks, I wasn't sure of this since the Duration writer above has branches for time units that we don't use. I'll remove time32.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks, I wasn't sure of this since the Duration writer above has branches for time units that we don't use.

Ah, then we can remove those as well. :) Can you do that?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yup. Drafting because I'm at work so no local CI so I'm relying on GitHub to help me lint/format.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ok yeah this will have to wait until tonight.

@mcrumiller mcrumiller marked this pull request as draft February 25, 2025 13:26
@mcrumiller mcrumiller marked this pull request as ready for review February 25, 2025 22:59
@mcrumiller
Copy link
Contributor Author

@ritchie46 I noticed that we have a few other functions in polars-arrow that correspond to dtype+time unit combinations that polars doesn't support in other areas unrelated to json. I wonder if these were included for potential future support, e.g. Datetime(s). I can maybe remove those in another PR?

@ritchie46
Copy link
Member

I wonder if these were included for potential future support, e.g. Datetime(s). I can maybe remove those in another PR?

No, that's what's leftover from arrow2. We don't need it in Polars, so it can be removed.

@ritchie46 ritchie46 merged commit 841cecb into pola-rs:main Feb 26, 2025
28 checks passed
@mcrumiller mcrumiller deleted the json-time branch February 26, 2025 15:41
anath2 pushed a commit to anath2/polars that referenced this pull request Mar 5, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or an improvement of an existing feature python Related to Python Polars rust Related to Rust Polars
Projects
None yet
Development

Successfully merging this pull request may close these issues.

PanicException: not yet implemented: Writing Time64(Nanosecond) to JSON
2 participants